Gints Jēkabsons

IR & NLP

Benchmarking 24 Large Language Models for Automated Multiple-Choice Question Generation in Latvian

Paper & GitHub repo (Daupare & Jēkabsons, 2025)

The study investigates the generation of Multiple-Choice Questions in Latvian to assess both the ability of LLMs to generate high-quality questions and answers and, more broadly, their capability to process Latvian, a lower-resourced language that has received relatively little attention in LLM research. The study benchmarks 24 different LLMs, specifically those developed by Anthropic, DeepSeek, OpenAI, Google, Meta, Mistral, and Microsoft. The findings highlight the varying capabilities of these models in handling Latvian, producing grammatically correct, coherent, and meaningful text.

Comparison of Language Models for English-Latvian Semantic Search

Paper & GitHub repo (Kucheravy & Jēkabsons, 2025)

The study evaluates and compares ten transformer models in an English-Latvian semantic information retrieval setting, where the indexed collection of documents is written in English while the query documents are written in Latvian. A dataset of 77736 pairs of articles from Latvian and English Wikipedia was created, transformed into embedding vectors, and used for retrieval experiments with brute force search, Hierarchical Navigable Small World method, and Inverted File Indexing method.

Evaluation of Fingerprint Selection Algorithms for Two-Stage Plagiarism Detection (Java code)

Version 1.0 (December 30, 2021) - download (GNU LGPL license)

This software was used for the experiments in the paper "Evaluation of Fingerprint Selection Algorithms for Two-Stage Plagiarism Detection" (also available from the publisher here).

This software is developed for evaluating the effectiveness of fingerprint selection algorithms for a two-stage (source retrieval + aligning) local text reuse detection. It implements Full fingerprinting, Every p-th, 0 mod p, Winnowing, Hailstorm, Frequency-Biased Winnowing (FBW), and Modified Frequency-Biased Winnowing (MFBW) (see the paper above or the paper here for details). Indexing of the fingerprints is implemented using the Apache Lucene library.

Fingerprint selection algorithms for local text reuse detection (Java code)

Version 1.0 (June 5, 2020) - download (GNU LGPL license)

This software was used for the experiments in the paper "Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection" (also available from the publisher here).

This software is developed for evaluating the effectiveness of fingerprint selection algorithms for the source retrieval stage of a local text reuse detection system. It implements the following fingerprint selection algorithms (see the paper for details):

• Full fingerprinting;
• Every p-th;
• 0 mod p;
• Winnowing;
• Hailstorm;
• Frequency-Biased Winnowing (FBW);
• Modified Frequency-Biased Winnowing (MFBW) - proposed in the paper.

Local text reuse occurs when parts of a document, such as a paragraph or a sentence, are reused in another document. The reused text may also be modified by inserting, removing, replacing, or rearranging words or sentences, as well as interleaving text from one source with a text from another source. Detection of such reuse is central to a variety of applications, including plagiarism detection, origin detection, and information flow analysis.

StemmerLV: Latvian lemmatizer and stemmer for Java

Version 1.2 (November 6, 2020) - download (GNU LGPL license)

StemmerLV uses Hunspell affix and dictionary files created by Jānis Eisaks (available here).

StemmerLV was initially developed as a pure Java substitute for the HunspellJNA library to do lemmatization for Latvian but quickly got some additional functionality.

What StemmerLV can do:

• Lemmatize a word according to the affix and dictionary files. The result is the same as with HunspellJNA (but unfortunately it works on average about 15% slower).
• Save time by returning as soon as the first lemma is found. In this mode it works on average almost 3 times faster than HunspellJNA but never returns more than one lemma.
• Stem a word by either finding or guessing its lemma and then stemming the lemma. Lemma guessing allows finding consistent short stems for unknown words that are not included in the dictionary.
• List all word forms for a given lemma.
• List all lemmas included in the dictionary together with all their word forms.

How StemmerLV does stemming:

1. Uses the affix file to generate lemma candidates for a given word.
2. Checks if any of the lemma candidates exist in the dictionary. If at least one candidate is there, discards all the candidates that are not there. If none of candidates are there and guessing is disabled, just returns the original word.
3. If none of the lemmas exist in the dictionary, filters out those with weird unnatural endings but keeps all the rest of the candidates as guesses for the lemma. There can be up to about 20 different guesses. (This step is skipped if guessing is disabled.)
4. Stems all lemmas using four simple character removal rules designed specifically for stemming Latvian lemmas and returns the shortest stem.

To use any of the functionality of StemmerLV, add the .jar file to your project, create a StemmerLV object, and see the list of available functions - their names are pretty self-explanatory. Source code is included in the .jar file.

EN/ET/LT/LV/RU machine-readable dictionaries extracted from Wiktionary

download

Machine-readable English-Estonian, English-Lithuanian, English-Latvian, English-Russian, Lithuanian-Estonian, Lithuanian-Russian, Latvian-Estonian, Latvian-Lithuanian, Latvian-Russian, and Russian-Estonian dictionaries extracted from English Wiktionary dump 20210720.

The dictionaries are licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License (“CC BY-SA”).