Package org.omegat.tokenizer

Class LuceneBulgarianTokenizer

java.lang.Object

org.omegat.tokenizer.BaseTokenizer

org.omegat.tokenizer.LuceneBulgarianTokenizer

All Implemented Interfaces:

ITokenizer

public class LuceneBulgarianTokenizer
extends BaseTokenizer

The LuceneBulgarianTokenizer class is a tokenizer implementation designed for processing Bulgarian text. It extends the BaseTokenizer class and provides customized behavior for tokenization based on the specified properties such as stemming and stop words filtering.

This class utilizes the Lucene BulgarianAnalyzer to perform tokenization and optionally applies stemming and stop word removal based on the configuration.

Nested Class Summary

Nested classes/interfaces inherited from interface org.omegat.tokenizer.ITokenizer

ITokenizer.StemmingMode

Field Summary

Fields inherited from class org.omegat.tokenizer.BaseTokenizer

TOKENIZER_DEBUG_PROVIDER

Constructor Summary

Constructors

Constructor	Description
LuceneBulgarianTokenizer()

Method Summary

Methods inherited from class org.omegat.tokenizer.BaseTokenizer

getSupportedLanguages, tokenizeVerbatim, tokenizeVerbatimToStrings, tokenizeWords, tokenizeWordsToStrings

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LuceneBulgarianTokenizer

public LuceneBulgarianTokenizer()