Package org.omegat.tokenizer

Class LuceneGreekTokenizer

java.lang.Object

org.omegat.tokenizer.BaseTokenizer

org.omegat.tokenizer.LuceneGreekTokenizer

All Implemented Interfaces:

ITokenizer

public class LuceneGreekTokenizer
extends BaseTokenizer

A tokenizer specifically designed for tokenizing Greek text. This class leverages the capabilities of the Lucene library to provide tokenization based on the GreekAnalyzer. It supports the inclusion or exclusion of stemming and stop words depending on the specified configuration.

The class extends the BaseTokenizer, overriding functionality for token stream generation to address Greek language processing requirements.

Nested Class Summary

Nested classes/interfaces inherited from interface org.omegat.tokenizer.ITokenizer

ITokenizer.StemmingMode

Field Summary

Fields inherited from class org.omegat.tokenizer.BaseTokenizer

TOKENIZER_DEBUG_PROVIDER

Constructor Summary

Constructors

Constructor	Description
LuceneGreekTokenizer()

Method Summary

Methods inherited from class org.omegat.tokenizer.BaseTokenizer

getSupportedLanguages, tokenizeVerbatim, tokenizeVerbatimToStrings, tokenizeWords, tokenizeWordsToStrings

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LuceneGreekTokenizer

public LuceneGreekTokenizer()