Package org.omegat.tokenizer

Class LuceneItalianTokenizer

java.lang.Object

org.omegat.tokenizer.BaseTokenizer

org.omegat.tokenizer.LuceneItalianTokenizer

All Implemented Interfaces:

ITokenizer

public class LuceneItalianTokenizer
extends BaseTokenizer

Tokenizer implementation for processing Italian language text using Apache Lucene. This class provides functionality for creating token streams tailored for the Italian language, supporting options for stemming and stop word filtering.

The tokenizer applies Italian-specific text processing based on the Apache Lucene library, utilizing the ItalianAnalyzer when stemming is enabled. When stemming is disabled, a standard token stream is used.

Nested Class Summary

Nested classes/interfaces inherited from interface org.omegat.tokenizer.ITokenizer

ITokenizer.StemmingMode

Field Summary

Fields inherited from class org.omegat.tokenizer.BaseTokenizer

TOKENIZER_DEBUG_PROVIDER

Constructor Summary

Constructors

Constructor	Description
LuceneItalianTokenizer()

Method Summary

Methods inherited from class org.omegat.tokenizer.BaseTokenizer

getSupportedLanguages, tokenizeVerbatim, tokenizeVerbatimToStrings, tokenizeWords, tokenizeWordsToStrings

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LuceneItalianTokenizer

public LuceneItalianTokenizer()