Package org.omegat.tokenizer

Class LuceneCatalanTokenizer

java.lang.Object

org.omegat.tokenizer.BaseTokenizer

org.omegat.tokenizer.LuceneCatalanTokenizer

All Implemented Interfaces:

ITokenizer

public class LuceneCatalanTokenizer
extends BaseTokenizer

Tokenizer implementation for processing Catalan language text. This tokenizer utilizes Lucene's CatalanAnalyzer to provide language-specific tokenization and optional stemming and stop word filtering.

The tokenizer can process text while either including or excluding stems and stop words based on the specified configuration.

Nested Class Summary

Nested classes/interfaces inherited from interface org.omegat.tokenizer.ITokenizer

ITokenizer.StemmingMode

Field Summary

Fields inherited from class org.omegat.tokenizer.BaseTokenizer

TOKENIZER_DEBUG_PROVIDER

Constructor Summary

Constructors

Constructor	Description
LuceneCatalanTokenizer()

Method Summary

Methods inherited from class org.omegat.tokenizer.BaseTokenizer

getSupportedLanguages, tokenizeVerbatim, tokenizeVerbatimToStrings, tokenizeWords, tokenizeWordsToStrings

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LuceneCatalanTokenizer

public LuceneCatalanTokenizer()