org.omegat.tokenizer

Class LuceneRomanianTokenizer

java.lang.Object

org.omegat.tokenizer.BaseTokenizer

org.omegat.tokenizer.LuceneRomanianTokenizer

All Implemented Interfaces:

ITokenizer

public class LuceneRomanianTokenizer
extends BaseTokenizer

Nested Class Summary

Nested classes/interfaces inherited from interface org.omegat.tokenizer.ITokenizer

ITokenizer.StemmingMode

Field Summary

Fields inherited from class org.omegat.tokenizer.BaseTokenizer

DEFAULT_TOKENS_COUNT, EMPTY_STRING_LIST, EMPTY_TOKENS_LIST, shouldDelegateTokenizeExactly, TOKENIZER_DEBUG_PROVIDER

Constructor Summary

Constructors

Constructor and Description
LuceneRomanianTokenizer()

Method Summary

Modifier and Type	Method and Description
protected org.apache.lucene.analysis.TokenStream	getTokenStream(java.lang.String strOrig, boolean stemsAllowed, boolean stopWordsAllowed)

Methods inherited from class org.omegat.tokenizer.BaseTokenizer

getEffectiveLanguage, getProjectLanguage, getStandardTokenStream, getSupportedLanguages, printTest, test, tokenize, tokenizeByCodePoint, tokenizeByCodePointToStrings, tokenizeToStrings, tokenizeVerbatim, tokenizeVerbatimToStrings, tokenizeWords, tokenizeWordsToStrings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LuceneRomanianTokenizer

public LuceneRomanianTokenizer()

Method Detail

getTokenStream

protected org.apache.lucene.analysis.TokenStream getTokenStream(java.lang.String strOrig,
                                                                boolean stemsAllowed,
                                                                boolean stopWordsAllowed)
                                                         throws java.io.IOException

Specified by:

getTokenStream in class BaseTokenizer

Throws:

java.io.IOException