org.omegat.tokenizer

Annotation Type Tokenizer

@Retention(value=RUNTIME)
public @interface Tokenizer

Annotation to indicate the languages for which a tokenizer is intended for use.

Field Summary

Fields

Modifier and Type	Fields and Description
static java.lang.String	DISCOVER_AT_RUNTIME Value for the languages() member that indicates that the supported languages should be determined at runtime via BaseTokenizer.getSupportedLanguages().

Required Element Summary

Required Elements

Modifier and Type	Required Element and Description
java.lang.String[]	languages The languages supported by the tokenizer.

Optional Element Summary

Optional Elements

Modifier and Type	Optional Element and Description
boolean	isDefault When multiple tokenizers support the same language, this indicates the one that should be preferred above the others.

Field Detail

DISCOVER_AT_RUNTIME

public static final java.lang.String DISCOVER_AT_RUNTIME

Value for the languages() member that indicates that the supported languages should be determined at runtime via BaseTokenizer.getSupportedLanguages().

Element Detail

languages

public abstract java.lang.String[] languages

The languages supported by the tokenizer. E.g. LuceneCJKTokenizer supports { zh, ja, ko }.

isDefault

public abstract boolean isDefault

When multiple tokenizers support the same language, this indicates the one that should be preferred above the others.

Default:

false