| Class | Description |
|---|---|
| AlphabeticTokenizer |
Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.
|
| CharacterDelimitedTokenizer |
Abstract superclass for tokenizers that take characters as delimiters.
|
| NGramTokenizer |
Splits a string into an n-gram with min and max
grams.
|
| Tokenizer |
A superclass for all tokenizer algorithms.
|
| WordTokenizer |
A simple tokenizer that is using the java.util.StringTokenizer class to tokenize the strings.
|