public abstract class Tokenizer extends java.lang.Object implements java.util.Enumeration, OptionHandler, java.io.Serializable, RevisionHandler
Constructor and Description |
---|
Tokenizer() |
Modifier and Type | Method and Description |
---|---|
java.lang.String[] |
getOptions()
Gets the current option settings for the OptionHandler.
|
abstract java.lang.String |
globalInfo()
Returns a string describing the stemmer
|
abstract boolean |
hasMoreElements()
Tests if this enumeration contains more elements.
|
java.util.Enumeration |
listOptions()
Returns an enumeration of all the available options..
|
abstract java.lang.Object |
nextElement()
Returns the next element of this enumeration if this enumeration object
has at least one more element to provide.
|
static void |
runTokenizer(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the
tokenizer over all the remaining strings in the options array.
|
void |
setOptions(java.lang.String[] options)
Sets the OptionHandler's options using the given list.
|
abstract void |
tokenize(java.lang.String s)
Sets the string to tokenize.
|
static java.lang.String[] |
tokenize(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the
tokenizer over all the remaining strings in the options array.
|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getRevision
public abstract java.lang.String globalInfo()
public java.util.Enumeration listOptions()
listOptions
in interface OptionHandler
public java.lang.String[] getOptions()
getOptions
in interface OptionHandler
public void setOptions(java.lang.String[] options) throws java.lang.Exception
setOptions
in interface OptionHandler
options
- the list of options as an array of stringsjava.lang.Exception
- if an option is not supportedpublic abstract boolean hasMoreElements()
hasMoreElements
in interface java.util.Enumeration
public abstract java.lang.Object nextElement()
nextElement
in interface java.util.Enumeration
public abstract void tokenize(java.lang.String s)
s
- the string to tokenizepublic static java.lang.String[] tokenize(Tokenizer tokenizer, java.lang.String[] options) throws java.lang.Exception
tokenizer
- the tokenizer to useoptions
- the options for the tokenizerjava.lang.Exception
- if setting of options or tokenization failspublic static void runTokenizer(Tokenizer tokenizer, java.lang.String[] options)
tokenizer
- the tokenizer to useoptions
- the options for the tokenizer