WekaClassifierMapTask

java.lang.Object
- weka.distributed.WekaClassifierMapTask

All Implemented Interfaces:

java.io.Serializable, weka.core.EnvironmentHandler, weka.core.OptionHandler
```
public class WekaClassifierMapTask
extends java.lang.Object
implements weka.core.OptionHandler, weka.core.EnvironmentHandler, java.io.Serializable
```
A map task for building classifiers. Can handle batch and incremental classifiers, which are either Aggregateable or not. Non-aggregateable classifiers are wrapped up in a Vote meta classifeir by the reduce task. Incremental classifiers are trained as instances are presented to the processInstance() method. Batch classifiers are trained when finalizeTask() is called. Instances are collected and held in memory for batch classifiers, although reservior sampling may be used to ensure that a fixed number of instances is used for batch learning. There are options to force batch learning for updateable classifiers and to force the generation of a Vote ensemble for Aggregateable classifiers.
Classifiers may be trained on all the incoming data or on a particular cross-validation fold (this functionality is used directly by the evaluation map and reduce tasks). In the case of batch classifiers, the data for the map will be stratified (if the class is nominal) and randomized before extracting the fold to train on. In the case of incremental classifiers, a modulus operation is used to pull out the instance corresponding to the selected fold from the incoming instance stream.
Classifiers can optionally have their training data passed through one or more filters as a pre-processing step. The class will determine how to wrap the base classifier and filters based on the nature of the filters specified and whether the classifier is batch/incremental and Aggregateable. Aggregateable classifiers (batch or incremental) can only be aggregated to one final model if the filters used with them (if using filters) are all StreamableFilters (i.e. they can determine their output structure immediately without having to see any instances).
It is also possible to specify a special "preconstructed" filter to use in conjunction with, or instead of, regular filters. At present, there is just one Preconstructed filter implemented by the distributed system. PreConstructedPCA can produce a "trained" PCA filter using a correlation matrix produced by the CorrelationMatrixMap/Reduce tasks.

Version:

$Revision: 12589 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`static java.lang.String`	`TOTAL_NUMBER_OF_MAPS` If this property is set then we can adjust the total number of requested iterations for IteratedSingleClassifierEnhancers according to the number of maps that are going to run.

Constructor Summary

Constructors
Constructor and Description

WekaClassifierMapTask()

Constructors
Constructor and Description
`WekaClassifierMapTask()`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addPreconstructedFilterToUse(PreconstructedFilter f)` Add a Preconstructed filter (such as PreConstructedPCA) to use with the classifier.
`void`	`addToTrainingHeader(weka.core.Instance toAdd)` Add the supplied instance to the training header
`void`	`addToTrainingHeader(weka.core.Instances toAdd)` Add the supplied instances to the training header
`java.lang.String`	`classifierTipText()` The tool tip text for this property.
`java.lang.String`	`continueTrainingUpdateableClassifierTipText()` The tool tip text for this property.
`java.lang.String`	`filtersToUseTipText()` The tool tip text for this property.
`void`	`finalizeTask()` Finish up the map task.
`java.lang.String`	`foldNumberTipText()` The tool tip text for this property.
`java.lang.String`	`forceBatchLearningForUpdateableClassifiersTipText()` The tool tip text for this property.
`java.lang.String`	`forceVotedEnsembleCreation()` The tool tip text for this property.
`weka.classifiers.Classifier`	`getClassifier()` Get the classifier to use
`boolean`	`getContinueTrainingUpdateableClassifier()` Get whether to continue training an incremental (updateable) classifier.
`weka.filters.Filter[]`	`getFiltersToUse()` Get the filters to wrap up with the base classifier
`int`	`getFoldNumber()` Get the fold number to train the classifier with.
`boolean`	`getForceBatchLearningForUpdateableClassifiers()` Get whether to force batch training for incremental (Updateable) classifiers
`boolean`	`getForceVotedEnsembleCreation()` Get whether to force the creation of a Vote ensemble for Aggregateable classifiers
`int`	`getNumTrainingInstances()` Get the number of training instances actually used to train the classifier.
`java.lang.String[]`	`getOptions()`
`int`	`getReservoirSampleSize()` Get the sample size for reservoir sampling
`java.lang.String`	`getSeed()` Get the seed for randomizing the data when batch learning and for reservoir sampling.
`int`	`getTotalNumFolds()` Get the total number of folds to use.
`boolean`	`getUseReservoirSamplingWhenBatchLearning()` Get whether to use reservoir sampling when batch learning
`java.util.Enumeration<weka.core.Option>`	`listOptions()`
`static void`	`main(java.lang.String[] args)`
`void`	`processInstance(weka.core.Instance inst)` Process the supplied instance.
`java.lang.String`	`reservoirSampleSizeTipText()` The tool tip text for this property.
`java.lang.String`	`seedTipText()` The tool tip text for this property.
`void`	`setClassifier(weka.classifiers.Classifier classifier)` Set the classifier to use
`void`	`setContinueTrainingUpdateableClassifier(boolean u)` Set whether to continue training an incremental (updateable) classifier.
`void`	`setEnvironment(weka.core.Environment env)`
`void`	`setFiltersToUse(weka.filters.Filter[] toUse)` Set the filters to wrap up with the base classifier
`void`	`setFoldNumber(int fn)` Set the fold number to train the classifier with.
`void`	`setForceBatchLearningForUpdateableClassifiers(boolean force)` Set whether to force batch training for incremental (Updateable) classifiers
`void`	`setForceVotedEnsembleCreation(boolean f)` Set whether to force the creation of a Vote ensemble for Aggregateable classifiers
`void`	`setOptions(java.lang.String[] options)`
`void`	`setReservoirSampleSize(int size)` Set the sample size for reservoir sampling
`void`	`setSeed(java.lang.String seed)` Set the seed for randomizing the data when batch learning and for reservoir sampling.
`void`	`setTotalNumFolds(int t)` Set the total number of folds to use.
`void`	`setup(weka.core.Instances trainingHeader)` Initialize the map task
`void`	`setUseReservoirSamplingWhenBatchLearning(boolean r)` Set whether to use reservoir sampling when batch learning
`java.lang.String`	`totalNumberOfFoldsTipText()` The tool tip text for this property.
`java.lang.String`	`useReservoirSamplingWhenBatchLearningTipText()` The tool tip text for this property.

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface weka.core.OptionHandler
makeCopy

- Field Detail
  - TOTAL_NUMBER_OF_MAPS
```
public static final java.lang.String TOTAL_NUMBER_OF_MAPS
```
    If this property is set then we can adjust the total number of requested iterations for IteratedSingleClassifierEnhancers according to the number of maps that are going to run. This is useful for schemes that build independent base models (e.g. Bagging) in order to get approximately the requested number of models in the end. For boosting methods it will be necessary to set their number of iterations to a higher value than required as it will be adjusted downwards for each map.
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - WekaClassifierMapTask
```
public WekaClassifierMapTask()
```
- Method Detail
  - main
```
public static void main(java.lang.String[] args)
```
  - listOptions
```
public java.util.Enumeration<weka.core.Option> listOptions()
```
    Specified by:
    
    listOptions in interface weka.core.OptionHandler
  - getOptions
```
public java.lang.String[] getOptions()
```
    Specified by:
    
    getOptions in interface weka.core.OptionHandler
  - setOptions
```
public void setOptions(java.lang.String[] options)
                throws java.lang.Exception
```
    Specified by:
    
    setOptions in interface weka.core.OptionHandler
    
    Throws:
    
    java.lang.Exception
  - getClassifier
```
public weka.classifiers.Classifier getClassifier()
```
    Get the classifier to use
    
    Returns:
    
    the classifier to use
  - setClassifier
```
public void setClassifier(weka.classifiers.Classifier classifier)
```
    Set the classifier to use
    
    Parameters:
    
    classifier - the classifier to use
  - classifierTipText
```
public java.lang.String classifierTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getSeed
```
public java.lang.String getSeed()
```
    Get the seed for randomizing the data when batch learning and for reservoir sampling.
    
    Returns:
    
    the seed to use
  - setSeed
```
public void setSeed(java.lang.String seed)
```
    Set the seed for randomizing the data when batch learning and for reservoir sampling.
    
    Parameters:
    
    seed - the seed to use
  - seedTipText
```
public java.lang.String seedTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getForceVotedEnsembleCreation
```
public boolean getForceVotedEnsembleCreation()
```
    Get whether to force the creation of a Vote ensemble for Aggregateable classifiers
    
    Returns:
    
    true if a Vote ensemble is to be created even in the case where the base classifier is directly aggregateable
  - setForceVotedEnsembleCreation
```
public void setForceVotedEnsembleCreation(boolean f)
```
    Set whether to force the creation of a Vote ensemble for Aggregateable classifiers
    
    Parameters:
    
    f - true if a Vote ensemble is to be created even in the case where the base classifier is directly aggregateable
  - forceVotedEnsembleCreation
```
public java.lang.String forceVotedEnsembleCreation()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getFiltersToUse
```
public weka.filters.Filter[] getFiltersToUse()
```
    Get the filters to wrap up with the base classifier
    
    Returns:
    
    the filters to wrap up with the base classifier
  - setFiltersToUse
```
public void setFiltersToUse(weka.filters.Filter[] toUse)
```
    Set the filters to wrap up with the base classifier
    
    Parameters:
    
    toUse - filters to wrap up with the base classifier
  - filtersToUseTipText
```
public java.lang.String filtersToUseTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - addPreconstructedFilterToUse
```
public void addPreconstructedFilterToUse(PreconstructedFilter f)
```
    Add a Preconstructed filter (such as PreConstructedPCA) to use with the classifier. This can be used instead of, or in conjunction with, a set of standard filters.
    
    Parameters:
    
    f - the Preconstructed filter to use.
  - getUseReservoirSamplingWhenBatchLearning
```
public boolean getUseReservoirSamplingWhenBatchLearning()
```
    Get whether to use reservoir sampling when batch learning
    
    Returns:
    
    true if reservoir sampling is to be used
  - setUseReservoirSamplingWhenBatchLearning
```
public void setUseReservoirSamplingWhenBatchLearning(boolean r)
```
    Set whether to use reservoir sampling when batch learning
    
    Parameters:
    
    r - true if reservoir sampling is to be used
  - useReservoirSamplingWhenBatchLearningTipText
```
public java.lang.String useReservoirSamplingWhenBatchLearningTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getReservoirSampleSize
```
public int getReservoirSampleSize()
```
    Get the sample size for reservoir sampling
    
    Returns:
    
    the sample size to use for reservoir sampling
  - setReservoirSampleSize
```
public void setReservoirSampleSize(int size)
```
    Set the sample size for reservoir sampling
    
    Parameters:
    
    size - the sample size to use for reservoir sampling
  - reservoirSampleSizeTipText
```
public java.lang.String reservoirSampleSizeTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getForceBatchLearningForUpdateableClassifiers
```
public boolean getForceBatchLearningForUpdateableClassifiers()
```
    Get whether to force batch training for incremental (Updateable) classifiers
    
    Returns:
    
    true if incremental classifiers should be batch trained
  - setForceBatchLearningForUpdateableClassifiers
```
public void setForceBatchLearningForUpdateableClassifiers(boolean force)
```
    Set whether to force batch training for incremental (Updateable) classifiers
    
    Parameters:
    
    force - true if incremental classifiers should be batch trained
  - forceBatchLearningForUpdateableClassifiersTipText
```
public java.lang.String forceBatchLearningForUpdateableClassifiersTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getContinueTrainingUpdateableClassifier
```
public boolean getContinueTrainingUpdateableClassifier()
```
    Get whether to continue training an incremental (updateable) classifier. Assumes that the classifier set via setClassifier() has already been trained (or at least initialized via buildClassifier()).
    
    Returns:
    
    true if continuing to train an incremental classifier rather than starting from scratch
  - setContinueTrainingUpdateableClassifier
```
public void setContinueTrainingUpdateableClassifier(boolean u)
```
    Set whether to continue training an incremental (updateable) classifier. Assumes that the classifier set via setClassifier() has already been trained (or at least initialized via buildClassifier()).
    
    Parameters:
    
    u - true to continue training an incremental classifier rather than starting from scratch
  - continueTrainingUpdateableClassifierTipText
```
public java.lang.String continueTrainingUpdateableClassifierTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getFoldNumber
```
public int getFoldNumber()
```
    Get the fold number to train the classifier with. -1 indicates that all the data is to be used for training.
    
    Returns:
    
    the fold number to train on.
  - setFoldNumber
```
public void setFoldNumber(int fn)
```
    Set the fold number to train the classifier with. -1 indicates that all the data is to be used for training.
    
    Parameters:
    
    fn - the fold number to train on.
  - foldNumberTipText
```
public java.lang.String foldNumberTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getTotalNumFolds
```
public int getTotalNumFolds()
```
    Get the total number of folds to use. Use in conjunction with setFoldNumber(). Only has an effect if setFoldNumber() is set to something other than -1.
    
    Returns:
    
    the total number of folds
  - setTotalNumFolds
```
public void setTotalNumFolds(int t)
```
    Set the total number of folds to use. Use in conjunction with setFoldNumber(). Only has an effect if setFoldNumber() is set to something other than -1.
    
    Parameters:
    
    t - the total number of folds
  - totalNumberOfFoldsTipText
```
public java.lang.String totalNumberOfFoldsTipText()
```
    The tool tip text for this property.
    
    Returns:
    
    the tool tip text for this property
  - getNumTrainingInstances
```
public int getNumTrainingInstances()
```
    Get the number of training instances actually used to train the classifier.
    
    Returns:
    
    the number of training instances used to train the classifier
  - addToTrainingHeader
```
public void addToTrainingHeader(weka.core.Instances toAdd)
```
    Add the supplied instances to the training header
    
    Parameters:
    
    toAdd - the instances to add
  - addToTrainingHeader
```
public void addToTrainingHeader(weka.core.Instance toAdd)
```
    Add the supplied instance to the training header
    
    Parameters:
    
    toAdd - the instance to add
  - setup
```
public void setup(weka.core.Instances trainingHeader)
           throws DistributedWekaException
```
    Initialize the map task
    
    Parameters:
    
    trainingHeader - the header of the incoming training data.
    
    Throws:
    
    DistributedWekaException - if something goes wrong
  - processInstance
```
public void processInstance(weka.core.Instance inst)
                     throws DistributedWekaException
```
    Process the supplied instance. Incremental classifiers are updated immediately with the instance. In the case of batch learning, the instance is cached in memory and the entire data set is processed in the finalizeTask() method.
    
    Parameters:
    
    inst - the instance to train with
    
    Throws:
    
    DistributedWekaException - if a problem occurs
  - finalizeTask
```
public void finalizeTask()
                  throws DistributedWekaException
```
    Finish up the map task. If the classifier is being batch trained then this is where the action happens.
    
    Throws:
    
    DistributedWekaException - if something goes wrong
  - setEnvironment
```
public void setEnvironment(weka.core.Environment env)
```
    Specified by:
    
    setEnvironment in interface weka.core.EnvironmentHandler

Class WekaClassifierMapTask

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface weka.core.OptionHandler

Field Detail

TOTAL_NUMBER_OF_MAPS

Constructor Detail

WekaClassifierMapTask

Method Detail

main

listOptions

getOptions

setOptions

getClassifier

setClassifier

classifierTipText

getSeed

setSeed

seedTipText

getForceVotedEnsembleCreation

setForceVotedEnsembleCreation

forceVotedEnsembleCreation

getFiltersToUse

setFiltersToUse

filtersToUseTipText

addPreconstructedFilterToUse

getUseReservoirSamplingWhenBatchLearning

setUseReservoirSamplingWhenBatchLearning

useReservoirSamplingWhenBatchLearningTipText

getReservoirSampleSize

setReservoirSampleSize

reservoirSampleSizeTipText

getForceBatchLearningForUpdateableClassifiers

setForceBatchLearningForUpdateableClassifiers

forceBatchLearningForUpdateableClassifiersTipText

getContinueTrainingUpdateableClassifier

setContinueTrainingUpdateableClassifier

continueTrainingUpdateableClassifierTipText

getFoldNumber

setFoldNumber

foldNumberTipText

getTotalNumFolds

setTotalNumFolds

totalNumberOfFoldsTipText

getNumTrainingInstances

addToTrainingHeader

addToTrainingHeader

setup

processInstance

finalizeTask

setEnvironment