weka.distributed.hadoop

Class CorrelationMatrixHadoopJob

java.lang.Object

distributed.core.DistributedJob

weka.distributed.hadoop.HadoopJob

weka.distributed.hadoop.CorrelationMatrixHadoopJob

All Implemented Interfaces:

java.io.Serializable, CommandlineRunnable, EnvironmentHandler, OptionHandler, ImageProducer, TextProducer

public class CorrelationMatrixHadoopJob
extends HadoopJob
implements TextProducer, ImageProducer, CommandlineRunnable

Hadoop job for constructing a correlation matrix. Has an option to use the matrix constructed for performing (a non-distributed) PCA analysis.

Version:

$Revision: 10853 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class distributed.core.DistributedJob

distributed.core.DistributedJob.JobStatus

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	OUTPUT_SUBDIR Subdirectory of the output directory for storing results to

Fields inherited from class weka.distributed.hadoop.HadoopJob

COLT_JAR, DISTRIBUTED_WEKA_BASE_JAR, DISTRIBUTED_WEKA_HADOOP_JAR, JCOMMON_JAR, JFREECHART_JAR, LA4J_JAR, OPEN_CSV_JAR

Fields inherited from class distributed.core.DistributedJob

WEKA_ADDITIONAL_PACKAGES_KEY

Constructor Summary

Constructors

Constructor and Description
CorrelationMatrixHadoopJob() Constructor

Method Summary

Methods

Modifier and Type	Method and Description
java.lang.String	classAttributeTipText() Tip text for this property
java.lang.String	getClassAttribute() Get the name or index of the class attribute.
java.lang.String	getCorrelationMapTaskOptions() Get options for the correlation map tasks
java.lang.String	getCSVMapTaskOptions() Get the options for the csv map tasks
java.awt.Image	getImage()
java.lang.String[]	getJobOptionsOnly() Get just the job options
Matrix	getMatrix() Get the matrix generated by this job, or null if the job has not been run yet.
java.lang.String	getNumNodesInCluster() Get the number of nodes in the Hadoop cluster
java.lang.String[]	getOptions()
boolean	getRunPCA() Get whether to run a PCA analysis (using the generated correlation matrix as inpu) as a post-processing step
java.lang.String	getText()
java.lang.String	globalInfo() Textual help info for this job
java.util.Enumeration<Option>	listOptions()
static void	main(java.lang.String[] args) Main method for executing this job from the command line
java.lang.String	numNodesInClusterTipText() Tip text for this property
void	run(java.lang.Object toRun, java.lang.String[] args)
boolean	runJob()
java.lang.String	runPCATipText() Tip text for this property
void	setClassAttribute(java.lang.String c) Set the name or index of the class attribute.
void	setCorrelationMapTaskOptions(java.lang.String opts) Set options for the correlation map tasks
void	setCSVMapTaskOptions(java.lang.String opts) Set the options for the csv map tasks
void	setNumNodesInCluster(java.lang.String n) Set the number of nodes in the Hadoop cluster
void	setOptions(java.lang.String[] options)
void	setRunPCA(boolean runPCA) Set whether to run a PCA analysis (using the generated correlation matrix as inpu) as a post-processing step

Methods inherited from class weka.distributed.hadoop.HadoopJob

additionalWekaPackagesTipText, cleanOutputDirectory, deubgTipText, getAdditionalWekaPackages, getBaseOptionsOnly, getDebug, getLoggingInterval, getMapNumber, getMapReduceJobConfig, getMapReduceNumber, getPathToWekaJar, getReduceNumber, loggingIntervalTipText, pathToWekaJarTipText, setAdditionalWekaPackages, setDebug, setLoggingInterval, setMapReduceJobConfig, setPathToWekaJar

Methods inherited from class distributed.core.DistributedJob

environmentSubstitute, getAdditionalWekaPackageNames, getJobName, getJobStatus, getLog, makeOptionsStr, parseInstance, setEnvironment, setJobDescription, setJobName, setJobStatus, setLog, setStatusMessagePrefix, stopJob

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_SUBDIR

public static final java.lang.String OUTPUT_SUBDIR

Subdirectory of the output directory for storing results to

See Also:

Constant Field Values

Constructor Detail

CorrelationMatrixHadoopJob

public CorrelationMatrixHadoopJob()

Constructor

Method Detail

globalInfo

public java.lang.String globalInfo()

Textual help info for this job

Returns:

help info

setCSVMapTaskOptions

public void setCSVMapTaskOptions(java.lang.String opts)

Set the options for the csv map tasks

Parameters:

opts - options for the csv map taksk

getCSVMapTaskOptions

public java.lang.String getCSVMapTaskOptions()

Get the options for the csv map tasks

Returns:

options for the csv map taksk

setCorrelationMapTaskOptions

public void setCorrelationMapTaskOptions(java.lang.String opts)

Set options for the correlation map tasks

Parameters:

opts - options for the correlation map tasks

getCorrelationMapTaskOptions

public java.lang.String getCorrelationMapTaskOptions()

Get options for the correlation map tasks

Returns:

options for the correlation map tasks

numNodesInClusterTipText

public java.lang.String numNodesInClusterTipText()

Tip text for this property

Returns:

the tip text for this property

setNumNodesInCluster

public void setNumNodesInCluster(java.lang.String n)

Set the number of nodes in the Hadoop cluster

Parameters:

n - the number of nodes in the Hadoop cluster

getNumNodesInCluster

public java.lang.String getNumNodesInCluster()

Get the number of nodes in the Hadoop cluster

Returns:

the number of nodes in the Hadoop cluster

getMatrix

public Matrix getMatrix()

Get the matrix generated by this job, or null if the job has not been run yet.

Returns:

the matrix generated by this job or null if the job has not been run yet.

listOptions

public java.util.Enumeration<Option> listOptions()

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class HadoopJob

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class HadoopJob

Throws:

java.lang.Exception

getOptions

public java.lang.String[] getOptions()

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class HadoopJob

getJobOptionsOnly

public java.lang.String[] getJobOptionsOnly()

Get just the job options

Returns:

the job options

classAttributeTipText

public java.lang.String classAttributeTipText()

Tip text for this property

Returns:

the tip text for this property

setClassAttribute

public void setClassAttribute(java.lang.String c)

Set the name or index of the class attribute.

Parameters:

c - the name or index of the class attribute

getClassAttribute

public java.lang.String getClassAttribute()

Get the name or index of the class attribute.

Returns:

name or index of the class attribute

runPCATipText

public java.lang.String runPCATipText()

Tip text for this property

Returns:

the tip text for this property

setRunPCA

public void setRunPCA(boolean runPCA)

Set whether to run a PCA analysis (using the generated correlation matrix as inpu) as a post-processing step

Parameters:

runPCA - true if PCA should be run after the correlation job finishes

getRunPCA

public boolean getRunPCA()

Get whether to run a PCA analysis (using the generated correlation matrix as inpu) as a post-processing step

Returns:

true if PCA should be run after the correlation job finishes

runJob

public boolean runJob()
               throws weka.distributed.DistributedWekaException

Specified by:

runJob in class distributed.core.DistributedJob

Throws:

weka.distributed.DistributedWekaException

getText

public java.lang.String getText()

Specified by:

getText in interface TextProducer

getImage

public java.awt.Image getImage()

Specified by:

getImage in interface ImageProducer

main

public static void main(java.lang.String[] args)

Main method for executing this job from the command line

Parameters:

args - arguments to the job

run

public void run(java.lang.Object toRun,
       java.lang.String[] args)

Specified by:

run in interface CommandlineRunnable