weka.distributed.hadoop

Class ArffHeaderHadoopJob

java.lang.Object

distributed.core.DistributedJob

weka.distributed.hadoop.HadoopJob

weka.distributed.hadoop.ArffHeaderHadoopJob

All Implemented Interfaces:

java.io.Serializable, CommandlineRunnable, EnvironmentHandler, OptionHandler, InstancesProducer

public class ArffHeaderHadoopJob
extends HadoopJob
implements InstancesProducer, CommandlineRunnable

A Hadoop job for creating a unified ARFF header (including summary "meta" attributes).

Version:

$Revision: 11020 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class distributed.core.DistributedJob

distributed.core.DistributedJob.JobStatus

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	OUTPUT_SUBDIR Subdirectory of the output directory for storing results to

Fields inherited from class weka.distributed.hadoop.HadoopJob

COLT_JAR, DISTRIBUTED_WEKA_BASE_JAR, DISTRIBUTED_WEKA_HADOOP_JAR, JCOMMON_JAR, JFREECHART_JAR, LA4J_JAR, OPEN_CSV_JAR

Fields inherited from class distributed.core.DistributedJob

WEKA_ADDITIONAL_PACKAGES_KEY

Constructor Summary

Constructors

Constructor and Description
ArffHeaderHadoopJob() Constructor

Method Summary

Methods

Modifier and Type	Method and Description
java.lang.String	attributeNamesFileTipText() Tip text for this property
java.lang.String	attributeNamesTipText() Tip text for this property
java.lang.String	csvToArffTaskOptionsTipText() Tip text for this property
java.lang.String	generateChartsTipText() Tip text for this property
java.lang.String	getAggregatedHeaderPath()
java.lang.String	getAttributeNames() Get a comma-separated list of attribute names to use when generating the ARFF header.
java.lang.String	getAttributeNamesFile() Get the path to an file containing attribute names to use.
java.lang.String	getCsvToArffTaskOptions() Get the options to pass on to the underlying csv to arff task
Instances	getFinalHeader() Get the header that was generated by the job.
boolean	getGenerateCharts() Get whether to generate summary charts as png files in the output directory (histograms, pie charts, box plots).
Instances	getInstances()
java.lang.String[]	getJobOptionsOnly() Get the options specific to this job only.
java.lang.String[]	getOptions()
java.lang.String	getOutputHeaderFileName() Get the name of the header file to create in the output directory.
java.lang.String	getPathToExistingHeader() Get the path to an previously created header file to use.
java.util.List<java.awt.image.BufferedImage>	getSummaryCharts() Get the summary charts (if any)
java.lang.String	globalInfo() Returns textual help information for this job
java.util.Enumeration<Option>	listOptions()
static void	main(java.lang.String[] args)
java.lang.String	outputHeaderFileNameTipText() The tip text for this property
java.lang.String	pathToExistingHeaderTipText() The tip text for this property
void	run(java.lang.Object toRun, java.lang.String[] options)
boolean	runJob()
void	setAttributeNames(java.lang.String names) Set a comma-separated list of attribute names to use when generating the ARFF header.
void	setAttributeNamesFile(java.lang.String namesfile) Set the path to an file containing attribute names to use.
void	setCsvToArffTaskOptions(java.lang.String opts) Set the options to pass on to the underlying csv to arff task
void	setGenerateCharts(boolean g) Set whether to generate summary charts as png files in the output directory (histograms, pie charts, box plots).
void	setOptions(java.lang.String[] options)
void	setOutputHeaderFileName(java.lang.String name) Set the name of the header file to create in the output directory.
void	setPathToExistingHeader(java.lang.String path) Set the path to an previously created header file to use.

Methods inherited from class weka.distributed.hadoop.HadoopJob

additionalWekaPackagesTipText, cleanOutputDirectory, deubgTipText, getAdditionalWekaPackages, getBaseOptionsOnly, getDebug, getLoggingInterval, getMapNumber, getMapReduceJobConfig, getMapReduceNumber, getPathToWekaJar, getReduceNumber, loggingIntervalTipText, pathToWekaJarTipText, setAdditionalWekaPackages, setDebug, setLoggingInterval, setMapReduceJobConfig, setPathToWekaJar

Methods inherited from class distributed.core.DistributedJob

environmentSubstitute, getAdditionalWekaPackageNames, getJobName, getJobStatus, getLog, makeOptionsStr, parseInstance, setEnvironment, setJobDescription, setJobName, setJobStatus, setLog, setStatusMessagePrefix, stopJob

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_SUBDIR

public static final java.lang.String OUTPUT_SUBDIR

Subdirectory of the output directory for storing results to

See Also:

Constant Field Values

Constructor Detail

ArffHeaderHadoopJob

public ArffHeaderHadoopJob()

Constructor

Method Detail

globalInfo

public java.lang.String globalInfo()

Returns textual help information for this job

Returns:

help information for this job

getAggregatedHeaderPath

public java.lang.String getAggregatedHeaderPath()

listOptions

public java.util.Enumeration<Option> listOptions()

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class HadoopJob

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class HadoopJob

Throws:

java.lang.Exception

getJobOptionsOnly

public java.lang.String[] getJobOptionsOnly()

Get the options specific to this job only.

Returns:

the options specific to this job only

getOptions

public java.lang.String[] getOptions()

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class HadoopJob

getFinalHeader

public Instances getFinalHeader()

Get the header that was generated by the job. Returns null if the job hasn't been run yet.

Returns:

the header generated by the job or null if the job hasn't been run yet.

getInstances

public Instances getInstances()

Specified by:

getInstances in interface InstancesProducer

pathToExistingHeaderTipText

public java.lang.String pathToExistingHeaderTipText()

The tip text for this property

Returns:

the tip text for this property

setPathToExistingHeader

public void setPathToExistingHeader(java.lang.String path)

Set the path to an previously created header file to use. If set this prevents the job from running

Parameters:

path - the path to a previously created header

getPathToExistingHeader

public java.lang.String getPathToExistingHeader()

Get the path to an previously created header file to use. If set this prevents the job from running

Returns:

the path to a previously created header

outputHeaderFileNameTipText

public java.lang.String outputHeaderFileNameTipText()

The tip text for this property

Returns:

the tip text for this property

setOutputHeaderFileName

public void setOutputHeaderFileName(java.lang.String name)

Set the name of the header file to create in the output directory. If left unset then a name is generated automatically.

Parameters:

name - the name for the ARFF header file

getOutputHeaderFileName

public java.lang.String getOutputHeaderFileName()

Get the name of the header file to create in the output directory. If left unset then a name is generated automatically.

Returns:

the name for the ARFF header file

attributeNamesTipText

public java.lang.String attributeNamesTipText()

Tip text for this property

Returns:

the tip text for this property

setAttributeNames

public void setAttributeNames(java.lang.String names)

Set a comma-separated list of attribute names to use when generating the ARFF header. Use this or a names file to specify attribute names to use. Otherwise, names are generated automatically

Parameters:

names - the names of the attributes

getAttributeNames

public java.lang.String getAttributeNames()

Get a comma-separated list of attribute names to use when generating the ARFF header. Use this or a names file to specify attribute names to use. Otherwise, names are generated automatically

Returns:

the names of the attributes

attributeNamesFileTipText

public java.lang.String attributeNamesFileTipText()

Tip text for this property

Returns:

the tip text for this property

setAttributeNamesFile

public void setAttributeNamesFile(java.lang.String namesfile)

Set the path to an file containing attribute names to use. Names should be listed one per line in the file. The file may reside on the local file system or in HDFS. Use either this option or setAttributeNames to specify names for the attributes when creating the ARFF header. If left unset then names will be generated automatically.

Parameters:

namesfile - the path to a names file to use

getAttributeNamesFile

public java.lang.String getAttributeNamesFile()

Get the path to an file containing attribute names to use. Names should be listed one per line in the file. The file may reside on the local file system or in HDFS. Use either this option or setAttributeNames to specify names for the attributes when creating the ARFF header. If left unset then names will be generated automatically.

Returns:

the path to a names file to use

generateChartsTipText

public java.lang.String generateChartsTipText()

Tip text for this property

Returns:

the tip text for this property

setGenerateCharts

public void setGenerateCharts(boolean g)

Set whether to generate summary charts as png files in the output directory (histograms, pie charts, box plots).

Parameters:

g - true if summary charts are to be generated

getGenerateCharts

public boolean getGenerateCharts()

Get whether to generate summary charts as png files in the output directory (histograms, pie charts, box plots).

Returns:

true if summary charts are to be generated

csvToArffTaskOptionsTipText

public java.lang.String csvToArffTaskOptionsTipText()

Tip text for this property

Returns:

the tip text for this property

setCsvToArffTaskOptions

public void setCsvToArffTaskOptions(java.lang.String opts)

Set the options to pass on to the underlying csv to arff task

Parameters:

opts - options to pass on to the csv to arff map and reduce tasks

getCsvToArffTaskOptions

public java.lang.String getCsvToArffTaskOptions()

Get the options to pass on to the underlying csv to arff task

Returns:

options to pass on to the csv to arff map and reduce tasks

getSummaryCharts

public java.util.List<java.awt.image.BufferedImage> getSummaryCharts()

Get the summary charts (if any)

Returns:

a list of Image objects or null

runJob

public boolean runJob()
               throws weka.distributed.DistributedWekaException

Specified by:

runJob in class distributed.core.DistributedJob

Throws:

weka.distributed.DistributedWekaException

main

public static void main(java.lang.String[] args)

run

public void run(java.lang.Object toRun,
       java.lang.String[] options)
         throws java.lang.IllegalArgumentException

Specified by:

run in interface CommandlineRunnable

Throws:

java.lang.IllegalArgumentException