weka.distributed

Class CSVToARFFHeaderReduceTask

java.lang.Object

weka.distributed.CSVToARFFHeaderReduceTask

All Implemented Interfaces:

java.io.Serializable

public class CSVToARFFHeaderReduceTask
extends java.lang.Object
implements java.io.Serializable

Reduce task for ARFF header and summary attribute creation.

Version:

$Revision: 13989 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CSVToARFFHeaderReduceTask()

Method Summary

Modifier and Type	Method and Description
static weka.core.Instances	aggregate(java.util.List<weka.core.Instances> headers) Aggregates a list of Instances (headers) into a final Instances object.
static weka.core.Instances	aggregateHeadersAndQuartiles(java.util.List<CSVToARFFHeaderMapTask.HeaderAndQuantileDataHolder> toAggregate) Performs aggregation over a list of header and quantile data holder objects.
static boolean	headerContainsNumericAttributes(weka.core.Instances headerWithSummary) Returns true if the supplied header contains numeric attributes
static boolean	headerContainsQuartiles(weka.core.Instances headerWithSummary) Returns true if the supplied header already has quartile infomration calculated and there are numeric attributes in the data
static void	main(java.lang.String[] args)
static weka.core.Instances	stripSummaryAtts(weka.core.Instances insts) Utility method that returns a header Instances object without any summary attributes.
static weka.core.Instances	updateSummaryAttsWithQuartilesAndHistograms(weka.core.Instances trainingHeaderWithSummary, QuantileCalculator quartiles, java.util.Map<java.lang.Integer,NumericAttributeBinData> histograms) Updates a header that contains summary attributes with quartiles and histogram data.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CSVToARFFHeaderReduceTask

public CSVToARFFHeaderReduceTask()

Method Detail

aggregateHeadersAndQuartiles

public static weka.core.Instances aggregateHeadersAndQuartiles(java.util.List<CSVToARFFHeaderMapTask.HeaderAndQuantileDataHolder> toAggregate)
                                                        throws DistributedWekaException

Performs aggregation over a list of header and quantile data holder objects. Produces a final aggregated heaer with quantile and histogram data + normal stats for numeric attributes.

Parameters:

toAggregate - the list of header and quantile data holders to aggregate over

Returns:

a final aggregated header with normal stats plus quantiles and histograms for numeric attributes

Throws:

DistributedWekaException - if a problem occurs

aggregate

public static weka.core.Instances aggregate(java.util.List<weka.core.Instances> headers)
                                     throws DistributedWekaException

Aggregates a list of Instances (headers) into a final Instances object.

Parameters:

headers - a list of headers to aggregate

Returns:

a unified header as an Instances object

Throws:

DistributedWekaException - if a problem occurs

stripSummaryAtts

public static weka.core.Instances stripSummaryAtts(weka.core.Instances insts)
                                            throws DistributedWekaException

Utility method that returns a header Instances object without any summary attributes.

Parameters:

insts - the header to remove summary attributes from

Returns:

a new Instances object that does not contain any summary attributes

Throws:

DistributedWekaException - if a problem occurs

updateSummaryAttsWithQuartilesAndHistograms

public static weka.core.Instances updateSummaryAttsWithQuartilesAndHistograms(weka.core.Instances trainingHeaderWithSummary,
                                                                              QuantileCalculator quartiles,
                                                                              java.util.Map<java.lang.Integer,NumericAttributeBinData> histograms)
                                                                       throws DistributedWekaException

Updates a header that contains summary attributes with quartiles and histogram data. Assumes the header does not already contain these.

Parameters:

trainingHeaderWithSummary - header with first pass summary data

quartiles - QuartileCalculator containing quartiles to add to the header

histograms - Map (keyed by attribute index) of histogram data for numeric attributes to add to the header

Returns:

an updated header

Throws:

DistributedWekaException - if a problem occurs

headerContainsNumericAttributes

public static boolean headerContainsNumericAttributes(weka.core.Instances headerWithSummary)
                                               throws DistributedWekaException

Returns true if the supplied header contains numeric attributes

Parameters:

headerWithSummary - a header (with summary attributes) to check

Returns:

true if the header contains numeric attributes

Throws:

DistributedWekaException - if a problem occurs

headerContainsQuartiles

public static boolean headerContainsQuartiles(weka.core.Instances headerWithSummary)
                                       throws DistributedWekaException

Returns true if the supplied header already has quartile infomration calculated and there are numeric attributes in the data

Parameters:

headerWithSummary - the header to check

Returns:

true if the supplied header has quartile information

Throws:

DistributedWekaException - if a problem occurs

main

public static void main(java.lang.String[] args)