weka.distributed.hadoop

Class RandomizedDataChunkHadoopReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

weka.distributed.hadoop.RandomizedDataChunkHadoopReducer

public class RandomizedDataChunkHadoopReducer
extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Hadoop Reducer implementation for the job that creates randomly shuffled (and stratified if the class is nominal) data chunks. Keyed by data chunk number, it deals instances out per class in a round-robin fashion amongst the output files.

Version:

$Revision: 11837 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	NUM_DATA_CHUNKS The key in the Configuration that the options for this task are associated with

Constructor Summary

Constructors

Constructor and Description
RandomizedDataChunkHadoopReducer()

Method Summary

Modifier and Type	Method and Description
void	cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
void	reduce(org.apache.hadoop.io.Text key, java.lang.Iterable<org.apache.hadoop.io.Text> values, org.apache.hadoop.mapreduce.Reducer.Context context)
void	setup(org.apache.hadoop.mapreduce.Reducer.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NUM_DATA_CHUNKS

public static java.lang.String NUM_DATA_CHUNKS

The key in the Configuration that the options for this task are associated with

Constructor Detail

RandomizedDataChunkHadoopReducer

public RandomizedDataChunkHadoopReducer()

Method Detail

setup

public void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
           throws java.io.IOException

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

java.io.IOException

reduce

public void reduce(org.apache.hadoop.io.Text key,
                   java.lang.Iterable<org.apache.hadoop.io.Text> values,
                   org.apache.hadoop.mapreduce.Reducer.Context context)
            throws java.io.IOException,
                   java.lang.InterruptedException

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

java.io.IOException

java.lang.InterruptedException

cleanup

public void cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
             throws java.io.IOException,
                    java.lang.InterruptedException

Overrides:

cleanup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

java.io.IOException

java.lang.InterruptedException