weka.distributed.hadoop

Class RandomizedDataChunkHadoopMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

weka.distributed.hadoop.RandomizedDataChunkHadoopMapper

public class RandomizedDataChunkHadoopMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Hadoop Mapper implementation for the job that creates randomly shuffled (and stratified if the class is nominal) data chunks. Distributes rows by randomly assigning them a key from the range 1 - num chunks.

Version:

$Revision: 11095 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	RANDOMIZED_DATA_CHUNK_MAP_TASK_OPTIONS The key in the Configuration that the options for this task are associated with

Constructor Summary

Constructors

Constructor and Description
RandomizedDataChunkHadoopMapper()

Method Summary

Methods

Modifier and Type	Method and Description
void	map(org.apache.hadoop.io.LongWritable key, org.apache.hadoop.io.Text value, org.apache.hadoop.mapreduce.Mapper.Context context)
void	setup(org.apache.hadoop.mapreduce.Mapper.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

run

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RANDOMIZED_DATA_CHUNK_MAP_TASK_OPTIONS

public static java.lang.String RANDOMIZED_DATA_CHUNK_MAP_TASK_OPTIONS

The key in the Configuration that the options for this task are associated with

Constructor Detail

RandomizedDataChunkHadoopMapper

public RandomizedDataChunkHadoopMapper()

Method Detail

setup

public void setup(org.apache.hadoop.mapreduce.Mapper.Context context)
           throws java.io.IOException

Overrides:

setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

java.io.IOException

map

public void map(org.apache.hadoop.io.LongWritable key,
       org.apache.hadoop.io.Text value,
       org.apache.hadoop.mapreduce.Mapper.Context context)
         throws java.io.IOException

Overrides:

map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

java.io.IOException