weka.distributed.hadoop

Class KMeansCentroidSketchHadoopMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

weka.distributed.hadoop.KMeansCentroidSketchHadoopMapper

public class KMeansCentroidSketchHadoopMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Hadoop mapper for the k-means|| initialization procedure. Uses weighted reservoir sampling to maintain a random selection of the instances with the greatest distance from the current set of centroid candidates.

Version:

$Revision: 11837 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	CENTROID_SKETCH_MAP_TASK_OPTIONS The key in the Configuration that the options for this task are associated with
static java.lang.String	SKETCH_FILE_PREFIX File prefix for serialized sketch files

Constructor Summary

Constructors

Constructor and Description
KMeansCentroidSketchHadoopMapper()

Method Summary

Modifier and Type	Method and Description
void	cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)
void	map(org.apache.hadoop.io.LongWritable key, org.apache.hadoop.io.Text value, org.apache.hadoop.mapreduce.Mapper.Context context)
void	setup(org.apache.hadoop.mapreduce.Mapper.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

run

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CENTROID_SKETCH_MAP_TASK_OPTIONS

public static final java.lang.String CENTROID_SKETCH_MAP_TASK_OPTIONS

The key in the Configuration that the options for this task are associated with

See Also:

Constant Field Values

SKETCH_FILE_PREFIX

public static final java.lang.String SKETCH_FILE_PREFIX

File prefix for serialized sketch files

See Also:

Constant Field Values

Constructor Detail

KMeansCentroidSketchHadoopMapper

public KMeansCentroidSketchHadoopMapper()

Method Detail

setup

public void setup(org.apache.hadoop.mapreduce.Mapper.Context context)
           throws java.io.IOException

Overrides:

setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

java.io.IOException

map

public void map(org.apache.hadoop.io.LongWritable key,
                org.apache.hadoop.io.Text value,
                org.apache.hadoop.mapreduce.Mapper.Context context)
         throws java.io.IOException

Overrides:

map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

java.io.IOException

cleanup

public void cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)
             throws java.io.IOException,
                    java.lang.InterruptedException

Overrides:

cleanup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

java.io.IOException

java.lang.InterruptedException