weka.core.converters

Class HDFSLoader

java.lang.Object

weka.core.converters.AbstractLoader

weka.core.converters.HDFSLoader

All Implemented Interfaces:

java.io.Serializable, CommandlineRunnable, BatchConverter, IncrementalConverter, Loader, EnvironmentHandler, OptionHandler, RevisionHandler

public class HDFSLoader
extends AbstractLoader
implements BatchConverter, IncrementalConverter, EnvironmentHandler, OptionHandler, CommandlineRunnable

Loader for data stored in HDFS. Delegates to a base loader to do the actual data parsing/extraction to instances.

Version:

$Revision: 10408 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface weka.core.converters.Loader

Loader.StructureNotReadyException

Field Summary

Fields inherited from interface weka.core.converters.Loader

BATCH, INCREMENTAL, NONE

Constructor Summary

Constructors

Constructor and Description
HDFSLoader() Constructor

Method Summary

Methods

Modifier and Type	Method and Description
HDFSConfig	getConfig() Get the HDFSConfig
Instances	getDataSet()
java.lang.String	getHDFSPath() Get the path to save to in HDFS
AbstractFileLoader	getLoader() Get the base loader to use
Instance	getNextInstance(Instances structure)
java.lang.String[]	getOptions()
java.lang.String	getRevision()
Instances	getStructure()
java.lang.String	globalInfo() Help information for the HDFSLoader
java.lang.String	HDFSPathTipText() Tip text for this property
java.util.Enumeration<Option>	listOptions()
java.lang.String	loaderTipText() Tip text for this property
static void	main(java.lang.String[] args) Main method
void	reset()
void	run(java.lang.Object toRun, java.lang.String[] options)
void	setEnvironment(Environment env)
void	setHDFSPath(java.lang.String path) Set the path to load from in HDFS
void	setLoader(AbstractFileLoader loader) Set the base loader to use
void	setOptions(java.lang.String[] options)

Methods inherited from class weka.core.converters.AbstractLoader

setRetrieval, setSource, setSource

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HDFSLoader

public HDFSLoader()
           throws java.lang.Exception

Constructor

Throws:

java.lang.Exception

Method Detail

globalInfo

public java.lang.String globalInfo()

Help information for the HDFSLoader

Returns:

textual help information

listOptions

public java.util.Enumeration<Option> listOptions()

Specified by:

listOptions in interface OptionHandler

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Specified by:

setOptions in interface OptionHandler

Throws:

java.lang.Exception

getOptions

public java.lang.String[] getOptions()

Specified by:

getOptions in interface OptionHandler

getConfig

public HDFSConfig getConfig()

Get the HDFSConfig

Returns:

the HDFSConfig

HDFSPathTipText

public java.lang.String HDFSPathTipText()

Tip text for this property

Returns:

the tip text for this property

setHDFSPath

public void setHDFSPath(java.lang.String path)

Set the path to load from in HDFS

Parameters:

path - the path to load from

getHDFSPath

public java.lang.String getHDFSPath()

Get the path to save to in HDFS

Returns:

the path to save to

loaderTipText

public java.lang.String loaderTipText()

Tip text for this property

Returns:

the tip text for this property

setLoader

public void setLoader(AbstractFileLoader loader)

Set the base loader to use

Parameters:

loader - the base laoder to use

getLoader

public AbstractFileLoader getLoader()

Get the base loader to use

Returns:

the base loader to use

getRevision

public java.lang.String getRevision()

Specified by:

getRevision in interface RevisionHandler

reset

public void reset()
           throws java.lang.Exception

Specified by:

reset in interface Loader

Overrides:

reset in class AbstractLoader

Throws:

java.lang.Exception

getStructure

public Instances getStructure()
                       throws java.io.IOException

Specified by:

getStructure in interface Loader

Specified by:

getStructure in class AbstractLoader

Throws:

java.io.IOException

getDataSet

public Instances getDataSet()
                     throws java.io.IOException

Specified by:

getDataSet in interface Loader

Specified by:

getDataSet in class AbstractLoader

Throws:

java.io.IOException

getNextInstance

public Instance getNextInstance(Instances structure)
                         throws java.io.IOException

Specified by:

getNextInstance in interface Loader

Specified by:

getNextInstance in class AbstractLoader

Throws:

java.io.IOException

setEnvironment

public void setEnvironment(Environment env)

Specified by:

setEnvironment in interface EnvironmentHandler

main

public static void main(java.lang.String[] args)

Main method

Parameters:

args - should contain the options of a Loader

run

public void run(java.lang.Object toRun,
       java.lang.String[] options)

Specified by:

run in interface CommandlineRunnable