distributed.hadoop

Class HDFSConfig

java.lang.Object

distributed.core.DistributedJobConfig

distributed.hadoop.AbstractHadoopJobConfig

distributed.hadoop.HDFSConfig

All Implemented Interfaces:

java.io.Serializable, OptionHandler

public class HDFSConfig
extends AbstractHadoopJobConfig
implements OptionHandler

Config for HDFS filesystem related properties

Version:

$Revision: 12439 $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DEFAULT_HOST Default host for the name node
static java.lang.String	DEFAULT_PORT Default port for the name node
static java.lang.String	HADOOP_FS_DEFAULT_NAME property in core-site.xml that specifies the host running the namenode
static java.lang.String	HDFS_HOST Key for the HDFS host (name node)
static java.lang.String	HDFS_PORT Key for the HDFS port
static java.lang.String	IPC_CLIENT_CONNECT_MAX_RETRIES property in the core-site.xml that specifies how many times to try to contact the name node before giving up

Fields inherited from class distributed.hadoop.AbstractHadoopJobConfig

DEFAULT_PORT_YARN, JOBTRACKER_HOST, JOBTRACKER_PORT

Constructor Summary

Constructors

Constructor and Description
HDFSConfig() Constructor - sets defaults

Method Summary

Modifier and Type	Method and Description
void	configureForHadoop(org.apache.hadoop.conf.Configuration conf, Environment env) Populate the supplied Configuration object with HDFS-related settings
static java.lang.String	constructHostURL(HDFSConfig config, Environment env) Build an HDFS URL from the settings in the supplied HDFSConfig
java.lang.String	getHDFSHost() Get the HDFS (name node) host to use
java.lang.String	getHDFSPort() Get the HDFS (name node) port
java.lang.String[]	getOptions()
java.lang.String	HDFSHostTipText() Get the tool tip text for this property
java.lang.String	HDFSPortTipText() Get the tool tip text for this property
java.util.Enumeration<Option>	listOptions()
void	setHDFSHost(java.lang.String host) Set the HDFS (name node) host to use
void	setHDFSPort(java.lang.String port) Set the HDFS (name node) port
void	setOptions(java.lang.String[] options)

Methods inherited from class distributed.hadoop.AbstractHadoopJobConfig

getJobTrackerHost, getJobTrackerPort, isHadoop2, jobTrackerHostTipText, jobTrackerPortTipText, setJobTrackerHost, setJobTrackerPort

Methods inherited from class distributed.core.DistributedJobConfig

clearUserSuppliedProperties, getProperty, getPropertyNames, getUserSuppliedProperties, getUserSuppliedProperty, getUserSuppliedPropertyNames, isEmpty, setProperty, setUserSuppliedProperty

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HDFS_HOST

public static final java.lang.String HDFS_HOST

Key for the HDFS host (name node)

See Also:

Constant Field Values

HDFS_PORT

public static final java.lang.String HDFS_PORT

Key for the HDFS port

See Also:

Constant Field Values

HADOOP_FS_DEFAULT_NAME

public static final java.lang.String HADOOP_FS_DEFAULT_NAME

property in core-site.xml that specifies the host running the namenode

IPC_CLIENT_CONNECT_MAX_RETRIES

public static final java.lang.String IPC_CLIENT_CONNECT_MAX_RETRIES

property in the core-site.xml that specifies how many times to try to contact the name node before giving up

See Also:

Constant Field Values

DEFAULT_PORT

public static final java.lang.String DEFAULT_PORT

Default port for the name node

See Also:

Constant Field Values

DEFAULT_HOST

public static final java.lang.String DEFAULT_HOST

Default host for the name node

See Also:

Constant Field Values

Constructor Detail

HDFSConfig

public HDFSConfig()

Constructor - sets defaults

Method Detail

listOptions

public java.util.Enumeration<Option> listOptions()

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class distributed.core.DistributedJobConfig

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class distributed.core.DistributedJobConfig

Throws:

java.lang.Exception

getOptions

public java.lang.String[] getOptions()

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class distributed.core.DistributedJobConfig

HDFSHostTipText

public java.lang.String HDFSHostTipText()

Get the tool tip text for this property

Returns:

the tool tip text for this property

setHDFSHost

public void setHDFSHost(java.lang.String host)

Set the HDFS (name node) host to use

Parameters:

host - the name node's host

getHDFSHost

public java.lang.String getHDFSHost()

Get the HDFS (name node) host to use

Returns:

the name node's host

HDFSPortTipText

public java.lang.String HDFSPortTipText()

Get the tool tip text for this property

Returns:

the tool tip text for this property

setHDFSPort

public void setHDFSPort(java.lang.String port)

Set the HDFS (name node) port

Parameters:

port - the port that the name node is running on

getHDFSPort

public java.lang.String getHDFSPort()

Get the HDFS (name node) port

Returns:

the port that the name node is running on

constructHostURL

public static java.lang.String constructHostURL(HDFSConfig config,
                                                Environment env)

Build an HDFS URL from the settings in the supplied HDFSConfig

Parameters:

config - the HDFS config to use for building the URL

env - environment variables

Returns:

a HDFS URL as a String

configureForHadoop

public void configureForHadoop(org.apache.hadoop.conf.Configuration conf,
                               Environment env)

Populate the supplied Configuration object with HDFS-related settings

Parameters:

conf - the Configuration object to populate

env - environment variables