MakeResourceIntensive

java.lang.Object
- weka.knowledgeflow.steps.BaseStep
- - weka.knowledgeflow.steps.MakeResourceIntensive

All Implemented Interfaces:

java.io.Serializable, BaseStepExtender, Step
```
@KFStep(name="MakeResourceIntensive",
        category="Flow",
        toolTipText="Makes downstream connected steps resource intensive (or not). This shifts processing of such steps between the main step executor<br>service and the high resource executor service or vice versa.",
        iconPath="weka/gui/knowledgeflow/icons/DiamondPlain.gif")
public class MakeResourceIntensive
extends BaseStep
```
A Step that makes downstream steps that are directly connected to this step resource intensive (or not). This overrides whatever the downstream step may (or may not) have declared in it's KFStep class annotation with regards to whether it is resource intensive (cpu or memory). The Knowledge Flow execution environment uses two executor services - a primary one to execute batch processing for steps; and a secondary one for executing StepTasks (which are assumed to be resource intensive by default) or for executing batch processing for a Step when it declares itself to be resource intensive. This secondary executor service uses a limited (typically <= num cpu cores) number of threads. Steps that involve potentially intensive (cpu/memory) processing should declare themselves resource intensive so that less taxing steps (and the UI) get cpu cycles. E.g. the Classifier Step is resource intensive so that processing cross-validation folds in parallel for a large data set or computationally intensive classifier does not blow out memory or bog the system down.

Version:

$Revision: $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

See Also:

Serialized Form

Constructor Summary

Constructors
Constructor and Description

MakeResourceIntensive()

Constructors
Constructor and Description
`MakeResourceIntensive()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`java.util.List<java.lang.String>`	`getIncomingConnectionTypes()` Get a list of incoming connection types that this step can accept.
`boolean`	`getMakeResourceIntensive()` Get whether downstream steps are to be made resource intensive
`java.util.List<java.lang.String>`	`getOutgoingConnectionTypes()` Get a list of outgoing connection types that this step can produce.
`void`	`processIncoming(Data data)` Process incoming data
`void`	`setMakeResourceIntensive(boolean resourceIntensive)` Set whether downstream steps are to be made resource intensive or not
`void`	`stepInit()` Initialize the step

Methods inherited from class weka.knowledgeflow.steps.BaseStep
environmentSubstitute, getCustomEditorForStep, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, outputStructureForConnectionType, outputStructureForConnectionType, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - MakeResourceIntensive
```
public MakeResourceIntensive()
```
- Method Detail
  - setMakeResourceIntensive
```
@OptionMetadata(displayName="Make downstream step(s) high resource",
                description="<html>Makes downstream connected steps resource intensive (or not)<br>This shifts processing of such steps between the main step executor service and the high resource executor service or vice versa.</html>")
public void setMakeResourceIntensive(boolean resourceIntensive)
```
    Set whether downstream steps are to be made resource intensive or not
    
    Parameters:
    
    resourceIntensive - true if the downstream connected steps are to be made resource intensive
  - getMakeResourceIntensive
```
public boolean getMakeResourceIntensive()
```
    Get whether downstream steps are to be made resource intensive
    
    Returns:
    
    true if downstream connected steps are to be made resource intensive
  - stepInit
```
public void stepInit()
              throws WekaException
```
    Initialize the step
    
    Throws:
    
    WekaException - if a problem occurs
  - getIncomingConnectionTypes
```
public java.util.List<java.lang.String> getIncomingConnectionTypes()
```
    Get a list of incoming connection types that this step can accept. Ideally (and if appropriate), this should take into account the state of the step and any existing incoming connections. E.g. a step might be able to accept one (and only one) incoming batch data connection.
    
    Returns:
    
    a list of incoming connections that this step can accept given its current state
  - getOutgoingConnectionTypes
```
public java.util.List<java.lang.String> getOutgoingConnectionTypes()
```
    Get a list of outgoing connection types that this step can produce. Ideally (and if appropriate), this should take into account the state of the step and the incoming connections. E.g. depending on what incoming connection is present, a step might be able to produce a trainingSet output, a testSet output or neither, but not both.
    
    Returns:
    
    a list of outgoing connections that this step can produce
  - processIncoming
```
public void processIncoming(Data data)
                     throws WekaException
```
    Process incoming data
    
    Specified by:
    
    processIncoming in interface BaseStepExtender
    
    Specified by:
    
    processIncoming in interface Step
    
    Overrides:
    
    processIncoming in class BaseStep
    
    Parameters:
    
    data - the data to process
    
    Throws:
    
    WekaException

Class MakeResourceIntensive

Constructor Summary

Method Summary

Methods inherited from class weka.knowledgeflow.steps.BaseStep

Methods inherited from class java.lang.Object

Constructor Detail

MakeResourceIntensive

Method Detail

setMakeResourceIntensive

getMakeResourceIntensive

stepInit

getIncomingConnectionTypes

getOutgoingConnectionTypes

processIncoming