@KFStep(name="MakeResourceIntensive", category="Flow", toolTipText="Makes downstream connected steps resource intensive (or not). This shifts processing of such steps between the main step executor<br>service and the high resource executor service or vice versa.", iconPath="weka/gui/knowledgeflow/icons/DiamondPlain.gif") public class MakeResourceIntensive extends BaseStep
KFStep
class annotation with
regards to whether it is resource intensive (cpu or memory). The Knowledge
Flow execution environment uses two executor services - a primary one to
execute batch processing for steps; and a secondary one for executing
StepTask
s (which are assumed to be resource intensive by default) or
for executing batch processing for a Step when it declares itself to be
resource intensive. This secondary executor service uses a limited (typically
<= num cpu cores
) number of threads. Steps that involve potentially
intensive (cpu/memory) processing should declare themselves resource
intensive so that less taxing steps (and the UI) get cpu cycles. E.g. the
Classifier Step is resource intensive so that processing cross-validation
folds in parallel for a large data set or computationally intensive
classifier does not blow out memory or bog the system down.Constructor and Description |
---|
MakeResourceIntensive() |
Modifier and Type | Method and Description |
---|---|
java.util.List<java.lang.String> |
getIncomingConnectionTypes()
Get a list of incoming connection types that this step can accept.
|
boolean |
getMakeResourceIntensive()
Get whether downstream steps are to be made resource intensive
|
java.util.List<java.lang.String> |
getOutgoingConnectionTypes()
Get a list of outgoing connection types that this step can produce.
|
void |
processIncoming(Data data)
Process incoming data
|
void |
setMakeResourceIntensive(boolean resourceIntensive)
Set whether downstream steps are to be made resource intensive or not
|
void |
stepInit()
Initialize the step
|
environmentSubstitute, getCustomEditorForStep, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, outputStructureForConnectionType, outputStructureForConnectionType, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop
@OptionMetadata(displayName="Make downstream step(s) high resource", description="<html>Makes downstream connected steps resource intensive (or not)<br>This shifts processing of such steps between the main step executor service and the high resource executor service or vice versa.</html>") public void setMakeResourceIntensive(boolean resourceIntensive)
resourceIntensive
- true if the downstream connected steps are to be
made resource intensivepublic boolean getMakeResourceIntensive()
public void stepInit() throws WekaException
WekaException
- if a problem occurspublic java.util.List<java.lang.String> getIncomingConnectionTypes()
public java.util.List<java.lang.String> getOutgoingConnectionTypes()
public void processIncoming(Data data) throws WekaException
processIncoming
in interface BaseStepExtender
processIncoming
in interface Step
processIncoming
in class BaseStep
data
- the data to processWekaException