weka.python

Class PythonSession

java.lang.Object

weka.python.PythonSession

public class PythonSession
extends java.lang.Object

Class that manages interaction with the python micro server. Launches the server and shuts it down on VM exit.

Version:

$Revision: $

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PythonSession.PythonVariableType

Method Summary

Modifier and Type	Method and Description
static PythonSession	acquireSession(java.lang.Object requester) Acquire the default session for the requester
static PythonSession	acquireSession(java.lang.String pythonCommand, java.lang.String ownerID, java.lang.Object requester)
boolean	checkIfPythonVariableIsSet(java.lang.String varName, boolean debug) Check if a named variable is set/exists in the python environment
java.util.List<java.lang.String>	executeScript(java.lang.String pyScript, boolean debug) Execute an arbitrary script in python
weka.core.Instances	getDataFrameAsInstances(java.lang.String frameName, boolean debug) Retrieve a pandas data frame from Python and convert it to a set of instances.
java.awt.image.BufferedImage	getImageFromPython(java.lang.String varName, boolean debug) Retrieve an image from python.
java.util.List<java.lang.String>	getPythonDebugBuffer(boolean debug) Grab the contents of the debug buffer from the python server.
static java.lang.String	getPythonEnvCheckResults() Gets the result of running the checks in python
static java.lang.String	getPythonEnvCheckResults(java.lang.String pythonCommand, java.lang.String ownerID) Gets the result of running the checks in python for the given python path + optional ownerID.
PythonSession.PythonVariableType	getPythonVariableType(java.lang.String varName, boolean debug) Get the type of a variable in python
java.util.List<java.lang.String[]>	getVariableListFromPython(boolean debug) Get a list of variables that are set in python.
java.lang.Object	getVariableValueFromPythonAsJson(java.lang.String varName, boolean debug) Attempt to retrieve the value of a variable in python using serialization to Json.
java.lang.String	getVariableValueFromPythonAsPickledObject(java.lang.String varName, boolean debug) Attempt to retrieve the value of a variable in python using pickle serialization.
java.lang.String	getVariableValueFromPythonAsPlainString(java.lang.String varName, boolean debug) Attempt to retrieve the value of a variable in python as a plain string (i.e.
static boolean	initSession(java.lang.String pythonCommand, boolean debug) Initialize the default session.
static boolean	initSession(java.lang.String pythonCommand, java.lang.String ownerID, java.lang.String pathEntries, boolean debug) Initialize a server/session for a user-supplied python path and (optional) ownerID.
void	instancesToPython(weka.core.Instances instances, java.lang.String pythonFrameName, boolean debug) Transfer Weka instances into python as a named pandas data frame
void	instancesToPythonAsScikitLearn(weka.core.Instances instances, java.lang.String pythonFrameName, boolean debug) Transfer Weka instances into python as a pandas data frame and then extract out numpy arrays of input and target features/columns.
static void	main(java.lang.String[] args) Some quick tests...
static boolean	pythonAvailable() Returns true if (at least) the default python environment/server is available
static boolean	pythonAvailable(java.lang.String pythonCommand, java.lang.String ownerID) Returns true if the user-specified python environment/server (as specified by pythonCommand (and optional ownerID) is available.
static void	releaseSession(java.lang.Object requester) Release the default session so that other clients can obtain it.
static void	releaseSession(java.lang.String pythonCommand, java.lang.String ownerID, java.lang.Object requester) Release the user-specified python session.
void	setLog(weka.gui.Logger log) Set a log
void	setPythonPickledVariableValue(java.lang.String varName, java.lang.String varValue, boolean debug) Push a pickled python variable value back into python.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

acquireSession

public static PythonSession acquireSession(java.lang.Object requester)
                                    throws weka.core.WekaException

Acquire the default session for the requester

Parameters:

requester - the object requesting the session

Returns:

the default session singleton

Throws:

weka.core.WekaException - if python is not available

acquireSession

public static PythonSession acquireSession(java.lang.String pythonCommand,
                                           java.lang.String ownerID,
                                           java.lang.Object requester)
                                    throws weka.core.WekaException

Parameters:

pythonCommand - command (either fully qualified path or that which is in the PATH). This, plus the optional ownerID is used to lookup and return a session/server.

ownerID - an optional ownerID string for acquiring the session. This can be used to restrict the session/server to one (or more) clients (i.e. those with the ID "ownerID").

requester - the object requesting the session/server

Returns:

a session

Throws:

weka.core.WekaException - if the requested session/server is not available (or does not exist).

releaseSession

public static void releaseSession(java.lang.Object requester)

Release the default session so that other clients can obtain it. This method does nothing if the requester is not the current session holder

Parameters:

requester - the session holder

releaseSession

public static void releaseSession(java.lang.String pythonCommand,
                                  java.lang.String ownerID,
                                  java.lang.Object requester)
                           throws weka.core.WekaException

Release the user-specified python session.

Parameters:

pythonCommand - command (either fully qualified path or that which is in the PATH). This, plus the optional ownerID is used to lookup a session/server.

ownerID - an optional ownerID string for identifying the session. This can be used to restrict the session/server to one (or more) clients (i.e. those with the ID "ownerID").

requester - the object requesting the session/server

Throws:

weka.core.WekaException - if the requested session/server is not available (or does not exist).

pythonAvailable

public static boolean pythonAvailable()

Returns true if (at least) the default python environment/server is available

Returns:

true if the default python environment/server is available

pythonAvailable

public static boolean pythonAvailable(java.lang.String pythonCommand,
                                      java.lang.String ownerID)

Returns true if the user-specified python environment/server (as specified by pythonCommand (and optional ownerID) is available.

Parameters:

pythonCommand - command (either fully qualified path or that which is in the PATH). This, plus the optional ownerID is used to lookup a session/server.

ownerID - an optional ownerID string for identifying the session. This can be used to restrict the session/server to one (or more) clients (i.e. those with the ID "ownerID").

setLog

public void setLog(weka.gui.Logger log)

Set a log

Parameters:

log - the log to use

getPythonVariableType

public PythonSession.PythonVariableType getPythonVariableType(java.lang.String varName,
                                                              boolean debug)
                                                       throws weka.core.WekaException

Get the type of a variable in python

Parameters:

varName - the name of the variable to get the type for

debug - true for debugging output

Returns:

the type of the variable. Known types, for which we can do useful things with in the Weka environment, are pandas data frames (can be converted to instances), pyplot figure/images (retrieve as png) and textual data. Any variable of type unknown should be able to be retrieved in string form.

Throws:

weka.core.WekaException - if a problem occurs

instancesToPython

public void instancesToPython(weka.core.Instances instances,
                              java.lang.String pythonFrameName,
                              boolean debug)
                       throws weka.core.WekaException

Transfer Weka instances into python as a named pandas data frame

Parameters:

instances - the instances to transfer

pythonFrameName - the name of the data frame to use in python

debug - true for debugging output

Throws:

weka.core.WekaException - if a problem occurs

instancesToPythonAsScikitLearn

public void instancesToPythonAsScikitLearn(weka.core.Instances instances,
                                           java.lang.String pythonFrameName,
                                           boolean debug)
                                    throws weka.core.WekaException

Transfer Weka instances into python as a pandas data frame and then extract out numpy arrays of input and target features/columns. These arrays are named X and Y respectively in python. If there is no class set in the instances then only an X array is extracted.

Parameters:

instances - the instances to transfer

pythonFrameName - the name of the pandas data frame to use in python

debug - true for debugging output

Throws:

weka.core.WekaException - if a problem occurs

getDataFrameAsInstances

public weka.core.Instances getDataFrameAsInstances(java.lang.String frameName,
                                                   boolean debug)
                                            throws weka.core.WekaException

Retrieve a pandas data frame from Python and convert it to a set of instances. The resulting set of instances will not have a class index set.

Parameters:

frameName - the name of the pandas data frame to extract and convert to instances

debug - true for debugging output

Returns:

an Instances object

Throws:

weka.core.WekaException - if the named data frame does not exist in python or is not a pandas data frame

executeScript

public java.util.List<java.lang.String> executeScript(java.lang.String pyScript,
                                                      boolean debug)
                                               throws weka.core.WekaException

Execute an arbitrary script in python

Parameters:

pyScript - the script to execute

debug - true for debugging output

Returns:

a List of strings - index 0 contains std out from the script and index 1 contains std err

Throws:

weka.core.WekaException - if a problem occurs

checkIfPythonVariableIsSet

public boolean checkIfPythonVariableIsSet(java.lang.String varName,
                                          boolean debug)
                                   throws weka.core.WekaException

Check if a named variable is set/exists in the python environment

Parameters:

varName - the name of the variable to check

debug - true for debugging output

Returns:

true if the variable is set in python

Throws:

weka.core.WekaException - if a problem occurs

getVariableValueFromPythonAsJson

public java.lang.Object getVariableValueFromPythonAsJson(java.lang.String varName,
                                                         boolean debug)
                                                  throws weka.core.WekaException

Attempt to retrieve the value of a variable in python using serialization to Json. If successful, then the resulting Object is either a Map or List containing more Maps and Lists that represent the Json structure of the serialized variable

Parameters:

varName - the name of the variable to retrieve

debug - true for debugging output

Returns:

a Map/List based structure

Throws:

weka.core.WekaException - if a problem occurs

getVariableValueFromPythonAsPickledObject

public java.lang.String getVariableValueFromPythonAsPickledObject(java.lang.String varName,
                                                                  boolean debug)
                                                           throws weka.core.WekaException

Attempt to retrieve the value of a variable in python using pickle serialization. If successful, then the result is a string containing the pickled object.

Parameters:

varName - the name of the variable to retrieve

debug - true for debugging output

Returns:

a string containing the pickled variable value

Throws:

weka.core.WekaException - if a problem occurs

getVariableValueFromPythonAsPlainString

public java.lang.String getVariableValueFromPythonAsPlainString(java.lang.String varName,
                                                                boolean debug)
                                                         throws weka.core.WekaException

Attempt to retrieve the value of a variable in python as a plain string (i.e. executes a 'str(varName)' in python).

Parameters:

varName - the name of the variable to retrieve

debug - true for debugging output

Returns:

the value of the variable as a plain string

Throws:

weka.core.WekaException - if a problem occurs

getVariableListFromPython

public java.util.List<java.lang.String[]> getVariableListFromPython(boolean debug)
                                                             throws weka.core.WekaException

Get a list of variables that are set in python. Returns a list of two element string arrays. Each entry in the list is a variable. The first element of the array is the name of the variable and the second is its type in python.

Parameters:

debug - true if debugging info is to be output

Returns:

a list of variables set in python

Throws:

weka.core.WekaException - if a problem occurs

setPythonPickledVariableValue

public void setPythonPickledVariableValue(java.lang.String varName,
                                          java.lang.String varValue,
                                          boolean debug)
                                   throws weka.core.WekaException

Push a pickled python variable value back into python. Deserializes the value in python.

Parameters:

varName - the name of the variable in python that will hold the deserialized value

varValue - the pickled string value of the variable

debug - true for debugging output

Throws:

weka.core.WekaException - if a problem occurs

getPythonDebugBuffer

public java.util.List<java.lang.String> getPythonDebugBuffer(boolean debug)
                                                      throws weka.core.WekaException

Grab the contents of the debug buffer from the python server. The server redirects both sys out and sys err to StringIO objects. If debug has been specified, then server debugging output will have been collected in these buffers. Note that the buffers will potentially also contain output from the execution of arbitrary scripts too. Calling this method also resets the buffers.

Parameters:

debug - true for debugging output (from the execution of this specific command)

Returns:

the contents of the sys out and sys err streams. Element 0 in the list contains sys out and element 1 contains sys err

Throws:

weka.core.WekaException - if a problem occurs

getImageFromPython

public java.awt.image.BufferedImage getImageFromPython(java.lang.String varName,
                                                       boolean debug)
                                                throws weka.core.WekaException

Retrieve an image from python. Assumes that the image in python is stored as a matplotlib.figure.Figure object. Returns a BufferedImage containing the image.

Parameters:

varName - the name of the variable in python that contains the image

debug - true to output debugging info

Returns:

a BufferedImage

Throws:

weka.core.WekaException - if the variable doesn't exist, doesn't contain a Figure object or there is a comms error.

initSession

public static boolean initSession(java.lang.String pythonCommand,
                                  boolean debug)
                           throws weka.core.WekaException

Initialize the default session. This needs to be called exactly once in order to run checks and launch the server. Creates a session singleton.

Parameters:

pythonCommand - the python command

debug - true for debugging output

Returns:

true if the server launched successfully

Throws:

weka.core.WekaException - if there was a problem - missing packages in python, or python could not be started for some reason

initSession

public static boolean initSession(java.lang.String pythonCommand,
                                  java.lang.String ownerID,
                                  java.lang.String pathEntries,
                                  boolean debug)
                           throws weka.core.WekaException

Initialize a server/session for a user-supplied python path and (optional) ownerID.

Parameters:

pythonCommand - command (either fully qualified path or that which is in the PATH). This, plus the optional ownerID is used to lookup and return a session/server.

ownerID - an optional ownerID string for acquiring the session. This can be used to restrict the session/server to one (or more) clients (i.e. those with the ID "ownerID").

pathEntries - optional entries that need to be in the PATH in order for the python environment to work correctly

debug - true for debugging info

Returns:

true if the server launched successfully

Throws:

weka.core.WekaException - if the requested session/server is not available (or does not exist).

getPythonEnvCheckResults

public static java.lang.String getPythonEnvCheckResults()

Gets the result of running the checks in python

Returns:

a string containing the possible errors

getPythonEnvCheckResults

public static java.lang.String getPythonEnvCheckResults(java.lang.String pythonCommand,
                                                        java.lang.String ownerID)
                                                 throws weka.core.WekaException

Gets the result of running the checks in python for the given python path + optional ownerID.

Parameters:

pythonCommand - command (either fully qualified path or that which is in the PATH). This, plus the optional ownerID is used to lookup and return a session/server.

ownerID - an optional ownerID string for acquiring the session. This can be used to restrict the session/server to one (or more) clients (i.e. those with the ID "ownerID").

Returns:

a string containing the possible errors

Throws:

weka.core.WekaException - if the requested server does not exist

main

public static void main(java.lang.String[] args)

Some quick tests...

Parameters:

args -