public class PairedTTester extends java.lang.Object implements OptionHandler, Tester, RevisionHandler
-D <index,index2-index4,...> Specify list of columns that specify a unique dataset. First and last are valid indexes. (default none)
-R <index> Set the index of the column containing the run number
-F <index> Set the index of the column containing the fold number
-G <index1,index2-index4,...> Specify list of columns that specify a unique 'result generator' (eg: classifier name and options). First and last are valid indexes. (default none)
-S <significance level> Set the significance level for comparisons (default 0.05)
-V Show standard deviations
-L Produce table comparisons in Latex table format
-csv Produce table comparisons in CSV table format
-html Produce table comparisons in HTML table format
-significance Produce table comparisons with only the significance values
-gnuplot Produce table comparisons output suitable for GNUPlot
Constructor and Description |
---|
PairedTTester() |
Modifier and Type | Method and Description |
---|---|
void |
assign(Tester tester)
retrieves all the settings from the given Tester
|
PairedStats |
calculateStatistics(Instance datasetSpecifier,
int resultset1Index,
int resultset2Index,
int comparisonColumn)
Computes a paired t-test comparison for a specified dataset between
two resultsets.
|
boolean |
displayResultset(int index)
Checks whether the resultset with the given index shall be displayed.
|
Range |
getDatasetKeyColumns()
Get the value of DatasetKeyColumns.
|
int[] |
getDisplayedResultsets()
Gets the indices of the the datasets that are displayed (if
null
then all are displayed). |
java.lang.String |
getDisplayName()
returns the name of the tester
|
int |
getFoldColumn()
Get the value of FoldColumn.
|
Instances |
getInstances()
Get the value of Instances.
|
int |
getNumDatasets()
Gets the number of datasets in the resultsets
|
int |
getNumResultsets()
Gets the number of resultsets in the data.
|
java.lang.String[] |
getOptions()
Gets current settings of the PairedTTester.
|
ResultMatrix |
getResultMatrix()
Gets the instance that produces the output.
|
Range |
getResultsetKeyColumns()
Get the value of ResultsetKeyColumns.
|
java.lang.String |
getResultsetName(int index)
Gets a string descriptive of the specified resultset.
|
java.lang.String |
getRevision()
Returns the revision string.
|
int |
getRunColumn()
Get the value of RunColumn.
|
boolean |
getShowStdDevs()
Returns true if standard deviations have been requested.
|
double |
getSignificanceLevel()
Get the value of SignificanceLevel.
|
int |
getSortColumn()
Returns the column to sort on, -1 means the default sorting.
|
java.lang.String |
getSortColumnName()
Returns the name of the column to sort on.
|
java.lang.String |
getToolTipText()
returns a string that is displayed as tooltip on the "perform test"
button in the experimenter
|
java.lang.String |
header(int comparisonColumn)
Creates a "header" string describing the current resultsets.
|
java.util.Enumeration |
listOptions()
Lists options understood by this object.
|
static void |
main(java.lang.String[] args)
Test the class from the command line.
|
java.lang.String |
multiResultsetFull(int baseResultset,
int comparisonColumn)
Creates a comparison table where a base resultset is compared to the
other resultsets.
|
java.lang.String |
multiResultsetRanking(int comparisonColumn)
returns a ranking of the resultsets
|
java.lang.String |
multiResultsetSummary(int comparisonColumn)
Carries out a comparison between all resultsets, counting the number
of datsets where one resultset outperforms the other.
|
int[][] |
multiResultsetWins(int comparisonColumn,
int[][] nonSigWin)
Carries out a comparison between all resultsets, counting the number
of datsets where one resultset outperforms the other.
|
java.lang.String |
resultsetKey()
Creates a key that maps resultset numbers to their descriptions.
|
void |
setDatasetKeyColumns(Range newDatasetKeyColumns)
Set the value of DatasetKeyColumns.
|
void |
setDisplayedResultsets(int[] cols)
Sets the indicies of the datasets to display (
null means all). |
void |
setFoldColumn(int newFoldColumn)
Set the value of FoldColumn.
|
void |
setInstances(Instances newInstances)
Set the value of Instances.
|
void |
setOptions(java.lang.String[] options)
Parses a given list of options.
|
void |
setResultMatrix(ResultMatrix matrix)
Sets the matrix to use to produce the output.
|
void |
setResultsetKeyColumns(Range newResultsetKeyColumns)
Set the value of ResultsetKeyColumns.
|
void |
setRunColumn(int newRunColumn)
Set the value of RunColumn.
|
void |
setShowStdDevs(boolean s)
Set whether standard deviations are displayed or not.
|
void |
setSignificanceLevel(double newSignificanceLevel)
Set the value of SignificanceLevel.
|
void |
setSortColumn(int newSortColumn)
Set the column to sort on, -1 means the default sorting.
|
public void setResultMatrix(ResultMatrix matrix)
setResultMatrix
in interface Tester
matrix
- the instance to use to produce the outputResultMatrix
public ResultMatrix getResultMatrix()
getResultMatrix
in interface Tester
public void setShowStdDevs(boolean s)
setShowStdDevs
in interface Tester
s
- true if standard deviations are to be displayedpublic boolean getShowStdDevs()
getShowStdDevs
in interface Tester
public int getNumDatasets()
getNumDatasets
in interface Tester
public int getNumResultsets()
getNumResultsets
in interface Tester
public java.lang.String getResultsetName(int index)
getResultsetName
in interface Tester
index
- the index of the resultsetpublic boolean displayResultset(int index)
displayResultset
in interface Tester
index
- the index of the resultset to check whether it shall be displayedpublic PairedStats calculateStatistics(Instance datasetSpecifier, int resultset1Index, int resultset2Index, int comparisonColumn) throws java.lang.Exception
calculateStatistics
in interface Tester
datasetSpecifier
- the dataset specifierresultset1Index
- the index of the first resultsetresultset2Index
- the index of the second resultsetcomparisonColumn
- the column containing values to comparejava.lang.Exception
- if an error occurspublic java.lang.String resultsetKey()
resultsetKey
in interface Tester
public java.lang.String header(int comparisonColumn)
public int[][] multiResultsetWins(int comparisonColumn, int[][] nonSigWin) throws java.lang.Exception
multiResultsetWins
in interface Tester
comparisonColumn
- the index of the comparison columnnonSigWin
- for storing the non-significant winsjava.lang.Exception
- if an error occurspublic java.lang.String multiResultsetSummary(int comparisonColumn) throws java.lang.Exception
multiResultsetSummary
in interface Tester
comparisonColumn
- the index of the comparison columnjava.lang.Exception
- if an error occurspublic java.lang.String multiResultsetRanking(int comparisonColumn) throws java.lang.Exception
multiResultsetRanking
in interface Tester
comparisonColumn
- the column to compare withjava.lang.Exception
- if something goes wrongpublic java.lang.String multiResultsetFull(int baseResultset, int comparisonColumn) throws java.lang.Exception
multiResultsetFull
in interface Tester
baseResultset
- the index of the base resultsetcomparisonColumn
- the index of the column to compare overjava.lang.Exception
- if an error occurspublic java.util.Enumeration listOptions()
listOptions
in interface OptionHandler
public void setOptions(java.lang.String[] options) throws java.lang.Exception
-D <index,index2-index4,...> Specify list of columns that specify a unique dataset. First and last are valid indexes. (default none)
-R <index> Set the index of the column containing the run number
-F <index> Set the index of the column containing the fold number
-G <index1,index2-index4,...> Specify list of columns that specify a unique 'result generator' (eg: classifier name and options). First and last are valid indexes. (default none)
-S <significance level> Set the significance level for comparisons (default 0.05)
-V Show standard deviations
-L Produce table comparisons in Latex table format
-csv Produce table comparisons in CSV table format
-html Produce table comparisons in HTML table format
-significance Produce table comparisons with only the significance values
-gnuplot Produce table comparisons output suitable for GNUPlot
setOptions
in interface OptionHandler
options
- an array containing options to set.java.lang.Exception
- if invalid options are givenpublic java.lang.String[] getOptions()
getOptions
in interface OptionHandler
public Range getResultsetKeyColumns()
getResultsetKeyColumns
in interface Tester
public void setResultsetKeyColumns(Range newResultsetKeyColumns)
setResultsetKeyColumns
in interface Tester
newResultsetKeyColumns
- Value to assign to ResultsetKeyColumns.public int[] getDisplayedResultsets()
null
then all are displayed). The base is always displayed.getDisplayedResultsets
in interface Tester
public void setDisplayedResultsets(int[] cols)
null
means all).
The base is always displayed.setDisplayedResultsets
in interface Tester
cols
- the indices of the datasets to displaypublic double getSignificanceLevel()
getSignificanceLevel
in interface Tester
public void setSignificanceLevel(double newSignificanceLevel)
setSignificanceLevel
in interface Tester
newSignificanceLevel
- Value to assign to SignificanceLevel.public Range getDatasetKeyColumns()
getDatasetKeyColumns
in interface Tester
public void setDatasetKeyColumns(Range newDatasetKeyColumns)
setDatasetKeyColumns
in interface Tester
newDatasetKeyColumns
- Value to assign to DatasetKeyColumns.public int getRunColumn()
getRunColumn
in interface Tester
public void setRunColumn(int newRunColumn)
setRunColumn
in interface Tester
newRunColumn
- Value to assign to RunColumn.public int getFoldColumn()
getFoldColumn
in interface Tester
public void setFoldColumn(int newFoldColumn)
setFoldColumn
in interface Tester
newFoldColumn
- Value to assign to FoldColumn.public java.lang.String getSortColumnName()
getSortColumnName
in interface Tester
public int getSortColumn()
getSortColumn
in interface Tester
public void setSortColumn(int newSortColumn)
setSortColumn
in interface Tester
newSortColumn
- the new sort column.public Instances getInstances()
getInstances
in interface Tester
public void setInstances(Instances newInstances)
setInstances
in interface Tester
newInstances
- Value to assign to Instances.public void assign(Tester tester)
public java.lang.String getToolTipText()
getToolTipText
in interface Tester
public java.lang.String getDisplayName()
getDisplayName
in interface Tester
public java.lang.String getRevision()
getRevision
in interface RevisionHandler
public static void main(java.lang.String[] args)
args
- contains options for the instance ttests