| Author: | | Mark Hall <mhall{[at]}pentaho.com> |
| Category: | | Distributed |
| Changes: | | Now replaces the arpack_combined_all.jar that Spark includes with the 'clean' version from Weka that does not contain the core java CJFormat class in org.j_paine.formatter which is not usable for commercial purposes |
| Date: | | 2018-02-28 |
| Depends: | | weka (>=3.8.1-SNAPSHOT|<3.9.0), distributedWekaBase (>=1.0.17) |
| Description: | | Development continuation of distributedWekaSpark. Adds access to source files (CSV, Parquet and Avro) via Spark DataFrames, and adds support for both desktop and distributed execution of MLlib algorithms. Provides Spark wrappers for the classes in distributedWekaBase. Includes generic Spark 1.6.3 libraries - To run against Hadoop/HDFS, it is necessary to delete all the libraries in ${user.home}/wekafiles/distributedWekaSpark/lib and copy in the spark-assembly-A.B.C-hadoopX.Y-Z.jar file that is bundled with the distribution of Spark compiled for your version of Hadoop. |
| License: | | GPL 3 |
| Maintainer: | | Mark Hall <mhall{[at]}pentaho.com> |
| MessageToDisplayOnInstall: | | Includes generic Spark 1.6.3 libraries, which are sufficent for running local mode on the local filesystem out of the box. To run against Hadoop/HDFS, it is necessary to delete all the libraries in ${WEKA_HOME}/distributedWekaSpark/lib and copy in the spark-assembly-a.b.c-hadoopX.Y-Z.jar file that is bundled with the distribution of Spark compiled for your version of Hadoop. |
| PackageURL: | | http://downloads.sourceforge.net/weka/weka-packages/distributedWekaSparkDev1.0.2.zip |
| Precludes: | | distributedWekaSpark |
| URL: | | http://markahall.blogspot.co.nz/2017/07/integrating-spark-mllib-into-weka.html |
| Version: | | 1.0.2 |