How to successfully run kmeans clustering using Mahout (esp. get human-readable output)

0
votes

I tried to follow many online tutorials to run kmeans example present in Mahout. But did not succeed yet to get meaningful output. The main problem I am facing is, the conversion from text file to sequencefile and back.

  1. When I followed the steps of "Mahout Wiki" for "Clustering of synthetic control data" (https://cwiki.apache.org/MAHOUT/clustering-of-synthetic-control-data.html) I could run the clustering process (using $MAHOUT_HOME/bin/mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.Job) and that created some readable console output. But I wish to get output files (as the size is large) from the clustering process. The output files which were generated by Mahout clustering are all sequence file and I cant convert them to readable files. When I tried to do "clusterdump" ($MAHOUT_HOME/bin/mahout clusterdump --seqFileDir output/clusters-10...) I got errors. First it complains that "seqFileDir" option is unexpected and I guess either there is no "seqFileDir" for clusterdump or I am missing something.

  2. Trying to use Mahout in the way of "mahout in action" seems tricky. I am not sure what are the required classes ("import ??") to compile that code.

Can you please suggest me the steps to successfully RUN kmeans on Mahout ? Specially how to get readable output from sequence files ?

2
mavenhadoopcluster-analysismahoutk-means

2 Answers

1
votes

Regarding 2nd question - you can obtain source code for the book from the repository. The code in master branch is for Mahout 0.5, while code in the branches mahout-0.6 & mahout-0.7 is for corresponding Mahout's version.

The source code is also posted to book's site, so you download it there (but this is version only for Mahout 0.5)

P.S. If you're reading book right now, then I recommend to use Mahout 0.5 or 0.6, as all code was checked for version 0.5, while for other versions it will be different - this is especially true for clustering code in Mahout 0.7

0
votes

As for seqFileDir in clusterdump, you need to use --input not --seqFileDir.

I'm using Mahout 0.7. The call to clusterdump that i use to (for example) get a simple dump is:

mahout clusterdump --input output/clusters-9-final --pointsDir output/clusteredPoints --output <absolute path of dir where you want to output>/clusteranalyze.txt

Be sure that the path to the directory output/clusters-9-final above is correct for your system. Depending on the clustering algorithm, this directory may be different. Look in the output directory and make sure you use the directory with the word "final" init.

To dump data as CSV or GRAPH_ML, you would add the -of CSV argument to the above call. For e.g.: 

mahout clusterdump --input output/clusters-9-final -of CSV --pointsDir output/clusteredPoints --output <absolute path of dir where you want to output>/clusteranalyze.txt

Hope that helps.