How to choose the initial clusters for K-mean from Tf-IDF vectors

0
votes

I'm working with text clustering. I want to select specific documents (as a vector) to be a centroID fo k-means.

I have created the TF-IDF for my dataset by using Mahout, and I would like to choose the initial clusters from TFIDF vectors.

Anyone has an idea how I can specify the initial centroids in Mahout?

2
cluster-analysismahoutk-meanstext-miningtf-idf
Mahout has built-in selection of centroids, doesn't it? – Has QUIT--Anony-Mousse
Yes, Mahout can select the centroid randomlly or by using Canopy, but I would like to select them manually. – Darsh
It also has an option to specify them manually. – Has QUIT--Anony-Mousse

2 Answers

0
votes

bin/mahout kmeans
-c input clusters directory
-k optional number of initial clusters to sample from input vectors

If the -k argument is supplied, any clusters in the -c directory will be overwritten and -k random points will be sampled from the input vectors to become the initial cluster centers.

Reference: https://mahout.apache.org/users/clustering/k-means-clustering.html

0
votes

One possibility could be using Cosine similarity instead of TF-IDF, by looking at documents that are the farthest away from one another. Something like this:

  1. Pick a document 1.
  2. Pick the farthest document 2 from document 1.
  3. Pick the farthest document from documents 1 and 2.
  4. etc

Taking a look at this might help as well.