Levenshtein distance affinity propagation prediction query

Question

I do not know much about the affinity propagation as a concept, but in my project I found it useful to cluster the texts that I am working with. Currently I have extensively followed whatever code snippets I could find online.

So:

lev_similarity = -1*np.array([[Levenshtein.distance(w1,w2) for w1 in new_data] for w2 in new_data])
ap = sklearn.cluster.AffinityPropagation(affinity="euclidean", damping=0.5)
ap.fit(lev_similarity)

as you can see, I am using Levenshtein algorithm to define the distance between two points.

Now for my question:

http://scikit-learn.org/stable/modules/generated/sklearn.cluster.AffinityPropagation.html

At this point I want to use this model to give me the closest cluster of a new point, but I do not know how to vectorize my raw string. In order to use ap.predict method, how do I turn a string into the appropriate form in this case?
Or would it make more sense to input all of my data into fitting the model, and then querying with the completed model? In this case, how would I exactly query for the closest cluster of a given keyword in string?

EDIT:

Clearly I am working with pre-computed distance measure, so having affinity="precomputed" seems the right option. In that case,

lev_similarity = -1*np.array([[Levenshtein.distance(w1,w2) for w1 in new_data] for w2 in new_data])
ap = sklearn.cluster.AffinityPropagation(affinity="precomputed", damping=0.5)
ap.fit(lev_similarity)

where new_data encapsulates testing data as well.

Then the question becomes: how do I query with this model?

postoronnim postoronnim · Accepted Answer · 2016-11-29T20:24:48

To your first question, the documentation link that you provided says that 'predict' method accepts and returns and array - which is a list. In the User Guide (link in the doc), I found this example of input/output :

    labels_true = [0, 0, 0, 1, 1, 1]
    labels_pred = [0, 0, 1, 1, 2, 2].

So, if you already have a string with data to model, just convert it to a list.

The doc also describes what methods of AffinityPropagation class return to answer your second question.

Levenshtein distance affinity propagation prediction query

1 Answers