The relationship between latent Dirichlet allocation and documents clustering

Question

I would like to clarify the relationship between latent Dirichlet allocation (LDA) and the generic task of document clustering.

The LDA analysis tends to output the topic proportions for each document. If my understanding is correct, this is not the direct result of document clustering. However, we can treat this probability proportions as a feature reprsentation for each document. Afterwards, we can invoke other established clustering method based on the feature configurations generated by LDA analysis.

Is my understanding correct? Thanks.

Answer 1

Yes, you can treat the output of LDA as features for your documents; this is exactly what Blei, Ng and Jordan did in the paper that introduced LDA. They did it for classification, but for clustering the procedure is the same.

(In machine learning terminology, this use of LDA is called dimensionality reduction because it reduces the feature space's number of dimensions from |V|, the vocabulary size, to some number k of topics selected by the user.)