Infer LDA models

0
votes

I'm new to LDA and topic modeling and I would like to understand the inference mechanism. I would like to apply LDA on activity recognition. Say that I have defined 10 topics composed by a probability distribution of events. for example

TOPIC_1 = event1 (0.5), event2 (0.4), event3 (0.0), event4 (0.0) and event5 (0.1).

I would like to understand which topics are active across the day of a person. One day of a person is composed by a sequence of events sampled every minutes.

What I'm doing to see which topic is active is:

  1. select 1 hour window in the daily sequence
  2. calculate the distribution of the events occurring in that hour
  3. calculate the similarity with each of the topics found by LDA.

Is that correct? Do you have any suggestion for the similarity function to use?

1
ldatopic-modeling
Frankly speaking, I don't think that's an LDA task but rather of machine learning. I know there is a notion of dynamic topic models and Boyd-Graber et al. followed the changes of topics within political debates (to see who is leading the debate and controls the agenda) but that's all computational linguistics. There must some better tools to address your issue, e.g. prediction models(?). - Everst
well.. yes it is a machine learning task, but LDA and topic modeling in general are tools for machine learning.. - gabboshow

1 Answers

0
votes

We can formulate a topic model for activity recognition by treating each hour as a document. We can then define the vocabulary as a discretized vector of counts of observed activity signals (i.e. we can discretize each activity measurement using k-means and count the number of times the activity occurred during each hour). The discretization level will determine the vocabulary size. This way we can construct the term-document matrix and apply tf-idf (see the original Blei 2003 LDA paper for details).

Having formed the term-document matrix, we can use one of many inference algorithms to learn the topics: variational bayes, EM algorithm, Gibbs sampler etc. Each has its own advantages. A commonly used inference algorithm for large corpora is online variational bayes (written in python by Matt Hoffman and available as part of scikit-learn and gensim)

After fitting the LDA model on training activity data, we can compute topic similarity for new test data by quantizing and transforming the data to our learned topic space (done in the E-step, implemented as transform method in scikit-learn). Once we obtain, the test data topic proportions, we can use one of many similarity measures, e.g. cosine similarity or symmetrized KL divergence to retrieve documents with similar topic proportions).

A note about modeling assumptions: LDA assumes that documents are independent (i.e. hourly activity time-series are independent). In addition, the words are exchangeable (i.e. discretized activity measurements can be permuted within a document).