10.5 Fitting the LDA

Now we are ready to fit LDA on our text. For this we will use dtm. We have to decide on the number of topics. As this is a hyperparameter we have to tune, we can use a grid search. The optimum value of number of topics will give us the maximum likelihood. I don’t go into the details of this, but if you are interested, I recommend checking out Martin Ponweiser’s thesis where he provides the method and the R code to do this [PDF]: http://epub.wu.ac.at/3558/1/main.pdf.

LDA() gives us the method choice between Variational Expectation-Maximization (VEM) algorithm and Gibb’s sampler. We will use the latter. We select alpha equal to 0.2. Lower values of alpha lead to selecting only a few topics per document. Higher values of alpha give us diffused distribution. This is also a hyperparameter you might want to tune using grid search. We select 2,000 iterations out of which the first 1,000 are not used (that’s why “burn-in”).

The following code took about 30 seconds on my computer.

lda_model <- LDA(x = dtm,
                 k = 20,
                 method = "Gibbs",
                 control = list(seed = 5648,
                                alpha = 0.2,
                                iter = 2000,
                                burnin = 1000)
                 )