Advertisement

Simulated Annealing Based Algorithm for Tuning LDA Hyper Parameters

  • Nikhlesh PathikEmail author
  • Pragya Shukla
Conference paper
  • 22 Downloads
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1154)

Abstract

LDA is a very popular unsupervised model used to find thematic information about the documents. The performance of the LDA greatly depends on its hyper parameter values. If its parameters are tuned properly then LDA may produce much better results. This paper mainly focused on finding good LDA configurations for finding improved LDA output as compare to the traditional LDA model. We have proposed and implemented SA-LDA algorithm that uses Simulated Annealing (SA) to find optimal values of LDA parameters. An empirical evaluation using customer review datasets from three different domains namely Mobile, Hotel and Movie is conducted. The experiment results show that SA-LDA gives better performance when it is evaluated by the coherence score.

Keywords

Hyper parameters LDA Simulated annealing Reviews 

References

  1. 1.
    Blei, D., Carin, L., Dunson, D.: Probabilistic topic models. IEEE Signal Process. Mag. 27(6), 55–65 (2010)Google Scholar
  2. 2.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(1), 993–1022 (2003)zbMATHGoogle Scholar
  3. 3.
    Yarnguy, T., Kanarkard, W.: Tuning latent Dirichlet allocation parameters using ant colony optimization. J. Telecommun. Electron. Comput. Eng. (JTEC) 10(1–9), 21–4 (2018)Google Scholar
  4. 4.
    Dit, B., Panichella, A., Moritz, E., Oliveto, R., Di Penta, M., Poshyvanyk, D., De Lucia, A.: Configuring topic models for software engineering tasks in Tracelab. In: 2013 7th International Workshop on Traceability in Emerging Forms of Software Engineering (TEFSE), IEEE, pp. 105–109 (2013)Google Scholar
  5. 5.
    Bandyopadhyay, S., Saha, S., Maulik, U., Deb, K.: A simulated annealing-based multiobjective optimization algorithm: AMOSA. IEEE Trans. Evol. Comput. 12(3), 269–283 (2008)CrossRefGoogle Scholar
  6. 6.
    Gultekin, S., Zhang, A., Paisley, J.: Asymptotic simulated annealing for variational inference. In: IEEE Global Communications Conference (GLOBECOM), IEEE, pp. 1–7 (2018)Google Scholar
  7. 7.
    Foulds, J.R., Smyth, P.: Annealing Paths for the Evaluation of Topic Models. UAI, pp. 220–229 (2014)Google Scholar
  8. 8.
    Elhaddad, Y.R.: Combined simulated annealing and genetic algorithm to solve optimization problems. World Acad. Sci. Eng. Technol. 68, 1508–1510 (2012)Google Scholar
  9. 9.
    Zhao, W., Chen, J.J., Perkins, R., Liu, Z., Ge, W., Ding, Y., Zou, W.: A Heuristic approach to determine an appropriate number of topics in topic modeling. In: BMC Bioinformatics, vol. 16, no. 13, p. S8. BioMed Central (2015)Google Scholar
  10. 10.
    George, C.P., Doss, H.: Principled selection of hyper-parameters in the latent Dirichlet allocation model. J. Mach. Learn. Res. 18, 162 (2017)zbMATHGoogle Scholar
  11. 11.
    Gultekin, S., Zhang, A., Paisley, J.: Stochastic annealing for variational inference. arXiv:1505.06723 (2015)
  12. 12.
    Kuzmenko, A.: Gibbs sampling for LDA with asymmetric Dirichlet priors. akuz.me/wp-content/uploads/2014/01/akuz_lda_asym.pdf (2011)
  13. 13.
    Wallach, H.M., Mimno, D.M., McCallum, A.: Rethinking LDA: Why priors matter. In: Advances in Neural Information Processing Systems, pp. 1973–1981 (2009)Google Scholar
  14. 14.
    Pathik, N., Shukla, P.: An Ample analysis on extended LDA models for aspect based review analysis. Int. J. Comput. Sci. Appl. 14(2), 97–106 (2017)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Institute of Engineering & Technology, Devi Ahilya UniversityIndoreIndia

Personalised recommendations