Advertisement

Mapping the Energy Landscape of Non-convex Optimization Problems

  • Maira Pavlovskaia
  • Kewei Tu
  • Song-Chun Zhu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8932)

Abstract

An energy landscape map (ELM) characterizes and visualizes an energy function with a tree structure, in which each leaf node represents a local minimum and each non-leaf node represents the barrier between adjacent energy basins. We demonstrate the utility of ELMs in analyzing non-convex energy minimization problems with two case studies: clustering with Gaussian mixture models and learning mixtures of Bernoulli templates from images. By plotting the ELMs, we are able to visualize the impact of different problem settings on the energy landscape as well as to examine and compare the behaviors of different learning algorithms on the ELMs.

Keywords

Local Minimum Leaf Node Gaussian Mixture Model Energy Landscape Hypothesis Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Barbu, A., Zhu, S.C.: Generalizing swendsen-wang to sampling arbitrary posterior probabilities. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 1239–1253 (2005)CrossRefGoogle Scholar
  2. 2.
    Becker, O.M., Karplus, M.: The topology of multidimensional potential energy surfaces: Theory and application to peptide structure and kinetics. The Journal of Chemical Physics 106(4), 1495–1517 (1997)CrossRefGoogle Scholar
  3. 3.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  4. 4.
    Dasgupta, S., Schulman, L.J.: A two-round variant of em for gaussian mixtures. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, UAI 2000, pp. 152–159. Morgan Kaufmann Publishers Inc., San Francisco (2000)Google Scholar
  5. 5.
    Gelman, A., Rubin, D.B.: Inference from Iterative Simulation Using Multiple Sequences. Statistical Science 7(4), 457–472 (1992)CrossRefGoogle Scholar
  6. 6.
    Liang, F.: Generalized wang-landau algorithm for monte carlo computation. Journal of the American Statistical Association 100, 1311–1327 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Liang, F.: A generalized wang-landau algorithm for monte carlo computation. JASA. Journal of the American Statistical Association 100(472), 1311–1327 (2005)CrossRefzbMATHGoogle Scholar
  8. 8.
    Pavlovskaia, M., Tu, K., Zhu, S.C.: Mapping energy landscapes of non-convex learning problems. arXiv preprint arXiv:1410.0576 (2014)Google Scholar
  9. 9.
    Potts, R.B.: Some Generalized Order-Disorder Transformation. Transformations, Proceedings of the Cambridge Philosophical Society 48, 106–109 (1952)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Samdani, R., Chang, M.W., Roth, D.: Unified expectation maximization. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 688–698. Association for Computational Linguistics (2012)Google Scholar
  11. 11.
    Swendsen, R.H., Wang, J.S.: Nonuniversal critical dynamics in Monte Carlo simulations. Physical Review Letters 58(2), 86–88 (1987)CrossRefGoogle Scholar
  12. 12.
    Tu, K., Honavar, V.: Unambiguity regularization for unsupervised learning of probabilistic grammars. In: Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing and Natural Language Learning, EMNLP-CoNLL 2012 (2012)Google Scholar
  13. 13.
    Wang, F., Landaul, D.: Efficient multi-range random-walk algorithm to calculate the density of states. Phys. Rev. Lett. 86, 2050–2053 (2001)CrossRefGoogle Scholar
  14. 14.
    Zhou, Q.: Random walk over basins of attraction to construct ising energy landscapes. Phys. Rev. Lett. 106, 180602 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Maira Pavlovskaia
    • 1
  • Kewei Tu
    • 2
  • Song-Chun Zhu
    • 1
  1. 1.Department of StatisticsUniversity of California, Los AngelesLos AngelesUSA
  2. 2.School of Information Science and TechnologyShanghai Tech UniversityShanghaiChina

Personalised recommendations