Advertisement

Dynamic Multi-Objective Optimization with jMetal and Spark: A Case Study

  • José A. Cordero
  • Antonio J. NebroEmail author
  • Cristóbal Barba-González
  • Juan J. Durillo
  • José García-Nieto
  • Ismael Navas-Delgado
  • José F. Aldana-Montes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10122)

Abstract

Technologies for Big Data and Data Science are receiving increasing research interest nowadays. This paper introduces the prototyping architecture of a tool aimed to solve Big Data Optimization problems. Our tool combines the jMetal framework for multi-objective optimization with Apache Spark, a technology that is gaining momentum. In particular, we make use of the streaming facilities of Spark to feed an optimization problem with data from different sources. We demonstrate the use of our tool by solving a dynamic bi-objective instance of the Traveling Salesman Problem (TSP) based on near real-time traffic data from New York City, which is updated several times per minute. Our experiment shows that both jMetal and Spark can be integrated providing a software platform to deal with dynamic multi-optimization problems.

Keywords

Multi-objective optimization Dynamic optimization problem Big data technologies Spark Streaming processing jMetal 

Notes

Acknowledgments

This work is partially funded by Grants TIN2011-25840 (Ministerio de Ciencia e Innovación) and P11-TIC-7529 and P12-TIC-1519 (Plan Andaluz de Investigación, Desarrollo e Innovación). Cristóbal Barba-González is supported by Grant BES-2015-072209 (Ministerio de Economía y Competitividad).

References

  1. 1.
    Editorial: Community cleverness required. Nature 455, 1 (2008)Google Scholar
  2. 2.
    White, T.: Hadoop: The Definitive Guide, 1st edn. O’Reilly Media Inc., Sebastopol (2009)Google Scholar
  3. 3.
    Zaharia, M., Chowdhury, M., Franklin, M., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, Berkeley, CA, USA, HotCloud 2010, pp. 10. USENIX Association (2010)Google Scholar
  4. 4.
    Marr, M.: Big Data: Using SMART Big Data Analytics and Metrics to Make Better Decisions and Improve Performance. Wiley, Hoboken (2015)Google Scholar
  5. 5.
    Nam, T., Pardo, T.: Smart city as urban innovation: focusing on management, policy, and context. In: Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance, ICEGOV 2011, pp. 185–194. ACM (2011)Google Scholar
  6. 6.
    Garcia-Nieto, J., Olivera, A., Alba, E.: Optimal cycle program of traffic lights with particle swarm optimization. IEEE Trans. Evol. Comput. 17, 823–839 (2013)CrossRefGoogle Scholar
  7. 7.
    NYCDOT: New York City traffic speed detectors data set (2016). http://nyctmc.org
  8. 8.
    Papadimitriou, C.H.: The Euclidean travelling salesman problem is NP-complete. Theor. Comput. Sci. 4, 237–244 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gartner Inc.: Gartner’s hype cycle for advanced analytics and data science (2015). https://www.gartner.com/doc/3087721/hype-cycle-advanced-analytics-data
  10. 10.
    Durillo, J., Nebro, A.: jMetal: a java framework for multi-objective optimization. Adv. Eng. Softw. 42, 760–771 (2011)CrossRefGoogle Scholar
  11. 11.
    Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, New York (2001)zbMATHGoogle Scholar
  12. 12.
    Coello, C., Lamont, G., van Veldhuizen, D.: Multi-objective Optimization Using Evolutionary Algorithms, 2nd edn. Wiley, New York (2007)zbMATHGoogle Scholar
  13. 13.
    Farina, M., Deb, K., Amato, P.: Dynamic multiobjective optimization problems: test cases, approximations, and applications. IEEE Trans. Evol. Comput. 8, 425–442 (2004)CrossRefzbMATHGoogle Scholar
  14. 14.
    Nebro, A., Durillo, J.J., Vergne, M.: Redesigning the jMetal multi-objective optimization framework. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 1093–1100. ACM, New York (2015)Google Scholar
  15. 15.
    Reinelt, G.: TSPLIB - a traveling salesman problem library. INFORMS J. Comput. 3, 376–384 (1991)CrossRefzbMATHGoogle Scholar
  16. 16.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6, 182–197 (2002)CrossRefGoogle Scholar
  17. 17.
    Google Inc.: Encoded polyline algorithm format (2016). https://developers.google.com/maps/documentation/utilities/polylinealgorithm
  18. 18.
    Google Inc.: Google maps distance matrix API (2016). https://developers.google.com/maps/documentation/distance-matrix

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • José A. Cordero
    • 1
  • Antonio J. Nebro
    • 2
    Email author
  • Cristóbal Barba-González
    • 2
  • Juan J. Durillo
    • 3
  • José García-Nieto
    • 2
  • Ismael Navas-Delgado
    • 2
  • José F. Aldana-Montes
    • 2
  1. 1.European Organization for Nuclear Research (CERN)GenevaSwitzerland
  2. 2.Khaos Research Group, Ada Byron Research Building, Departamento de Lenguajes y Ciencias de la ComputaciónUniversity of MálagaMálagaSpain
  3. 3.Distributed and Parallel Systems GroupUniversity of InnsbruckInnsbruckAustria

Personalised recommendations