Abstract
We demonstrate a means of knowledge discovery through feature extraction that exploits the search history of an optimization run. We regress a symbolic model ensemble from optimization run search points and their objective scores. The frequency of a variable in the models of the ensemble indicates to what the extent it is an influential feature. Our demonstration uses a genetic programming symbolic regression software package that is designed to be “off-the-shelf”. By default, the only parameter needed in order to evolve a suite of models is how long the user is willing to wait. Then the user can easily specify which models should go forward in terms of sufficient accuracy and complexity. For illustration purposes, we consider a common design heuristic in serial sensor sequencing: “place the most reliable sensor last”. The heuristic is derived based on the mathematical form of the objective function that lays emphasis on the decision variable pertaining to the last sensor. Feature extraction on optimized sensor sequences indicates that the heuristic is usually effective though it is not always trustworthy. This is consistent with knowledge in sensor processing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vladislavleva, E.: Model-based Problem Solving through Symbolic Regression via Pareto Genetic Programming. PhD thesis, Tilburg University, Tilburg, The Netherlands (2008)
Keijzer, M.: Scientific Discovery Using Genetic Programming. PhD thesis, Danish Technical University, Danish Technical University (2002)
Papastavrou, J., Athans, M.: Distributed detection by a large team of sensors in tandem. IEEE Transactions on Aerospace and Electronic Systems 28(3), 639–653 (1992)
Veeramachaneni, K., Osadciw, L.: Swarm intelligence based optimization and control of decentralized serial sensor networks, pp. 1–8 (2008)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)
Smits, G., Vladislavleva, E.: Ordinal pareto genetic programming. In: Yen, G.G., Lucas, S.M., Fogel, G., Kendall, G., Salomon, R., Zhang, B.T., Coello, C.A.C., Runarsson, T.P. (eds.) Proceedings of the 2006 IEEE Congress on Evolutionary Computation, Vancouver, BC, Canada, pp. 3114–3120. IEEE Press, Los Alamitos (2006)
Kotanchek, M., Smits, G., Vladislavleva, E.: Pursuing the pareto paradigm tournaments, algorithm variations & ordinal optimization. In: Riolo, R.L., Soule, T., Worzel, B. (eds.) Genetic Programming Theory and Practice IV. Genetic and Evolutionary Computation, vol. 5, pp. 167–186. Springer, Ann Arbor (2006)
Kotanchek, M., Smits, G., Vladislavleva, E.: Trustable symoblic regression models. In: Riolo, R.L., Soule, T., Worzel, B. (eds.) Genetic Programming Theory and Practice V. Genetic and Evolutionary Computation, pp. 203–222. Springer, Ann Arbor (2007)
Kotanchek, M., Smits, G., Vladislavleva, E.: Exploiting trustable models via pareto GP for targeted data collection. In: Riolo, R.L., Soule, T., Worzel, B. (eds.) Genetic Programming Theory and Practice VI. Genetic and Evolutionary Computation, pp. 145–163. Springer, Ann Arbor (2008)
Kotanchek, M.E., Vladislavleva, E.Y., Smits, G.F.: Symbolic regression via GP as a discovery engine: Insights on outliers and prototypes. In: Riolo, R.L., O’Reilly, U.M., McConaghy, T. (eds.) Genetic Programming Theory and Practice VII. Genetic and Evolutionary Computation, pp. 55–72. Springer, Ann Arbor (2009)
Vladislavleva, E., Smits, G., Kotanchek, M.: Soft evolution of robust regression models. In: Riolo, R.L., Soule, T., Worzel, B. (eds.) Genetic Programming Theory and Practice V. Genetic and Evolutionary Computation, pp. 13–32. Springer, Ann Arbor (2007)
Vladislavleva, E.J., Smits, G.F., den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Transactions on Evolutionary Computation 13(2), 333–349 (2009)
Smits, G., Kordon, A., Vladislavleva, K., Jordaan, E., Kotanchek, M.: Variable selection in industrial datasets using pareto genetic programming. In: Yu, T., Riolo, R.L., Worzel, B. (eds.) Genetic Programming Theory and Practice III. Genetic Programming, vol. 9, pp. 79–92. Springer, Ann Arbor (2005)
Research, W.: Wolfram mathematica overview: Compute and visualize key capabilities (2009), http://www.wolfram.com/products/mathematica/overview/compute.html
Wikipedia: Mathematica entry in wikipedia (2009), http://en.wikipedia.org/wiki/Mathematica
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Veeramachaneni, K., Vladislavleva, K., O’Reilly, UM. (2010). Feature Extraction from Optimization Data via DataModeler’s Ensemble Symbolic Regression. In: Blum, C., Battiti, R. (eds) Learning and Intelligent Optimization. LION 2010. Lecture Notes in Computer Science, vol 6073. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13800-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-13800-3_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13799-0
Online ISBN: 978-3-642-13800-3
eBook Packages: Computer ScienceComputer Science (R0)