Skip to main content
SpringerLink
Log in

Regression Error Characteristic Optimisation of Non-Linear Models

  • Chapter
Multi-Objective Machine Learning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 16))

Abstract

In this chapter recent research in the area of multi-objective optimisation of regression models is presented and combined. Evolutionary multi-objective optimisation techniques are described for training a population of regression models to optimise the recently defined Regression Error Characteristic Curves (REC). A method which meaningfully compares across regressors and against benchmark models (i.e. ‘random walk’ and maximum a posteriori approaches) for varying error rates. Through bootstrapping training data, degrees of confident out-performance are also highlighted.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J.S. Armstrong and F. Collopy. Error measures for generalizing about forecasting methods: Empirical comparisons. International Journal of Forecasting, 8(1):69–80, 1992.

    Article  Google Scholar 

  2. J. Bi and K.P. Bennett. Regression Error Characteristic Curves. In Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), pages 43–50, Washington DC, 2003.

    Google Scholar 

  3. C.M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1998.

    Google Scholar 

  4. C.A. Coello Coello. A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques. Knowledge and Information Systems. An International Journal, 1(3):269–308, 1999.

    Google Scholar 

  5. I. Das and J. Dennis. A closer look at drawbacks of minimizing weighted sums of objectives for pareto set generation in multicriteria optimization problems. Structural Optimization, 14(1):63–69, 1997.

    Article  Google Scholar 

  6. K. Deb. Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, Chichester, 2001.

    MATH  Google Scholar 

  7. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. Fast and elitist multiobjective genetic algorithm: NSGA–II,. IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002.

    Article  Google Scholar 

  8. B. Efron and R.J. Tibshirani. An Introduction to the Bootstrap. Number 57 in Monographs on Statistics and Probability. Chapman & Hall, New York, 1993.

    MATH  Google Scholar 

  9. R.M. Everson, J.E. Fieldsend, and S. Singh. Full Elite Sets for Multi-Objective Optimisation. In I.C. Parmee, editor, Adaptive Computing in Design and Manufacture V, pages 343–354. Springer, 2002.

    Google Scholar 

  10. J.E. Fieldsend and R.M. Everson. ROC Optimisation of Safety Related Systems. In J. Hernández-Orallo, C. Ferri, N. Lachiche, and P. Flach, editors, Proceedings of ROCAI 2004, part of the 16th European Conference on Artificial Intelligence (ECAI), pages 37–44, Valencia, Spain, 2004.

    Google Scholar 

  11. J.E. Fieldsend and R.M. Everson. Multi-objective Optimisation in the Presence of Uncertainty. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’05), 2005. Forthcoming.

    Google Scholar 

  12. J.E. Fieldsend, R.M. Everson, and S. Singh. Using Unconstrained Elite Archives for Multi-Objective Optimisation. IEEE Transactions on Evolutionary Computation, 7(3):305–323, 2003.

    Article  Google Scholar 

  13. J.E. Fieldsend, J. Matatko, and M. Peng. Cardinality constrained portfolio optimisation. In Z.R. Yang, R. Everson, and H. Yin, editors, Proceedings of the Fifth International Conference on Intelligent Data Engineering and Automated Learning (IDEAL’04), number 3177 in Lecture Notes in Computer Science, pages 788–793. Springer, 2004.

    Google Scholar 

  14. J.E. Fieldsend and S. Singh. A Multi-Objective Algorithm based upon Particle Swarm Optimisation, an Efficient Data Structure and Turbulence. In Proceedings of UK Workshop on Computational Intelligence (UKCI’02), pages 37–44, Birmingham, UK, Sept. 2–4, 2002.

    Google Scholar 

  15. J.E. Fieldsend and S. Singh. Pareto Multi-Objective Non-Linear Regression Modelling to Aid CAPM Analogous Forecasting. In Proceedings of the 2002 IEEE International Joint Conference on Neural Networks, pages 388–393, Hawaii, May 12–17, 2002. IEEE Press.

    Google Scholar 

  16. J.E. Fieldsend and S. Singh. Optimizing forecast model complexity using multiobjective evolutionary algorithms. In C.A.C Coello and G.B. Lamont, editors, Applications of Multi-Objective Evolutionary Algorithms, pages 675–700. World Scientific, 2005.

    Google Scholar 

  17. C.M. Fonseca and P.J. Fleming. An Overview of Evolutionary Algorithms in Multiobjective Optimization. Evolutionary Computation, 3(1):1–16, 1995.

    Google Scholar 

  18. M.T. Jensen. Reducing the run-time complexity of multiobjective EAs: The NSGA–II and other algorithms. IEEE Transactions on Evolutionary Computation, 7(5):503–515, 2003.

    Article  Google Scholar 

  19. Y. Jin, T. Okabe, and B. Sendhoff. Neural network regularization and ensembling using multi-objective evolutionary algorithms. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’04), pages 1–8. IEEE Press, 2004.

    Google Scholar 

  20. J. Knowles and D. Corne. The pareto archived evolution strategy: A new baseline algorithm for pareto multiobjective optimisation. In Proceedings of the 1999 Congress on Evolutionary Computation, pages 98–105, Piscataway, NJ, 1999. IEEE Service Center.

    Chapter  Google Scholar 

  21. J.D. Knowles and D. Corne. Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy. Evolutionary Computation, 8(2):149–172, 2000.

    Article  Google Scholar 

  22. M. Laumanns, L. Thiele, and E. Zitzler. Running Time Analysis of Multiobjective Evolutionary Algorithms on Pseudo-Boolean Functions. IEEE Transactions on Evolutionary Computation, 8(2):170–182, 2004.

    Article  Google Scholar 

  23. M. Laumanns, L. Thiele, E. Zitzler, E. Welzl, and K. Deb. Running Time Analysis of Multi-objective Evolutionary Algorithms on a Simple Discrete Optimization Problem. In J.J. Merelo Guervós, P. Adamidis, H-G Beyer, J-L Fernández-Villacañas, and H-P Schwefel, editors, Parallel Problem Solving from Nature—PPSN VII, Lecture Notes in Computer Science, pages 44–53. Springer- Verlag, 2002 2002.

    Google Scholar 

  24. Y. LeCun, J. Denker, S. Solla, R. E. Howard, and L. D. Jackel. Optimal brain damage. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems II, pages 598–605, San Mateo, CA, 1990. Morgan Kauffman.

    Google Scholar 

  25. Y. Liu and X. Yao. Towards designing neural network ensembles by evolution. Lecture Notes in Computer Science, 1498:623–632, 1998.

    Article  Google Scholar 

  26. S. Mostaghim, J. Teich, and A. Tyagi. Comparison of Data Structures for Storing Pareto-sets in MOEAs. In Congess on Evolutionary Computation (CEC’2002), volume 1, pages 843–848, Piscataway, New Jersey, May 2002. IEEE Press.

    Google Scholar 

  27. I.T. Nabney. Netlab: Algorithms for Pattern Recognition. Springer, 2002.

    Google Scholar 

  28. Y. Raviv and N. Intrator. Bootstrapping with noise: An effective regularization technique. Connection Science, 8:356–372, 1996.

    Article  Google Scholar 

  29. K.I. Smith, R.M. Everson, and J.E. Fieldsend. Dominance Measures for Multi- Objective Simulated Annealing. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’04), pages 23–30. IEEE Press, 2004.

    Google Scholar 

  30. J. Utans and J. Moody. Selecting neural network architectures via the prediction risk: application to corporate bond rating prediction. In Proc. of the First Int. Conf on AI Applications on Wall Street, pages 35–41, Los Alamos, CA, 1991. IEEE Computer Society Press.

    Chapter  Google Scholar 

  31. D. Van Veldhuizen and G. Lamont. Multiobjective Evolutionary Algorithms: Analyzing the State-of-the-Art. Evolutionary Computation, 8(2):125–147, 2000.

    Article  Google Scholar 

  32. A. S. Weigend and N. A. Gershenfeld, editors. Time Series Prediction: Forecasting the Future and Understanding the Past. Addison-Wesley, Reading, MA, 1994.

    Google Scholar 

  33. D. Wolpert. On bias plus variance. Neural Computation, 9(6):1211–1243, 1997.

    Article  Google Scholar 

  34. X. Yao. Evolving Artificial Neural Networks. Proceedings of the IEEE, 87(9):1423–1447, 1999.

    Article  Google Scholar 

  35. E. Zitzler. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, Swiss Federal Institute of Technology Zurich (ETH), 1999. Diss ETH No. 13398.

    Google Scholar 

  36. E. Zitzler and L. Thiele. Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach. IEEE Transactions on Evolutionary Computation, 3(4):257–271, 1999.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Mathematics, University of Exeter, School of Engineering, Exeter, EX4 4QF, UK

    Jonathan E. Fieldsend

Authors
  1. Jonathan E. Fieldsend
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Honda Research Institute Europe GmbH, Carl-Legien-Str. 30, Offenbach, 63073, Germany

    Yaochu Jin Dr.

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer

About this chapter

Cite this chapter

Fieldsend, J.E. (2006). Regression Error Characteristic Optimisation of Non-Linear Models. In: Jin, Y. (eds) Multi-Objective Machine Learning. Studies in Computational Intelligence, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33019-4_5

Download citation

  • DOI: https://doi.org/10.1007/3-540-33019-4_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30676-4

  • Online ISBN: 978-3-540-33019-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation