Summary
System Identification and Machine Learning are developing mostly as independent subjects, although the underlying problem is the same: To be able to associate “outputs” with “inputs”. Particular areas in machine learning of substantial current interest are manifold learning and unsupervised and semi-supervised regression. We outline a general approach to semi-supervised regression, describe its links to Local Linear Embedding, and illustrate its use for various problems. In particular, we discuss how these techniques have a potential interest for the system identification world.
This work was supported by the Strategic Research Center MOVIII, funded by the Swedish Foundation for Strategic Research, SSF, and CADICS, a Linnaeus center funded by the Swedish Research Council.
Dedicated to Chris and Anders at the peak of their careers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bauwens, M., Ohlsson, H., Barbé, K., Beelaerts, V., Dehairs, F., Schoukens, J.: On climate reconstruction using bivalve shells: Three methods to interpret the chemical signature of a shell. In: 7th IFAC Symposium on Modelling and Control in Biomedical Systems (April 2009)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)
Bengio, Y., Delalleau, O., Le Roux, N.: Label propagation and quadratic criterion. In: Chapelle, O., Schölkopf, B., Zien, A. (eds.) Semi-Supervised Learning, pp. 193–216. MIT Press, Cambridge (2006)
Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)
Cybenko, G.: Just-in-time learning and estimation. In: Bittanti, S., Picci, G. (eds.) Identification, Adaptation, Learning. The Science of Learning Models from data. NATO ASI Series, pp. 423–434. Springer, Heidelberg (1996)
de Ridder, D., Duin, R.: Locally linear embedding for classification. Tech Report, PH-2002-01, Pattern Recognition Group, Dept. of Imaging Science & Technology, Delft University of Technology, Delft, The Netherlands (2002)
de Ridder, D., Kouropteva, O., Okun, O., Pietikäinen, M.: Supervised locally linear embedding. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 333–341. Springer, Heidelberg (2003)
Gillikin, D.P., Dehairs, F., Lorrain, A., Steenmans, D., Baeyens, W., André, L.: Barium uptake into the shells of the common mussel (Mytilus edulis) and the potential for estuarine paleo-chemistry reconstruction. Geochimica et Cosmochimica Acta 70(2), 395–407 (2006)
Gillikin, D.P., Lorrain, A., Bouillon, S., Willenz, P., Dehairs, F.: Stable carbon isotopic composition of Mytilus edulis shells: relation to metabolism, salinity, \(\delta^{13}\mbox{C}_{DIC}\) and phytoplankton. Organic Geochemistry 37(10), 1371–1382 (2006)
Goldberg, A.B., Zhu, X.: Seeing stars when there aren’t many stars: Graph-based semi-supervised learning for sentiment categorization. In: HLT-NAACL 2006 Workshop on Textgraphs: Graph-based Algorithms for Natural Language Processing (2006)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer, New York (2001)
Hosmer Jr., D.W.: A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics 29(4), 761–770 (1973)
Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, vol. 30. Springer, Berlin (1995)
Merz, C.J., St. Clair, D.C., Bond, W.E.: Semi-supervised adaptive resonance theory (smart2). In: International Joint Conference on Neural Networks, IJCNN, Jun. 1992, vol. 3, pp. 851–856 (1992)
Narendra, K.S., Li, S.-M.: Neural networks in control systems. In: Smolensky, P., Mozer, M.C., Rumelhard, D.E. (eds.) Mathematical Perspectives on Neural Networks, pp. 347–394. Lawrence Erlbaum Associates, Mahwah (1996)
Navaratnam, R., Fitzgibbon, A.W., Cipolla, R.: The joint manifold model for semi-supervised multi-valued regression. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, October 2007, pp. 1–8 (2007)
Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Learning to classify text from labeled and unlabeled documents. In: AAAI ’98/IAAI ’98: Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, pp. 792–799. AAAI Press, Menlo Park (1998)
Ohlsson, H., Roll, J., Ljung, L.: Manifold-constrained regressors in system identification. In: Proc. 47st IEEE Conference on Decision and Control, December 2008, pp. 1364–1369 (2008)
Putten, E.V., Dehairs, F., André, L., Baeyens, W.: Quantitative in situ microanalysis of minor and trace elements in biogenic calcite using infrared laser ablation - inductively coupled plasma mass spectrometry: a critical evaluation. Analytica Chimica Acta 378(1-3), 261–272 (1999)
Rahimi, A., Recht, B., Darrell, T.: Learning to transform time series with a few examples. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(10), 1759–1775 (2007)
Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290(5500), 2323–2326 (2000)
olde Scheper, T., Klinkenberg, D., Pennartz, C., van Pelt, J.: A Mathematical Model for the Intracellular Circadian Rhythm Generator. J. Neurosci. 19(1), 40–47 (1999)
Stenman, A.: Model on Demand: Algorithms, Analysis and Applications. Linköping Studies in science and Technology. Thesis No 571, Linköping University, SE-581 83 Linköping, Sweden (April 1999)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Wang, F., Zhang, C.: Label propagation through linear neighborhoods. IEEE Transactions on Knowledge and Data Engineering 20(1), 55–67 (2008)
Yang, X., Fu, H., Zha, H., Barlow, J.: Semi-supervised nonlinear dimensionality reduction. In: ICML ’06: Proceedings of the 23rd international conference on Machine learning, pp. 1065–1072. ACM, New York (2006)
Zhao, L., Zhang, Z.: Supervised locally linear embedding with probability-based distance for classification. Comput. Math. Appl. 57(6), 919–926 (2009)
Zhu, X.: Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer Berlin Heidelberg
About this chapter
Cite this chapter
Ohlsson, H., Ljung, L. (2010). Semi-supervised Regression and System Identification, . In: Hu, X., Jonsson, U., Wahlberg, B., Ghosh, B. (eds) Three Decades of Progress in Control Sciences. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11278-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-11278-2_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-11277-5
Online ISBN: 978-3-642-11278-2
eBook Packages: EngineeringEngineering (R0)