Abstract
The classical perceptron rule provides a varying upper bound on the maximum margin, namely the length of the current weight vector divided by the total number of updates up to that time. Requiring that the perceptron updates its internal state whenever the normalized margin of a pattern is found not to exceed a certain fraction of this dynamic upper bound we construct a new approximate maximum margin classifier called the perceptron with dynamic margin (PDM). We demonstrate that PDM converges in a finite number of steps and derive an upper bound on them. We also compare experimentally PDM with other perceptron-like algorithms and support vector machines on hard margin tasks involving linear kernels which are equivalent to 2-norm soft margin.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blum, A.: Lectures on machine learning theory. Carnegie Mellon University, USA, http://www.cs.cmu.edu/~avrim/ML09/lect0126.pdf
Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines Cambridge. Cambridge University Press, Cambridge (2000)
Duda, R.O., Hart, P.E.: Pattern classsification and scene analysis. Wiley, Chichester (1973)
Freund, Y., Shapire, R.E.: Large margin classification using the perceptron algorithm. Machine Learning 37(3), 277–296 (1999)
Gentile, C.: A new approximate maximal margin classification algorithm. Journal of Machine Learning Research 2, 213–242 (2001)
Joachims, T.: Making large-scale SVM learning practical. In: Advances in Kernel Methods-Support Vector Learning. MIT Press, Cambridge (1999)
Joachims, T.: Training linear SVMs in linear time. In: KDD, pp. 217–226 (2006)
Hsieh, C.-J., Chang, K.-W., Lin, C.-J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: ICML, pp. 408–415 (2008)
Ishibashi, K., Hatano, K., Takeda, M.: Online learning of approximate maximum p-norm margin classifiers with bias. In: COLT, pp. 69–80 (2008)
Krauth, W., Mézard, M.: Learning algorithms with optimal stability in neural networks. Journal of Physics A20, L745–L752 (1987)
Li, Y., Long, P.: The relaxed online maximum margin algorithm. Machine Learning 46(1-3), 361–387 (2002)
Novikoff, A.B.J.: On convergence proofs on perceptrons. In: Proc. Symp. Math. Theory Automata, vol. 12, pp. 615–622 (1962)
Panagiotakopoulos, C., Tsampouka, P.: The margin perceptron with unlearning. In: ICML, pp. 855–862 (2010)
Panagiotakopoulos, C., Tsampouka, P.: The margitron: A generalized perceptron with margin. IEEE Transactions on Neural Networks 22(3), 395–407 (2011)
Platt, J.C.: Sequential minimal optimization: A fast algorithm for training support vector machines. Microsoft Res. Redmond WA, Tech. Rep. MSR-TR-98-14 (1998)
Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65(6), 386–408 (1958)
Tsampouka, P., Shawe-Taylor, J.: Perceptron-like large margin classifiers. Tech. Rep., ECS, University of Southampton, UK (2005), Obtainable from, http://eprints.ecs.soton.ac.uk/10657
Tsampouka, P., Shawe-Taylor, J.: Analysis of generic perceptron-like large margin classifiers. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 750–758. Springer, Heidelberg (2005)
Tsampouka, P., Shawe-Taylor, J.: Constant rate approximate maximum margin algorithms. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 437–448. Springer, Heidelberg (2006)
Tsampouka, P., Shawe-Taylor, J.: Approximate maximum margin algorithms with rules controlled by the number of mistakes. In: ICML, pp. 903–910 (2007)
Vapnik, V.: Statistical learning theory. Wiley, Chichester (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Panagiotakopoulos, C., Tsampouka, P. (2011). The Perceptron with Dynamic Margin. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2011. Lecture Notes in Computer Science(), vol 6925. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24412-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-24412-4_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24411-7
Online ISBN: 978-3-642-24412-4
eBook Packages: Computer ScienceComputer Science (R0)