Abstract
We investigate a multi-task approach to similarity discriminant analysis, where we propose treating the estimation of the different class-conditional distributions of the pairwise similarities as multiple tasks. We show that regularizing these estimates together using a least-squares regularization weighted by a task-relatedness matrix can reduce the resulting maximum a posteriori classification errors. Results are given for benchmark data sets spanning a range of applications. In addition, we present a new application of similarity-based learning to analyzing the rhetoric of multiple insurgent groups in Iraq. We show how to produce the necessary task relatedness information from standard given training data, as well as how to derive task-relatedness information if given side information about the class relatedness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agarwal, A., Daumé III, H., Gerber, S.: Learning multiple tasks using manifold regularization. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 23, pp. 46–54 (2010)
Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Machine Learning 73(3), 243–272 (2008)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Bonilla, E.V., Chai, K.M.A., Williams, C.K.I.: Multi-task Gaussian process prediction. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems. MIT Press, Cambridge (2008)
Cazzanti, L., Gupta, M.R.: Local similarity discriminant analysis. In: Proc. Intl. Conf. Machine Learning (2007)
Cazzanti, L., Gupta, M.R.: Regularizing the local similarity discriminant analysis classifier. In: Proc. 8th Intl. Conf. Machine Learning and Applications (December 2009)
Cazzanti, L., Gupta, M.R., Koppal, A.J.: Generative models for similarity-based classification. Pattern Recognition 41(7), 2289–2297 (2008)
Chen, J., Ye, J.: Training svm with indefinite kernels. In: Proc. of the Intl. Conf. on Machine Learning (2008)
Chen, Y., Garcia, E.K., Gupta, M.R., Rahimi, A., Cazzanti, L.: Similarity-based classification: Concepts and algorithms. Journal of Machine Learning Research 10, 747–776 (2009)
Chen, Y., Gupta, M.R.: Learning kernels from indefinite similarities. In: Proc. of the Intl. Conf. on Machine Learning (2009)
Daumé III, H., Marcu, D.: Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research 26, 101–126 (2006)
Driskell, J.E., McDonald, T.: Identification of incomplete networks. Florida Maxima Corporation Technical Report (08-01) (2008)
Evgeniou, T., Micchelli, C., Pontil, M.: Learning multiple tasks with kernel methods. Journal of Machine Learning Research (6) (April 2005)
Evgeniou, T., Pontil, M.: Regularized multi–task learning. In: KDD 2004, pp. 109–117. ACM, New York (2004)
Feng, S., Krim, H., Kogan, I.A.: 3D face recognition using Euclidean integral invariants signature. In: IEEE/SP 14th Workshop on Statistical Signal Processing, SSP 2007 (2007)
Friedman, J.H.: Regularized discriminant analysis. Journal American Statistical Association 84(405), 165–175 (1989)
Garcia, E.K., Feldman, S., Gupta, M.R., Srivastava, S.: Completely lazy learning. IEEE Trans. Knowledge and Data Engineering 22(9), 1274–1285 (2010)
Graepel, T., Herbrich, R., Bollmann-Sdorra, P., Obermayer, K.: Classification on pairwise proximity data. In: Advances in Neural Information Processing Systems, vol. 11, pp. 438–444 (1998)
Gupta, M.R., Chen, Y.: Theory and use of the em method. Foundations and Trends in Signal Processing 4(3), 223–296 (2010)
Handcock, M., Hunter, D.R., Goodreau, S.: Goodness of fit of social network models. Journal American Statistical Association 103, 248–258 (2008)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)
Hofmann, T., Buhmann, J.: Pairwise data clustering by deterministic annealing. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(1) (January 1997)
Kato, T., Kashima, H., Sugiyama, M., Asai, K.: Multi-task learning via conic programming. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems, vol. 20, pp. 737–744. MIT Press, Cambridge (2008)
Liao, L., Noble, W.S.: Combining pairwise sequence similarity and support vector machines for detecting remote protein evolutionary and structural relationships. Journal of Computational Biology 10(6), 857–868 (2003)
Liu, Q., Liao, X., Li, H., Stack, J.R., Carin, L.: Semisupervised multitask learning. IEEE Trans. Pattern Analysis and Machine Intelligence (6) (June 2009)
Luss, R., d’Aspremont, A.: Support vector machine classification with indefinite kernels. Mathematical Programming Computation 1(2), 97–118 (2009)
Micchelli, C.A., Pontil, M.: Kernels for multi–task learning. In: Advances in Neural Information Processing Systems (2004)
Pekalska, E., Duin, R.P.W.: Dissimilarity representations allow for building good classifiers. Pattern Recognition Letters 23(8), 943–956 (2002)
Pekalska, E., Paclíc, P., Duin, R.P.W.: A generalized kernel approach to dissimilarity-based classification. Journal of Machine Learning Research, 175–211 (2001)
Philips, S., Pitton, J., Atlas, L.: Perceptual feature identification for active sonar echoes. In: Proc. of the 2006 IEEE OCEANS Conf. (2006)
Roth, V., Laub, J., Kawanabe, M., Buhmann, J.M.: Optimal cluster preserving embedding of nonmetric proximity data. IEEE Trans. Pattern Anal. and Machine Intel. 25(12), 1540–1551 (2003)
Sadowski, P., Cazzanti, L., Gupta, M.R.: Bayesian and pairwise local similarity discriminant analysis. In: Proc. Intl. Workshop on Cognitive Information Processing (CIP), Isola d’Elba, Italy (June 2010)
Sheldon, D.: Graphical multi-task learning. In: Neural Information Processing Systems Workshops (2008), http://web.engr.oregonstate.edu/~sheldon ,
Sheldon, D.: Graphical multi-task learning (2010) (unpublished manuscript), http://web.engr.oregonstate.edu/~sheldon
Srivastava, S., Gupta, M.R., Frigyik, B.: Bayesian quadratic discriminant analysis. Journal of Machine Learning Research 8, 1277–1305 (2007)
Stanfill, C., Waltz, D.: Toward memory-based reasoning. Communications of the ACM 29(12), 1213–1228 (1986)
Wu, G., Chang, E.Y., Zhang, Z.: An analysis of transformation on non-positive semidefinite similarity matrix for kernel machines. Tech. rep., University of California, Santa Barbara (March 2005)
Ying, Y., Campbell, C., Girolami, M.: Analysis of svm with indefinite kernels. In: Advances in Neural Information Processing Systems. MIT Press, Cambridge (2009)
Zhang, H., Berg, A.C., Maire, M., Malik, J.: SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 2126–2136 (2006)
Zhang, Y., Yeung, D.Y.: A convex formulation for learning task relationships. In: Grünwald, P., Spirtes, P. (eds.) Proc. of the 26th Conference on Uncertainty in Artificial Intelligence, UAI 2010 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cazzanti, L., Feldman, S., Gupta, M.R., Gabbay, M. (2011). Multi-task Regularization of Generative Similarity Models. In: Pelillo, M., Hancock, E.R. (eds) Similarity-Based Pattern Recognition. SIMBAD 2011. Lecture Notes in Computer Science, vol 7005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24471-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-24471-1_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24470-4
Online ISBN: 978-3-642-24471-1
eBook Packages: Computer ScienceComputer Science (R0)