Abstract
Small sample is an acute problem in many application domains, which may be partially addressed by feature selection or dimensionality reduction. For the purpose of distance learning, we describe a method for feature selection using equivalence constraints between pairs of datapoints. The method is based on L1 regularization and optimization. Feature selection is then incorporated into an existing non-parametric method for distance learning, which is based on the boosting of constrained generative models. Thus the final algorithm employs dynamical feature selection, where features are selected anew in each boosting iteration based on the weighted training data. We tested our algorithm on the classification of facial images, using two public domain databases. We show the results of extensive experiments where our method performed much better than a number of competing methods, including the original boosting-based distance learning method and two commonly used Mahalanobis metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Athitsos, V., Alon, J., Sclaroff, S., Kollios, G.: BoostMap: A method for efficient approximate similarity rankings. In: Proc. CVPR (2004)
Bart, E., Ullman, S.: Cross-generalization: learning novel classes from a single example by feature replacement. In: Proc. CVPR, pp. 672–679 (2005)
Bilenko, M., Basu, S., Mooney, R.J.: Integrating constraints and metric learning in semi-supervised clustering. In: ACM International Conference Proceeding Series (2004)
Chang, H., Yeung, D.Y.: Locally linear metric adaptation for semi-supervised clustering. In: ACM International Conference Proceeding Series (2004)
De Bie, T., Momma, M., Cristianini, N.: Efficiently learning the metric with side-information. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds.) ALT 2003. LNCS (LNAI), vol. 2842, pp. 175–189. Springer, Heidelberg (2003)
Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of the National Academy of Sciences 100(5), 2197–2202 (2003)
Ferencz, A., Learned-Miller, E., Malik, J.: Building a classification cascade for visual identification from one example. In: Proc. ICCV, pp. 286–293 (2005)
Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: generative models for recognition under variable pose and illumination. In: IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 277–284 (2000)
Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. Advances in Neural Information Processing Systems, 17 (2005)
Hertz, T., Bar-Hillel, A., Weinshall, D.: Boosting margin based distance functions for clustering. In: ICML (2004)
Hertz, T., Bar-Hillel, A., Weinshall, D.: Learning a kernel function for classification with small training samples. In: ICML (2006)
Li, F.F., Fergus, R., Perona, P.: One-shot learning of object categories. IEEE PAMI 28(4), 594–611 (2006)
Ng, A.Y.: Feature selection, L1 vs. L2 regularization, and rotational invariance. In: ACM International Conference Proceeding Series (2004)
Shental, N., Bar-Hillel, A., Hertz, T., Weinshall, D.: Computing gaussian mixture models with em using equivalence constraints. In: NIPS (2003)
Shental, N., Hertz, T., Weinshall, D., Pavel, M.: Adjustment learning and relevant component analysis. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, Springer, Heidelberg (2002)
Sudderth, E.B., Torralba, A., Freeman, W.T., Willsky, A.S.: Learning hierarchical models of scenes, objects, and parts. In: Proc. ICCV (2005)
Tibshirani, R.: Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological) 58(1), 267–288 (1996)
Tsymbal, A., Puuronen, S., Skrypnyk, I.: Ensemble feature selection with dynamic integration of classifiers. In: Int. ICSC-CIMA (2001)
Zheng, A.X., Jordan, M.I., Liblit, B., Aiken, A.: Statistical debugging of sampled programs. Advances in Neural Information Processing Systems 17 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Weinshall, D., Zamir, L. (2007). Image Classification from Small Sample, with Distance Learning and Feature Selection. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2007. Lecture Notes in Computer Science, vol 4842. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76856-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-76856-2_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-76855-5
Online ISBN: 978-3-540-76856-2
eBook Packages: Computer ScienceComputer Science (R0)