Principal Component Analysis over Continuous Subspaces and Intersection of Half-Spaces

  • Anat Levin
  • Amnon Shashua
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2352)


Principal Component Analysis (PCA) is one of the most popular techniques for dimensionality reduction of multivariate data points with application areas covering many branches of science. However, conventional PCA handles the multivariate data in a discrete manner only, i.e., the covariance matrix represents only sample data points rather than higher-order data representations.

In this paper we extend conventional PCA by proposing techniques for constructing the covariance matrix of uniformly sampled continuous regions in parameter space. These regions include polytops defined by convex combinations of sample data, and polyhedral regions defined by intersection of half spaces. The applications of these ideas in practice are simple and shown to be very effective in providing much superior generalization properties than conventional PCA for appearance-based recognition applications.


Covariance Matrix Independent Component Analysis Principal Vector False Detection Rate Principal Component Analysis Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    J.J. Atick, P.A. Griffin, and N.A. Redlich. Statistical approach to shape-from-shading: deriving 3d face surfaces from single 2d images. Neural Computation, 1997.Google Scholar
  2. 2.
    R. Basri and D. Jacobs. Photometric stereo with general, unknown lighting. In iccv, Vancouver, Canada, July 2001.Google Scholar
  3. 3.
    P.N Belhumeur, J.P. Hespanha, and D.J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. In Proceedings of the European Conference on Computer Vision, 1996.Google Scholar
  4. 4.
    P.N Belhumeur, J.P. Hespanha, and D.J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. In Proceedings of the European Conference on Computer Vision, 1996.Google Scholar
  5. 5.
    A.J. Bell and T.J. Sejnowski. An information maximization approach to blind separation and blind deconvolution. Neural Computation 7(6), pages 1129–1159, 1995.CrossRefGoogle Scholar
  6. 6.
    Michael J. Black and D. Jepson. Eigen-tracking: Robust matching and tracking of articulated objects using a view-based representation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 329–342, Cambridge, England, 1996.Google Scholar
  7. 7.
    C. Bregler and S.M. Omohundro. Nonlinear manifold learning for visual speech recognition. In iccv, Boston, Jun 1995.Google Scholar
  8. 8.
    P. Comon. Independent component analysis, a new concept? Signal processing 36(3), pages 11–20, 1994.CrossRefGoogle Scholar
  9. 9.
    P. Hallinan. A low-dimentional representation of human faces for arbitrary lightening conditions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 995–999, 1994.Google Scholar
  10. 10.
    T. Hastie and W. Stuetzle. Principal curves. Journal of Americam Statistical Association 84, pages 502–516, 1989.zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    T. Heap and D. Hogg. Wormholes in shape space: Tracking through discontinuous changes in shape. In iccv, 1998.Google Scholar
  12. 12.
    I.T. Jolliffe. Principal Component Analysis. Springer-Verlag, New York, 1986.Google Scholar
  13. 13.
    M.A. Kramer. Non linear principal component analysis using autoassociative neural networks. AI Journal 37(2), pages 233–243, 1991.CrossRefGoogle Scholar
  14. 14.
    K.C. Lee, J. Ho and D. Kriegman. Nine points of light: Acquiring subspaces for face recognition under variable lighting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2001.Google Scholar
  15. 15.
    A.M. Martinez and A.C. Kak. PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(1):228–233, 2001.CrossRefGoogle Scholar
  16. 16.
    B. Moghaddam A. Pentland and B. Starner. View-based and modular eigenspaoes for face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 84–91, 1994.Google Scholar
  17. 17.
    H. Murase and S.K. Nayar. Learning and recognition of 3D objects from appearance. In IEEE 2nd Qualitative Vision Workshop, pages 39–50, New York, NY, June 1993.Google Scholar
  18. 18.
    E. Grimson P. Lipson and P. Sinha. Configuration based scene classification and image-indexing. In cvpr, San Juan, Puerto Rico, 1997.Google Scholar
  19. 19.
    R. Ramamoorthi and P. Hanrahan. On the relationship between Radiance and Irradiance: Determining the illumination from images of a convex Lambertian object. In Journal of the Optical Society of America (JOSA A), Oct. 2001, pages 2448–2459.Google Scholar
  20. 20.
    H. Schneiderman and T. Kanade. A statistical model for 3d object detection applied to faces and cars. In cvpr, South Carolina, June 2000.Google Scholar
  21. 21.
    V. Silva, J.B. Tenenbaum and J.C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290, December 2000.Google Scholar
  22. 22.
    A. Shashua. Illumination and view position in 3D visual recognition. In Proceedings of the conference on Neural Information Processing Systems (NIPS), Denver, CO, December 1991.Google Scholar
  23. 23.
    A. Shashua. On photometric issues in 3D visual recognition from a single 2D image. International Journal of Computer Vision, 21:99–122, 1997.CrossRefGoogle Scholar
  24. 24.
    P. Sinha. Object recognition via image invariances. Investigative Ophthalmology and Visual Science 35/4:#1735, 1994.Google Scholar
  25. 25.
    L. Sirovich and M. Kirby. Low dimensional procedure for the characterization of human faces. Journal of the Optical Society of America, 4(3):519–524, 1987.CrossRefGoogle Scholar
  26. 26.
    M. Turk and A. Pentland. Eigen faces for recognition. J. of Cognitive Neuroscience, 3(1), 1991.Google Scholar
  27. 27.
    A.R. Webb. An approach to nonlinear principal components-analysis using radially symmetrical kernel functions. Statistics and computing 6(2), pages 159–168, 1996.CrossRefGoogle Scholar
  28. 28.
    J.M. Winn C.M. Bishop. Non-linear bayesian image modelling. In Proceedings of the European Conference on Computer Vision, Dublin, Ireland, June 2000.Google Scholar
  29. 29.
    L. Zhao and Y.H. Yang. Theoretical analysis of illumination in PCA-based vision systems. Pattern Recognition, 32:547–564, 1999.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Anat Levin
    • 1
  • Amnon Shashua
    • 1
  1. 1.Computer Science DepartmentStanford UniversityStanford

Personalised recommendations