Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Improving mixture of experts for view-independent face recognition using teacher-directed learning

  • 159 Accesses

  • 12 Citations

Abstract

In this paper we develop a new learning method, called teacher-directed learning (TDL), for mixture of experts (ME) to perform view-independent face recognition. In the basic form of ME the problem space is automatically divided into several subspaces for the experts, and the outputs of experts are combined by a gating network. In our proposed method, the ME is directed to adapt to a particular partitioning corresponding to predetermined views. To do this, we apply a new learning method to ME, called TDL, in a way that according to the pose of the input training sample, only the weights of the corresponding experts are updated. We apply TDL to MEs, composed of MLP experts and a radial basis function gating network, with different representation schemes: global, single-view and overlapping eigenspace. We test them with previously intermediate unseen views of faces. The experimental results support our claim that directing the experts to a predetermined partitioning of the face space improves the performance of the conventional ME for view-independent face recognition. Comparison with some of the most related methods indicates that the proposed model yields excellent recognition rate in view-independent face recognition.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Belhumeur P.N., Hespanha J., Kriegman D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. (Special Issue on Face Recognition) 17(7), 711–720 (1997)

  2. 2

    Beymer, D.J.: Face recognition under varying pose. Technical Report 1461, MIT AI Lab, Massachusetts Institute of Technology, Cambridge, MA (1993)

  3. 3

    Bowyer K.W., Chang K., Flynn P.: A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Comput. Vis. Image Underst. 101(1), 1–15 (2006)

  4. 4

    Chen K., Xu L., Chi H.: Improved learning algorithms for mixture of experts in multiclass classification. Neural Netw. 12(9), 1229–1252 (1999)

  5. 5

    Cootes T.F., Edwards J.F., Taylor C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

  6. 6

    Czyz, J., Kittler, J., Vandendorpe, L.: Combining face verification experts. In: Proceedings of International Conference Pattern Recognition, vol. 2, pp. 20028–20031 (2002)

  7. 7

    Dailey M.N., Cottrell G.W.: Organization of face and object recognition in modular neural networks. Neural Netw. 12, 1053–1073 (1999)

  8. 8

    Du S., Ward R.K.: Face recognition under pose variations. J. Frankl. Inst. 343, 596–613 (2006)

  9. 9

    Ebrahimpour R., Kabir E., Esteky H., Yousefi M.R.: A mixture of multilayer perceptron experts network for modeling face/nonface recognition in cortical face processing regions. Intell. Autom. Soft Comput. 14(2), 145–156 (2008)

  10. 10

    Ebrahimpour R., Kabir E., Esteky H., Yousefi M.R.: View-independent face recognition with mixture of experts. Neurocomputing 71, 1103–1107 (2008)

  11. 11

    Ebrahimpour R., Kabir E., Yousefi M.R.: Teacher-directed learning with mixture of experts for view-independent face recognition. Lect. Notes Comput. Sci. 4362, 601–611 (2007)

  12. 12

    Ebrahimpour R., Kabir E., Yousefi M.R.: View-based eigenspaces with mixture of experts for view-independent face recognition. Lect. Notes Comput. Sci. 4472, 131–140 (2007)

  13. 13

    Ghaderi, R.: Arranging simple neural networks to solve complex classification problems. Ph.D. thesis, Surrey University (2000)

  14. 14

    Goldman, S., Rivest, R., Schapire, R.: Learning binary relations and total orders. In: Proceedings of the 30th Annual IEEE Symposium on Foundations of Computer Science, pp. 46–51 (1989)

  15. 15

    Goldman, S.: Learning binary relations, total orders, and read-once formulas. Technical Report 483, MIT Laboratory for Computer Science (1990)

  16. 16

    Goldman, S., Kearns, M.: On the complexity of teaching. In: Proceedings of the 4th Annual ACM Workshop on Computational Learning Theory, pp. 303–314 (1991)

  17. 17

    Goldman, S., Mathias, D.: Teaching a smart learner. In: Proceedings of the 6th Annual ACM Workshop on Computational Learning Theory, pp. 67–76 (1993)

  18. 18

    Goldman S.A., Mathias H.D.: Teaching a smarter learner. J. Comput. Syst. Sci. 52(2), 255–267 (1996)

  19. 19

    Goodband J.H., Haasa O.C.L., Millsb J.A.: A mixture of experts committee machine to design compensators for intensity modulated radiation therapy. Pattern Recognit. 39, 1704–1714 (2006)

  20. 20

    Guler I., Ubeyli E.D.: A modified mixture of experts network structure for ECG beats classification with diverse features. Eng. Appl. Artif. Intell. 18, 845–856 (2005)

  21. 21

    Haykin S.: Neural Networks: A Comprehensive Foundation. Prentice Hall, Englewood Cliffs (1999)

  22. 22

    Hong X., Harris C.J.: A mixture of experts network structure construction algorithm for modelling and control. Appl. Intell. 16(1), 59–69 (2002)

  23. 23

    Hong L., Jain A.: Integrating faces and fingerprints for personal identification. IEEE Trans. Pattern Anal. Mach. Intell. 20(12), 1295–1307 (1998)

  24. 24

    Jackson, J., Tompkins, A.: A computational model of teaching. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 319–326 (1992)

  25. 25

    Jacobs R.A., Jordan M.I., Nowlan S.E., Hinton G.E.: Adaptive mixture of experts. Neural Comput. 3, 79–87 (1991)

  26. 26

    Kamimura R.: Teacher-directed learning with Gaussian and sigmoid activation functions. Lect. Notes Comput. Sci. 3316, 530–536 (2004)

  27. 27

    Kim T., Kittler J.: Design and fusion of pose-invariant face identification experts. IEEE Trans. Circuits Syst. Video Technol. 16(9), 1096–1106 (2006)

  28. 28

    Kirby M., Sirovich L.: Application of the KL procedure for the characterization of human faces. IEEE Trans. Pattern Anal. Mach. Intell. 12(1), 103–108 (1990)

  29. 29

    Kittler, J., Ballette, M., Czyz, J., Roli, F., Vandendorpe, L.: Enhancing the performance of personal identity authentication systems by fusion of face verification experts. In: Proceedings of IEEE International Conference on Multimedia Expo, vol. 2, pp. 581–584 (2002)

  30. 30

    Kittler J., Hatef M., Duin R.P.W., Matas J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)

  31. 31

    Kuncheva L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, New York (2004)

  32. 32

    Kuncheva L., Bezdek J.C., Duin R.P.W.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognit. 34, 299–314 (2001)

  33. 33

    Lu J., Plataniotis K.N., Venetsanopoulos A.N.: Face recognition using kernel direct discriminant analysis algorithms. IEEE Trans. Neural Netw. 14(1), 117–126 (2003)

  34. 34

    Mathias, H.D.: DNF—if you can’t learn ‘em, teach ‘em: an interactive model of teaching. In: Proceedings of 8th Annu. Conf. on Comput. Learning Theory, pp. 222–229 (1995)

  35. 35

    Moghaddam B., Pentland A.: Probabilistic visual learning for object representation. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 696–710 (1997)

  36. 36

    Murase H., Nayar S.K.: Visual learning and recognition of 3-D objects from appearance. Int. J. Comput. Vis. 14, 5–24 (1995)

  37. 37

    Natarajan, B.K.: On learning Boolean functions. In: Proceedings of the 19th Annual ACM Symposium on Theory of Computing, pp. 296–304 (1987)

  38. 38

    Pandya A.S., Macy R.B.: Pattern Recognition with Neural Networks in C++. CRC Press, Boca Raton (1996)

  39. 39

    Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 84–91 (1994)

  40. 40

    Rivest, R., Lin, Y.L.: Being taught can be faster than asking questions. In: Proceedings of 8th ACM Conference on Computational Learning Theory, pp. 144–151 (1995)

  41. 41

    Salzberg, S., Delcher, A., Heath, D., Kasif, S.: Learning with a helpful teacher. In: Proceedings of IJCAI-91, pp. 705–711 (1991)

  42. 42

    Sehad A., Hocini H., Hadid A., Djeddi M., Ameur S.: Face recognition under varying views. Lect. Notes Comput. Sci. 1811, 258–267 (2000)

  43. 43

    Shakhnarovich G., Moghaddam B.: Face recognition in subspaces. In: Li, S.Z., Jain, A.K. (eds) Handbook of Face Recognition, Springer-Verlag, New York (2004)

  44. 44

    Shinohara A., Miyano S.: Teachability in computational learning. New Gener. Comput. 8, 337–347 (1991)

  45. 45

    Sim T., Baker S., Bsat M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)

  46. 46

    Sirovich L., Kirby M.: Low-dimensional procedure for characterization of human faces. J. Opt. Soc. Am. 4, 519–524 (1987)

  47. 47

    Tan X., Chen S., Zhou Z.H., Zhang F.: Face recognition from a single image per person: a survey. Pattern Recognit. 39, 1725–1745 (2006)

  48. 48

    Tang, H., Lyu, M., King, I.: Face recognition committee machine. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 837–840 (2003)

  49. 49

    Tolba A.S., El-Baz A.H., El-Harby A.A.: Face recognition: a literature review. Int. J. Signal Process. 2(2), 88–103 (2005)

  50. 50

    Tolba A.S., Abu-Rezq A.N.: Combined classifiers for invariant face recognition. Pattern Anal. Appl. 3(4), 289–302 (2000)

  51. 51

    Turk M., Pentland A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

  52. 52

    Ubeyli E.D., Guler I.: Improving medical diagnostic accuracy of ultrasound Doppler signals by combining neural network models. Comput. Biol. Med. 35, 405–420 (2005)

  53. 53

    Ullman S., Bart E.: Recognition invariance obtained by extended and invariant features. Neural Netw. 17, 833–848 (2004)

  54. 54

    Wiskott L., Fellous J.M., Von Der Malsburg C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19, 775–779 (1997)

  55. 55

    Xu L., Krzak A., Suen C.Y.: Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Trans. Syst. Man Cybern. 22(3), 418–435 (1992)

  56. 56

    Zhao, W., Chellappa, R., Rosenfeld, A., Phillips, P.J.: Face recognition: a literature survey. ACM Comput. Surv. 399–458 (2003)

  57. 57

    Zhang, W., Shan, S., Gao, W., Chang, Y., Cao, B., Yang, P.: Information fusion in face identification. In: Proceedings of International Conference Pattern Recognition, vol. 3, pp. 950–953 (2004)

Download references

Author information

Correspondence to Reza Ebrahimpour.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ebrahimpour, R., Kabir, E. & Yousefi, M.R. Improving mixture of experts for view-independent face recognition using teacher-directed learning. Machine Vision and Applications 22, 421–432 (2011). https://doi.org/10.1007/s00138-009-0232-9

Download citation

Keywords

  • View-independent face recognition
  • Mixture of experts
  • Teacher-directed learning
  • Single-view eigenspaces
  • Global eigenspace
  • Overlapping eigenspaces