Abstract
This chapter introduces a specific genre of kernels that draws a formal connection between the generative and discriminative models of learning. Both the paradigms offer unique complementary advantages over one another, yet there always existed a need to combine the best of both the worlds for solving complex problems. This gap was filled by Tommy Jaakola through the introduction of Fisher kernels in 1998 and since then it has played a key role in solving problems from computational biology, computer vision and machine learning. We introduce this concept here and show how to compute Fisher vector encodings from deep models using a toy example in MATLAB.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We must have twice differentiable likelihood so that the Fisher information I exists and I must be positive definite at the chosen \(\theta \).
References
Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Advances in Neural Information Processing Systems, pp. 487–493 (1999)
Nyffenegger, M., Chappelier, J., Gaussier, É.: Revisiting fisher kernels for document similarities. In: European Conference on Machine Learning, pp. 727–734, Springer (2006)
Perronnin, F., Dance, C.: Fisher kernels on visual vocabularies for image categorization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, IEEE (2007)
Maaten, L.: Learning discriminative fisher kernels. In: ICML 11, 217–224 (2011)
Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large scale image classification. In: European Conference on Computer Vision, pp. 143–156, Springer (2010)
Ahmed, S., Azim, T.: Compression techniques for deep fisher vectors. In: ICPRAM, pp. 217–224 (2017)
Jaakkola, T., Diekhans, M., Haussler, D.: A discriminative framework for detecting remote protein homologies. J. Comput. Biol. 7(1–2), 95–114 (2000)
Azim, T., Niranjan, M.: Inducing discrimination in biologically inspired models of visual scene recognition. In: IEEE International Workshop on Machine Learning for Signal Processing, (MLSP) (2013)
Smith, N., Niranjan, M.: Data-dependent kernels in SVM classification of speech patterns. Technical Report, Cambridge University (2001)
Moreno, P., Rifkin, R.: Using the fisher kernel method for web audio classification. In: Acoustics, Speech, and Signal Processing (ICASSP), vol. 4, pp. 2417–2420 (2000)
Vinokourov, A., Girolami, M.: Document Classification Employing the Fisher Kernel Derived from Probabilistic Hierarchic Corpus Representations. Springer (2001)
Chappelier, J., Eckard, E.: PLSI: The true fisher kernel and beyond. European Conference on Machine Learning and Knowledge Discovery in Databases: Part I, 195–210 (2009)
Sun, Q., Li, R., Luo, D., et al.: Text segmentation with LDA based fisher kernel. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, Association for Computational Linguistics, pp. 269–272 (2008)
Elkan, C.: Deriving TF-IDF as a fisher kernel. In: 12th International Conference on String Processing and Information Retrieval (SPIRE), pp. 296–301 (2005)
Azim, T.: Fisher kernels match deep models. Electron. Lett. 53(6), 397–399 (2017)
Azim, T., Niranjan, M.: Texture classification with Fisher kernel extracted from the continuous models of RBM. In: International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2, pp. 684–690, IEEE ( 2014)
Songa, Y., Hong, X., McLoughlin, I.: Image classification with CNN based fisher vector coding. In: Visual Communications and Image Processing (VCIP), pp. 1–4 (2016)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2018 The Author(s), under exclusive licence to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Azim, T., Ahmed, S. (2018). Fundamentals of Fisher Kernels. In: Composing Fisher Kernels from Deep Neural Models. SpringerBriefs in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-98524-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-98524-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98523-7
Online ISBN: 978-3-319-98524-4
eBook Packages: Computer ScienceComputer Science (R0)