Abstract
Deep networks and Fisher kernels are two competitive approaches showing strides of progress and improvement for computer vision tasks in specific the large scale object categorisation problem. One of the recent developments in this regard has been the use of a hybrid approach that encodes higher order statistics of deep models for Fisher vector encodings. In this chapter we shall discuss how to train a deep model for extracting Fisher kernel. The tips discussed here are validated by industrial practices and research community through mathematical proofs by LeCun et al. (Neural networks: tricks of the trade. Springer, pp 9–50 (1998), [1]), Bengio (Neural networks: tricks of the trade. Springer, pp 437–478 (2012), [2]) and case studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., Bottou, L., Orr, G.B., et al.: Efficient backprop. In: Neural Networks: Tricks of the Trade, pp. 9–50. Springer (1998)
Bengio, Y.: Practical recommendations for gradient-based training of deep architectures. In: Neural Networks: Tricks of the Trade, pp. 437–478. Springer (2012)
Marchesi, M.: Megapixel size image creation using generative adversarial networks (2017). arXiv preprint arXiv:1706.00082
Wang, J., Perez, L.: The effectiveness of data augmentation in image classification using deep learning (2017). arXiv preprint arXiv:1712.04621
Ahmed, S., Azim, T.: Compression techniques for deep fisher vectors. In: ICPRAM, pp. 217–224 (2017)
Collobert, R., Bengio, S.: Links between perceptrons, MLPs and SVMs. In: Proceedings of the Twenty-First International Conference on Machine Learning. ACM (2004)
Jarrett, K., Kavukcuoglu, K., LeCun, Y., et al.: What is the best multi-stage architecture for object recognition? In: IEEE 12th International Conference on Computer Vision, pp. 2146–2153. IEEE (2009)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2011)
Louizos, C., Welling, M., Kingma, D.: Learning sparse neural networks through \( {L\_0} \) regularization (2017). arXiv preprint arXiv:1712.01312
Theis, L., Korshunova, I., Tejani, A., et al.: Faster gaze prediction with dense networks and fisher pruning (2018). arXiv preprint arXiv:1801.05787
Blum, A.: Neural Networks in C++, vol. 697. Wiley, NY (1992)
Berry, M., Linoff, G.: Data Mining Techniques: For Marketing, Sales, and Customer Support. Wiley (1997)
Boger, Z., Guterman, H.: Knowledge extraction from artificial neural network models. In: IEEE International Conference on Systems, Man, and Cybernetics Computational Cybernetics and Simulation, vol. 4, pp. 3030–3035. IEEE (1997)
Ruder, S.: An overview of gradient descent optimization algorithms. Comput. Res. Repos. (2016). http://arxiv.org/abs/1609.04747. (CoRR) abs/1609.04747
Bengio, Y.: Practical recommendations for gradient-based training of deep architectures. Comput. Res. Repos. (2012). CoRR abs/1206.5533
Singh, S., Hoiem, D., Forsyth, D.: Swapout: learning an ensemble of deep architectures. In: Advances in Neural Information Processing Systems, pp. 28–36 (2016)
Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 1929–1958 (2014)
Huang, G., Sun, Y., Liu, Z., et al.: Deep networks with stochastic depth. In: European Conference on Computer Vision, pp. 646–661. Springer (2016)
Hinton, G.: Training products of experts by minimizing contrastive divergence. Neural Comput., 1771–1800 (2002)
Lin, J., Zhang, J.: A fast parameters selection method of support vector machine based on coarse grid search and pattern search. In: Fourth Global Congress on Intelligent Systems, pp. 77–81 (2013)
Staelin, C.: Parameter selection for support vector machines. Technical report, Hewlett-Packard Company, HPL-2002-354R1 (2003)
Salakhutdinov, R., Larochelle, H.: Efficient learning of deep Boltzmann machines. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 693–700 (2010)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2018 The Author(s), under exclusive licence to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Azim, T., Ahmed, S. (2018). Training Deep Models and Deriving Fisher Kernels: A Step Wise Approach. In: Composing Fisher Kernels from Deep Neural Models. SpringerBriefs in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-98524-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-98524-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98523-7
Online ISBN: 978-3-319-98524-4
eBook Packages: Computer ScienceComputer Science (R0)