Advertisement

Deep Metric Learning for Sequential Data Using Approximate Information

  • Stefan ThalerEmail author
  • Vlado Menkovski
  • Milan Petkovic
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10934)

Abstract

Learning a distance metric provides solutions to many problems where the data exists in a high dimensional space and hand-crafted distance metrics fail to capture its semantical structure. Methods based on deep neural networks such as Siamese or Triplet networks have been developed for learning such metrics. In this paper we present a metric learning method for sequence data based on a RNN-based triplet network. We posit that this model can be trained efficiently with regards to labels by using Jaccard distance as a proxy distance metric. We empirically demonstrate the performance and efficiency of the approach on three different computer log-line datasets.

Keywords

Efficient metric learning Triplet network Deep learning 

Notes

Acknowledgment

The work presented in this paper is part of a project which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 780495.

References

  1. 1.
    Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al.: TensorFlow: a system for large-scale machine learning. In: OSDI 2016, pp. 265–283 (2016)Google Scholar
  2. 2.
    Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a “siamese” time delay neural network. In: Advances in Neural Information Processing Systems, pp. 737–744 (1994)Google Scholar
  3. 3.
    Carlevaris-Bianco, N., Eustice, R.M.: Learning visual feature descriptors for dynamic lighting conditions. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 2769–2776. IEEE (2014)Google Scholar
  4. 4.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 539–546 (2005)Google Scholar
  5. 5.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  6. 6.
    Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: Feragen, A., Pelillo, M., Loog, M. (eds.) SIMBAD 2015. LNCS, vol. 9370, pp. 84–92. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24261-3_7CrossRefGoogle Scholar
  7. 7.
    Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, vol. 2 (2015)Google Scholar
  8. 8.
    Melis, G., Dyer, C., Blunsom, P.: On the state of the art of evaluation in neural language models. arXiv preprint arXiv:1707.05589 (2017)
  9. 9.
    Mueller, J., Thyagarajan, A.: Siamese recurrent architectures for learning sentence similarity. In: AAAI, pp. 2786–2792 (2016)Google Scholar
  10. 10.
    Neculoiu, P., Versteegh, M., Rotaru, M., Amsterdam, T.B.V.: Learning text similarity with Siamese recurrent networks. ACL 2016, 148 (2016)Google Scholar
  11. 11.
    Oliner, A.J., Stearley, J.: What supercomputers say : a study of five system logs. In: DSN, pp. 575–584. IEEE (2007)Google Scholar
  12. 12.
    Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. ICML 3(28), 1310–1318 (2013)Google Scholar
  13. 13.
    Rippel, O., Paluri, M., Dollar, P., Bourdev, L.: Metric learning with adaptive density discrimination. arXiv preprint arXiv:1511.05939 (2015)
  14. 14.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  15. 15.
    Socher, R., Ganjoo, M., Manning, C.D., Ng, A.: Zero-shot learning through cross-modal transfer. In: Advances in Neural Information Processing Systems, pp. 935–943 (2013)Google Scholar
  16. 16.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Thaler, S., Menkovski, V., Petkovic, M.: Unsupervised signature extraction from forensic logs. In: Altun, Y., et al. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10536, pp. 305–316. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-71273-4_25CrossRefGoogle Scholar
  18. 18.
    Tieleman, T., Hinton, G.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)Google Scholar
  19. 19.
    Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., Wu, Y.: Learning fine-grained image similarity with deep ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1386–1393 (2014)Google Scholar
  20. 20.
    Xing, E.P., Jordan, M.I., Russell, S.J., Ng, A.Y.: Distance metric learning with application to clustering with side-information. In: Advances in Neural Information Processing Systems, pp. 521–528 (2003)Google Scholar
  21. 21.
    Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 (2014)
  22. 22.
    Zhou, Z.H.: A brief introduction to weakly supervised learning. Natl. Sci. Rev. 5, 44–53 (2017)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Stefan Thaler
    • 1
    Email author
  • Vlado Menkovski
    • 1
  • Milan Petkovic
    • 1
    • 2
  1. 1.Technical University of EindhovenEindhovenNetherlands
  2. 2.Philips Research LaboratoriesEindhovenNetherlands

Personalised recommendations