Sequential Labeling with Structural SVM Under an Average Precision Loss

  • Guopeng Zhang
  • Massimo PiccardiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10029)


The average precision (AP) is an important and widely-adopted performance measure for information retrieval and classification systems. However, owing to its relatively complex formulation, very few approaches have been proposed to learn a classifier by maximising its average precision over a given training set. Moreover, most of the existing work is restricted to i.i.d. data and does not extend to sequential data. For this reason, we herewith propose a structural SVM learning algorithm for sequential labeling that maximises an average precision measure. A further contribution of this paper is an algorithm that computes the average precision of a sequential classifier at test time, making it possible to assess sequential labeling under this measure. Experimental results over challenging datasets which depict human actions in kitchen scenarios (i.e., TUM Kitchen and CMU Multimodal Activity) show that the proposed approach leads to an average precision improvement of up to 4.2 and \(5.7\,\%\) points against the runner-up, respectively.


Sequential labeling Structural SVM Average precision Loss-augmented inference 


  1. 1.
    De la Torre, F., Hodgins, J.K., Montano, J., Valcarcel, S.: Detailed human data acquisition of kitchen activities: the CMU-multimodal activity database (CMU-MMAC). In: CHI 2009 Workshops, pp. 1–5 (2009)Google Scholar
  2. 2.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2007 (VOC 2007) Results (2007).
  3. 3.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes (VOC) challenge. IJCV 88(2), 303–338 (2010)CrossRefGoogle Scholar
  4. 4.
    Joachims, T.: \({\rm SVM}^{\rm struct}\): Support vector machine for complex output 3.10 (2008).$svm_light$/$svm_struct$.html
  5. 5.
    Kishida, K.: Property of Average Precision and Its Generalization: An Examination of Evaluation Indicator for Information Retrieval Experiments. NII Technical report, National Institute of Informatics (2005)Google Scholar
  6. 6.
    Morgan, W., Greiff, W., Henderson, J.: Direct maximization of average precision by hill-climbing, with a comparison to a maximum entropy approach. In: HLT-NAACL 2004, HLT-NAACL-Short 2004, pp. 93–96 (2004)Google Scholar
  7. 7.
    Nilsson, D., Goldberger, J.: Sequentially finding the N-best list in hidden Markov models. In: IJCAI 2001, pp. 1280–1285 (2001)Google Scholar
  8. 8.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. IEEE Proc. 77, 257–286 (1989)CrossRefGoogle Scholar
  9. 9.
    Rosenfeld, N., Meshi, O., Tarlow, D., Globerson, A.: Learning structured models with the AUC loss and its generalizations. In: AISTATS 2014, pp. 841–849 (2014)Google Scholar
  10. 10.
    Taskar, B., Guestrin, C., Koller, D.: Max-margin Markov networks. In: NIPS, pp. 25–32 (2003)Google Scholar
  11. 11.
    Tenorth, M., Bandouch, J., Beetz, M.: The TUM kitchen data set of everyday manipulation activities for motion tracking and action recognition. In: IEEE International Workshop on Tracking Humans for the Evaluation of their Motion in Image Sequences (THEMIS), in Conjunction with ICCV 2009, pp. 1089–1096 (2009)Google Scholar
  12. 12.
    Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. JMLR 6, 1453–1484 (2005)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Vedaldi, A.: A MATLAB wrapper of \({\rm SVM}^{\rm struct}\) (2011).
  14. 14.
    Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms (2008).
  15. 15.
    Yilmaz, E., Aslam, J.A.: Estimating average precision with incomplete and imperfect judgments. In: ACM CIKM 2006, pp. 102–111 (2006)Google Scholar
  16. 16.
    Yue, Y., Finley, T., Radlinski, F., Joachims, T.: A support vector method for optimizing average precision. In: SIGIR, pp. 271–278 (2007)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Global Big Data Technologies CentreUniversity of Technology SydneyUltimoAustralia

Personalised recommendations