Prototyping and Preliminary Evaluation of Sign Language Translation System in the Railway Domain

  • Cristina BattaglinoEmail author
  • Carlo Geraci
  • Vincenzo Lombardo
  • Alessandro Mazzei
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9176)


This paper presents the prototype and the preliminary evaluation of an automatic translation system developed in the LIS4ALL project. The system domain is the corpus of railway station announcements in Italian. The output of the system is a 3D animated avatar that signs announcements in Italian Sign Language. The preliminary evaluation, which measures the accuracy of the translations at the sentence level, relies through the BLEU-RAC4 metric, a variant of the traditional BLEU metric used to evaluate Machine Translation, specifically designed for sentence level evaluation. The aim of the evaluation is to compare the LIS4ALL translation outputs with the human counterparts.


Sign Language Machine Translation Regular Expression Relative Clause Prepositional Phrase 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work has been partially supported by the project LIS4ALL partially funded by Regione Piemonte, Innovation Hub for ICT, 2011–2014, POR-FESR 07–13. Part of the research leading to these results also received funding from the European Research Council under the European Union’s Seventh Framework Program (FP/2007–2013) / ERC Grant Agreement n. 324115-FRONTSEM (PI: Schlenker). Research was partially conducted at Institut d’Etudes Cognitives (ENS), which is supported by grants ANR-10-IDEX-0001-02 PSL and ANR-10-LABX-0087 IEC.


  1. 1.
    Acoustic Announcements Manual - MAS, Rete Ferroviaria Italiana (RFI) (2011). (in Italian)
  2. 2.
    Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., Abbott, S.: Tessa, a system to aid communication with deaf people. In: Proceedings of the Fifth International ACM Conference on Assistive Technologies, pp. 205–212. ACM (2002)Google Scholar
  3. 3.
    Foster, M.E., White, M.: Techniques for text planning with xslt. In: Proceedings of the Workshop on NLP and XML (NLPXML-2004): RDF/RDFS and OWL in Language Technology, pp. 1–8. Association for Computational Linguistics (2004)Google Scholar
  4. 4.
    Geraci, C., Mazzei, A.: Last train to “rebaudengo fossano”: the case of some names in avatar translation. In: Proceedings of the 6th Workshop on the Representation and Processing of Sign Languages, pp. 63–66 (2013)Google Scholar
  5. 5.
    Huenerfauth, M.: Generating American Sign Language classifier predicates for English-to-ASL machine translation. Ph. D. thesis, University of Pennsylvania (2006)Google Scholar
  6. 6.
    Koehn, P., Monz, C.: Manual and automatic evaluation of machine translation between european languages. In: Proceedings of the Workshop on Statistical Machine Translation, pp. 102–121. Association for Computational Linguistics (2006)Google Scholar
  7. 7.
    Lombardo, V., Battaglino, C., Damiano, R., Nunnari, F.: An avatar-based interface for the Italian sign language. In: International Conference on Complex, Intelligent and Software Intensive Systems, CISIS 2011, June 30 - July 2, 2011, Korean Bible University, Seoul, Korea, pp. 589–594 (2011)Google Scholar
  8. 8.
    Lombardo, V., Nunnari, F., Damiano, R.: A virtual interpreter for the Italian sign language. In: Allbeck, A., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS, vol. 6356, pp. 201–207. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  9. 9.
    Mantovan, L., Geraci, C.: The syntax of cardinal numerals in LIS. In: Presented at the 45th Annual Meeting of the North Esta Linguistic Society (NELT), October 31 - November 2. MIT, Cambridge (2014)Google Scholar
  10. 10.
    Mazzei, A., Lesmo, L., Battaglino, C., Vendrame, M., Bucciarelli, M.: Deep natural language processing for Italian sign language translation. In: Baldoni, M., Baroglio, C., Boella, G., Micalizio, R. (eds.) AI*IA 2013. LNCS, vol. 8249, pp. 193–204. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  11. 11.
    Morrissey, S., Way, A., Stein, D., Bungeroth, J., Ney, H.: Combining data-driven mt systems for improved sign language translation. In: Proceedings of Machine Translation Summit XI (MT 2007) (2007)Google Scholar
  12. 12.
    Morrissey, S., Way, A.: Lost in translation: the problems of using mainstream mt evaluation metrics for sign language translation. In: Proceedings of the 5th SALTMIL Workshop on Minority Languages, May 23, 2006, Genoa, Italy (2006)Google Scholar
  13. 13.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)Google Scholar
  14. 14.
    Reiter, E., Dale, R., Feng, Z.: Building Natural Language Generation Systems. MIT Press, Cambridge (2000) CrossRefGoogle Scholar
  15. 15.
    San-Segundo, R., Montero, J.M., Córdoba, R., Sama, V., Fernández, F., D’Haro, L.F., López-Ludeña, V., Sánchez, D., García, A.: Design, development and field evaluation of a Spanish into sign language translation system. Pattern Anal. Appl. 15(2), 203–224 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Segundo, R.S., Pérez, A., Ortiz, D., Fernando, D.L., Torres, M., Casacuberta, F.: Evaluation of alternatives on speech to sign language translation. In: INTERSPEECH, pp. 2529–2532. Citeseer (2007)Google Scholar
  17. 17.
    Song, X., Cohn, T., Specia, L.: Bleu deconstructed: designing a better mt evaluation metric. In: Proceedings of the 14th International Conference on Intelligent Text Processing and Computational Linguistics (CICLING) (2013)Google Scholar
  18. 18.
    Steedman, M.: The Syntactic Process. MIT Press, Cambridge (2000) Google Scholar
  19. 19.
    White, M.: Efficient realization of coordinate structures in combinatory categorial grammar. Res. Lang. Comput. 4(1), 39–75 (2006)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Cristina Battaglino
    • 1
    Email author
  • Carlo Geraci
    • 2
  • Vincenzo Lombardo
    • 1
  • Alessandro Mazzei
    • 1
  1. 1.CIRMA and Department of InformaticsUniversity of TorinoTurinItaly
  2. 2.CNRS, Institut Jean-NicodParisFrance

Personalised recommendations