Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Chatbot Research and Design

CONVERSATIONS 2019: Chatbot Research and Design pp 94–110Cite as

Conversational Web Interaction: Proposal of a Dialog-Based Natural Language Interaction Paradigm for the Web

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11970))

Abstract

This paper lays the foundation for a new delivery paradigm for web-accessible content and functionality, i.e., conversational interaction. Instead of asking users to read text, click through links and type on the keyboard, the vision is to enable users to “speak to a website” and to obtain natural language, spoken feedback. The paper describes how state-of-the-art chatbot technology can enable a dialog between the user and the website, proposes a reference architecture for the automated inference of site-specific chatbots able to mediate between the user and the website, and discusses open challenges and research questions. The envisioned, bidirectional dialog paradigm advances current screen reader technology and aims to benefit both regular users in eyes-free usage scenarios as well as visually impaired users in everyday scenarios.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://html.spec.whatwg.org/multipage/microdata.html.

References

  1. Ahmed, F., Borodin, Y., Soviak, A., Islam, M., Ramakrishnan, I., Hedgpeth, T.: Accessible skimming: faster screen reading of web pages. In: UIST, pp. 367–378. ACM (2012)

    Google Scholar 

  2. Akpinar, M.E., Yeşilada, Y.: Discovering visual elements of web pages and their roles: users’ perception. Interact. Comput. 29(6), 845–867 (2017)

    Article  Google Scholar 

  3. Amershi, S., et al.: Guidelines for human-AI interaction. In: Proceedings of CHI 2019, p. 3. ACM (2019)

    Google Scholar 

  4. Asakawa, C.: What’s the web like if you can’t see it? In: W4A, pp. 1–8. ACM (2005)

    Google Scholar 

  5. Ashok, V., Borodin, Y., Puzis, Y., Ramakrishnan, I.: Capti-speak: a speech-enabled web screen reader. In: W4A, p. 22. ACM (2015)

    Google Scholar 

  6. Ashok, V., Puzis, Y., Borodin, Y., Ramakrishnan, I.: Web screen reading automation assistance using semantic abstraction. In: IUI, pp. 407–418. ACM (2017)

    Google Scholar 

  7. Asri, L.E., et al.: Frames: a corpus for adding memory to goal-oriented dialogue systems. arXiv preprint arXiv:1704.00057 (2017)

  8. Bigham, J.P.: Accessmonkey: enabling and sharing end user accessibility improvements. ACM SIGACCESS Access. Comput. 89, 3–6 (2007)

    Article  Google Scholar 

  9. Bigham, J.P., Lau, T., Nichols, J.: Trailblazer: enabling blind users to blaze trails through the web. In: IUI, pp. 177–186. ACM (2009)

    Google Scholar 

  10. Bigham, J.P., Lin, I., Savage, S.: The effects of not knowing what you don’t know on web accessibility for blind web users. In: ACM SIGACCESS. ACM (2017)

    Google Scholar 

  11. Billah, S.M., Ashok, V., Porter, D.E., Ramakrishnan, I.: Ubiquitous accessibility for people with visual impairments: are we there yet? In: CHI, pp. 5862–5868. ACM (2017)

    Google Scholar 

  12. Borodin, Y.: Automation of repetitive web browsing tasks with voice-enabled macros. In: Proceedings of SIGACCESS 2008, pp. 307–308. ACM (2008)

    Google Scholar 

  13. Borodin, Y., et al.: Hearsay: a new generation context-driven multi-modal assistive web browser. In: WWW, pp. 1233–1236. ACM (2010)

    Google Scholar 

  14. Christian, K., Kules, B., Shneiderman, B., Youssef, A.: A comparison of voice controlled and mouse controlled web browsing. In: Proceedings of the Fourth International ACM Conference on Assistive Technologies, pp. 72–79. ACM (2000)

    Google Scholar 

  15. Conradi, J., Alexander, T.: Analysis of visual performance during the use of mobile devices while walking. In: Harris, D. (ed.) EPCE 2014. LNCS (LNAI), vol. 8532, pp. 133–142. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07515-0_14

    Chapter  Google Scholar 

  16. De Rosa, A., Justice, D.: WebReader: a screen reader for everyone, everywhere. In: Proceedings of the 13th Web for All Conference, p. 10. ACM (2016)

    Google Scholar 

  17. Diggs, J., Craig, J., McCarron, S., Cooper, M.: Accessible rich Internet applications (WAI-ARIA) 1.1 (2016). Accessed 24 Apr 2016

    Google Scholar 

  18. Fecke, A., Jeleniowski, S., Joisten, M.: Accessible websites for the visually impaired: guidelines for designers. In: Mensch & Computer Workshopband, pp. 419–422 (2015)

    Google Scholar 

  19. Giraud, S., Thérouanne, P., Steiner, D.D.: Web accessibility: filtering redundant and irrelevant information improves website usability for blind users. Int. J. Hum Comput Stud. 111, 23–35 (2018)

    Article  Google Scholar 

  20. Guerreiro, J., Gonçalves, D.: Faster text-to-speeches: enhancing blind people’s information scanning with faster concurrent speech. In: ACM SIGACCESS, pp. 3–11. ACM (2015)

    Google Scholar 

  21. Guha, R.V., Brickley, D., Macbeth, S.: Schema.org: evolution of structured data on the web. Commun. ACM 59(2), 44–51 (2016)

    Article  Google Scholar 

  22. Harper, S., Patel, N.: Gist summaries for visually impaired surfers. In: ACM SIGACCESS Conference on Computers and Accessibility, pp. 90–97. ACM (2005)

    Google Scholar 

  23. Hyman Jr., I.E., Boss, S.M., Wise, B.M., McKenzie, K.E., Caggiano, J.M.: Did you see the unicycling clown? Inattentional blindness while walking and talking on a cell phone. Appl. Cogn. Psychol. 24(5), 597–607 (2010)

    Article  Google Scholar 

  24. Inan, F.A., Namin, A.S., Pogrund, R.L., Jones, K.S.: Internet use and cybersecurity concerns of individuals with visual impairments. J. Educ. Technol. Soc. 19(1), 28–40 (2016)

    Google Scholar 

  25. Insurance, L.M.: Pedestrian safety survey (2013). https://bit.ly/2WIDeOM

  26. Lau, T., Cerruti, J., Manzato, G., Bengualid, M., Bigham, J.P., Nichols, J.: A conversational interface to web automation. In: UIST, pp. 229–238. ACM (2010)

    Google Scholar 

  27. Lazar, J., Allen, A., Kleinman, J., Malarkey, C.: What frustrates screen reader users on the web: a study of 100 blind users. Int. J. Hum.-Comput. Interact. 22(3), 247–269 (2007)

    Article  Google Scholar 

  28. Leshed, G., Haber, E.M., Matthews, T., Lau, T.: CoScripter: automating & sharing how-to knowledge in the enterprise. In: CHI, pp. 1719–1728. ACM (2008)

    Google Scholar 

  29. Mahmud, J.U., Borodin, Y., Ramakrishnan, I.: Csurf: a context-driven non-visual web-browser. In: Proceedings of WWW 2007, pp. 31–40. ACM (2007)

    Google Scholar 

  30. Michail, S., Christos, K.: Adaptive browsing shortcuts: personalising the user interface of a specialised voice web browser for blind people. In: Data Engineering Workshop, 2007, pp. 818–825. IEEE (2007)

    Google Scholar 

  31. Mika, P.: On schema.org and why it matters for the web. IEEE Internet Comput. 19(4), 52–55 (2015)

    Article  Google Scholar 

  32. Murphy, E., Kuber, R., McAllister, G., Strain, P., Yu, W.: An empirical investigation into the difficulties experienced by visually impaired Internet users. Univ. Access Inf. Soc. 7(1–2), 79–91 (2008)

    Article  Google Scholar 

  33. Nasar, J.L., Troyer, D.: Pedestrian injuries due to mobile phone use in public places. Accid. Anal. Prev. 57, 91–95 (2013)

    Article  Google Scholar 

  34. Oney, S., Lundgard, A., Krosnick, R., Nebeling, M., Lasecki, W.S.: Arboretum and arbility: improving web accessibility through a shared browsing architecture. In: UIST, pp. 937–949. ACM (2018)

    Google Scholar 

  35. Oshry, M., Auburn, R., Baggia, P., Bodell, M., Burke, D., Burnett, D., et al.: Voice extensible markup language (voicexml) 2.1. w3c recommendation (2007)

    Google Scholar 

  36. Plessers, P., et al.: Accessibility: a web engineering approach. In: Proceedings of WWW 2005, pp. 353–362. ACM (2005)

    Google Scholar 

  37. Power, C., Freire, A., Petrie, H., Swallow, D.: Guidelines are only half of the story: accessibility problems encountered by blind users on the web. In: Proceedings of CHI 2012, pp. 433–442. ACM (2012)

    Google Scholar 

  38. Rohrbach, M., Qiu, W., Titov, I., Thater, S., Pinkal, M., Schiele, B.: Translating video content to natural language descriptions. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 433–440 (2013)

    Google Scholar 

  39. Ronallo, J.: Html5 microdata and schema.org. Code4Lib J. (16) (2012)

    Google Scholar 

  40. Sato, D., Takagi, H., Kobayashi, M., Kawanaka, S., Asakawa, C.: Exploratory analysis of collaborative web accessibility improvement. ACM TACCESS 3(2), 5 (2010)

    Google Scholar 

  41. Sears, A., Lin, M., Jacko, J., Xiao, Y.: When computers fade: pervasive computing and situationally-induced impairments and disabilities. HCI Int. 2, 1298–1302 (2003)

    Google Scholar 

  42. Takagi, H., Kawanaka, S., Kobayashi, M., Itoh, T., Asakawa, C.: Social accessibility: achieving accessibility through collaborative metadata authoring. In: ACM SIGACCESS Conference on Computers and Accessibility, pp. 193–200. ACM (2008)

    Google Scholar 

  43. Tran, K., et al.: Rich image captioning in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 49–56 (2016)

    Google Scholar 

  44. Voykinska, V., Azenkot, S., Wu, S., Leshed, G.: How blind people interact with visual content on social networking services. In: CSCW, pp. 1584–1595. ACM (2016)

    Google Scholar 

  45. Wang, K.: SALT: a spoken language interface for web-based multimodal dialog systems. In: Seventh International Conference on Spoken Language Processing (2002)

    Google Scholar 

  46. WWW-Consortium et al.: Web content accessibility guidelines (WCAG) 2.1 w3c candidate recommendation (2018). Accessed 30 Jan 2018

    Google Scholar 

  47. Yesilada, Y.: Web page segmentation: a review (2011)

    Google Scholar 

  48. Zhu, S., Sato, D., Takagi, H., Asakawa, C.: Sasayaki: an augmented voice-based web browsing experience. In: Proceedings of SIGACCESS 2010. ACM (2010)

    Google Scholar 

Download references

Acknowledgement

The study was supported by the Russian Science Foundation (project n. 19-18-00282).

Author information

Authors and Affiliations

  1. University of Trento, Trento, Italy

    Marcos Baez & Fabio Casati

  2. Politecnico di Milano, Milan, Italy

    Florian Daniel

  3. Tomsk Polytechnic University, Tomsk, Russia

    Fabio Casati

Authors
  1. Marcos Baez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Florian Daniel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabio Casati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcos Baez .

Editor information

Editors and Affiliations

  1. SINTEF, Oslo, Norway

    Asbjørn Følstad

  2. University of Amsterdam, Amsterdam, The Netherlands

    Theo Araujo

  3. CERTH-ITI, Thessaloniki, Greece

    Symeon Papadopoulos

  4. University of Leicester, Leicester, UK

    Effie Lai-Chong Law

  5. University of Agder, Grimstad, Norway

    Ole-Christoffer Granmo

  6. University of Edinburgh, Edinburgh, UK

    Ewa Luger

  7. University of Oslo, Oslo, Norway

    Petter Bae Brandtzaeg

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baez, M., Daniel, F., Casati, F. (2020). Conversational Web Interaction: Proposal of a Dialog-Based Natural Language Interaction Paradigm for the Web. In: Følstad, A., et al. Chatbot Research and Design. CONVERSATIONS 2019. Lecture Notes in Computer Science(), vol 11970. Springer, Cham. https://doi.org/10.1007/978-3-030-39540-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39540-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39539-1

  • Online ISBN: 978-3-030-39540-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation