BricklAyeR: A Platform for Building Rules for AmI Environments in AR

  • Evropi Stefanidi
  • Dimitrios Arampatzis
  • Asterios LeonidisEmail author
  • George Papagiannakis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11542)


With the proliferation of Intelligent Environments, the need for configuring their behaviors to address their users’ needs emerges. In combination with the current advances in Augmented and Virtual Reality and Conversational Agents, new opportunities arise for systems which allow people to program their environment. Whereas today this requires programming skills, soon, when most spaces will include smart objects, tools which allow their collaborative management by non-technical users will become a necessity. To that end, we present BricklAyeR, a novel collaborative platform for non-programmers, that allows to define the behavior of Intelligent Environments, through an intuitive, 3D building-block User Interface, following the Trigger-Action programming principle, in Augmented Reality, with the help of a Conversational Agent.


Augmented Reality Ambient Intelligence Chatbot conversational agent End user programming Graphical human-computer interaction 


  1. 1.
    Dahl, Y., Svendsen, R.-M.: End-user composition interfaces for smart environments: a preliminary study of usability factors. In: Marcus, A. (ed.) DUXU 2011. LNCS, vol. 6770, pp. 118–127. Springer, Heidelberg (2011). Scholar
  2. 2.
    Dey, A.K., Sohn, T., Streng, S., Kodama, J.: iCAP: interactive prototyping of context-aware applications. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) Pervasive 2006. LNCS, vol. 3968, pp. 254–271. Springer, Heidelberg (2006). Scholar
  3. 3.
    Truong, K.N., Huang, E.M., Abowd, G.D.: CAMP: a magnetic poetry interface for end-user programming of capture applications for the home. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, pp. 143–160. Springer, Heidelberg (2004). Scholar
  4. 4.
    Ur, B., McManus, E., Pak Yong Ho, M., Littman, M.L.: Practical trigger-action programming in the smart home. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 803–812. ACM, New York (2014)Google Scholar
  5. 5.
    McNerney, T.S.: Tangible programming bricks: an approach to making programming accessible to everyone (1999)Google Scholar
  6. 6.
    Vygotsky, L.S.: Mind in Society. Harvard University Press, Cambridge (1978)Google Scholar
  7. 7.
    Zidianakis, E., Papagiannakis, G., Stephanidis, C.: A cross-platform, remotely-controlled mobile avatar simulation framework for AmI environments. In: SIGGRAPH Asia 2014 Mobile Graphics and Interactive Applications, p. 12. ACM (2014)Google Scholar
  8. 8.
    Nosek, J.T.: The case for collaborative programming. Commun. ACM 41, 105–108 (1998)CrossRefGoogle Scholar
  9. 9.
    Oliveira, I., Tinoca, L., Pereira, A.: Online group work patterns: how to promote a successful collaboration. Comput. Educ. 57, 1348–1357 (2011)CrossRefGoogle Scholar
  10. 10.
    Vacchetti, L., et al.: A stable real-time AR framework for training and planning in industrial environments. In: Ong, S.K., Nee, A.Y.C. (eds.) Virtual and Augmented Reality Applications in Manufacturing, pp. 129–145. Springer, London (2004). Scholar
  11. 11.
    Kateros, S., et al.: A comparison of gamified, immersive VR curation methods for enhanced presence and human-computer interaction in digital humanities. Int. J. Heritage Digit. Era. 4, 221–233 (2015)CrossRefGoogle Scholar
  12. 12.
    Söderberg, R., Wärmefjord, K., Carlson, J.S., Lindkvist, L.: Toward a Digital Twin for real-time geometry assurance in individualized production. CIRP Ann. 66, 137–140 (2017)CrossRefGoogle Scholar
  13. 13.
    Stojkoska, B.L.R., Trivodaliev, K.V.: A review of Internet of Things for smart home: challenges and solutions. J. Clean. Prod. 140, 1454–1464 (2017)CrossRefGoogle Scholar
  14. 14.
    Walch, M., Rietzler, M., Greim, J., Schaub, F., Wiedersheim, B., Weber, M.: homeBLOX: making home automation usable. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 295–298. ACM (2013)Google Scholar
  15. 15.
    Stefanidi, E., et al.: ParlAmI: a multimodal approach for programming intelligent environments. Technologies 7, 11 (2019)CrossRefGoogle Scholar
  16. 16.
    Korozi, M., Leonidis, A., Antona, M., Stephanidis, C.: LECTOR: towards reengaging students in the educational process inside smart classrooms. In: Horain, P., Achard, C., Mallem, M. (eds.) IHCI 2017. LNCS, vol. 10688, pp. 137–149. Springer, Cham (2017). Scholar
  17. 17.
    Neumann, U., Majoros, A.: Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In: 1998 Proceedings of IEEE Virtual Reality Annual International Symposium, pp. 4–11. IEEE (1998)Google Scholar
  18. 18.
    Kasahara, S., Niiyama, R., Heun, V., Ishii, H.: exTouch: spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, pp. 223–228. ACM (2013)Google Scholar
  19. 19.
    Zaeh, M.F., Vogl, W.: Interactive laser-projection for programming industrial robots. In: 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. ISMAR 2006, pp. 125–128. IEEE (2006)Google Scholar
  20. 20.
    Heun, V., Hobin, J., Maes, P.: Reality editor: programming smarter objects. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 307–310. ACM, New York (2013)Google Scholar
  21. 21.
    Heun, V., Kasahara, S., Maes, P.: Smarter objects: using AR technology to program physical objects and their interactions. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 961–966. ACM (2013)Google Scholar
  22. 22.
    Mateo, C., Brunete, A., Gambao, E., Hernando, M.: Hammer: an android based application for end-user industrial robot programming. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014)Google Scholar
  23. 23.
    Chong, J.W.S., Ong, S.K., Nee, A.Y.C., Youcef-Youmi, K.: Robot programming using augmented reality: an interactive method for planning collision-free paths. Robot. Comput.-Integr. Manuf. 25, 689–701 (2009)CrossRefGoogle Scholar
  24. 24.
    Greenlee, B.J., Karanxha, Z.: A study of group dynamics in educational leadership cohort and non-cohort groups. J. Res. Leadersh. Educ. 5, 357–382 (2010)CrossRefGoogle Scholar
  25. 25.
    Brown, B., et al.: Lessons from the lighthouse: collaboration in a shared mixed reality system. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 577–584. ACM, New York (2003)Google Scholar
  26. 26.
    Leonidis, A., Arampatzis, D., Louloudakis, N., Stephanidis, C.: The AmI-Solertis system: creating user experiences in smart environments. In: Proceedings of the 13th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications (2017)Google Scholar
  27. 27.
    Korozi, M.: Empowering intelligent classrooms with attention monitoring and intervention cycles (2017)Google Scholar
  28. 28.
    Weintrop, D., Wilensky, U.: To block or not to block, that is the question: students’ perceptions of blocks-based programming. In: Proceedings of the 14th International Conference on Interaction Design and Children, pp. 199–208. ACM (2015)Google Scholar
  29. 29.
    Blackmon, M.H., Polson, P.G., Kitajima, M., Lewis, C.: Cognitive walkthrough for the web. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 463–470. ACM (2002)Google Scholar
  30. 30.
    Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 152–158. ACM (1994)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Evropi Stefanidi
    • 1
  • Dimitrios Arampatzis
    • 1
  • Asterios Leonidis
    • 1
    Email author
  • George Papagiannakis
    • 1
    • 2
  1. 1.Institute of Computer Science (ICS)Foundation for Research and Technology – Hellas (FORTH)HeraklionGreece
  2. 2.Department of Computer ScienceUniversity of CreteHeraklionGreece

Personalised recommendations