In the current era of IoT, home appliances like dishwashers, washing machines, and coffee makers, etc. are being equipped with internet access and sensing resources to provide autonomous or semi-autonomous functionality to its users. It is foreseeable that these appliances will soon be capable to collaborate by sharing their capabilities enabling dynamic operations as per the context and user preferences. This scenario, however, will also add to the complexity of programming/customizing the operational behavior of the appliances. Hence, traditional control interfaces will not suffice. This paper presents a generic architecture through which such collaborative operations of home appliances can be achieved. To minimize complexity for end-users, a centralized drag-n-drop visual interface for operational customization of the home appliances has been presented. In this proof-of-concept implementation, Intel Edison Platforms with associated sensing and actuating components have been utilized to mimic the sensing, actuation and computational resources of future home appliances by utilizing local WIFI network for communication purposes. Simple collaboration scenarios have been executed to find the execution delay of the customized collaborative operations. Based on the same scenarios, a preliminary usability study, based on system usability scale (SUS), has been conducted for the centralized visual interface via two different groups of end-users (programming skills-based). The average SUS score of 79.2 by both groups backs the claim of reduced complexity and ease of use for the end-users.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Anderson, C. (2014). Makers: The new industrial revolution. Competitiveness Review,24(2), 147–149. https://doi.org/10.1108/CR-04-2013-0039.
Atzori, L., Iera, A., & Morabito, G. (2010). The internet of things: A survey. Computer Networks,54(15), 2787–2805. https://doi.org/10.1016/j.comnet.2010.05.010.
Baig, F., Beg, S., & Khan, M. F. (2013). Zigbee based home appliances controlling through spoken commands using handheld devices. International Journal of Smart Home,7(1), 19–26.
Brooke, J. (1996). SUS—A quick and dirty usability scale. Usability Evaluation in Industry,189(194), 4–7. https://doi.org/10.1002/hbm.20701.
Chen, Y., Li, M., Chen, P., & Xia, S. (2019). Survey of cross-technology communication for IoT heterogeneous devices. IET Communications,13(12), 1709–1720.
Chi, C. F., & Dewi, R. S. (2014). Visual and auditory icons for the intelligent building. In Proceedings of 2014 international conference on intelligent green building and smart grid, IGBSG 2014. https://doi.org/10.1109/IGBSG.2014.6835280.
De Roeck, D., Slegers, K., Criel, J., Godon, M., Claeys, L., Kilpi, K., & Jacobs, A. (2012). I would DiYSE for it! In Proceedings of the 7th Nordic conference on human-computer interaction making sense through design—NordiCHI’12 (p. 170). https://doi.org/10.1145/2399016.2399044.
De Russis, L., & Corno, F. (2015). HomeRules: A tangible end-user programming interface for smart homes. In Proceedings of the 33rd annual ACM conference extended abstracts on human factors in computing systems—CHI EA’15 (pp. 2109–2114). https://doi.org/10.1145/2702613.2732795.
Dirienzo, T. P., Krishnan, N. A., Srija, & Santos, J. R. (2014). Effects of smart appliances on residential consumption patterns. In 2014 IEEE systems and information engineering design symposium, SIEDS 2014, (vol. 0, no. O, pp. 188–192). https://doi.org/10.1109/SIEDS.2014.6829870.
Dixon, C., Mahajan, R., Agarwal, S., Brush, A. J., Lee, B., Saroiu, S., & Bahl, P. (2012). An operating system for the home. In Proceedings of the 9th USENIX conference on networked systems design and implementation. USENIX association (pp. 337–352). https://doi.org/10.2790/32946.
Gama, K., Touseau, L., & Donsez, D. (2012). Combining heterogeneous service technologies for building an Internet of Things middleware. Computer Communications,35(4), 405–417. https://doi.org/10.1016/j.comcom.2011.11.003.
Horton, W. (1996): Designing icons and visual symbols. In Conference on human factors in computing systems—Proceedings (pp. 371–372). https://doi.org/10.1145/257089.257378.
Horton, W. K. (1994). The icon book: Visual symbols for computer systems and documentation. Hoboken: Wiley.
Karlheinz, R., Rosenbauer, M., & Stickel, M. (2012). Domestic appliance with the programmable control module, US Patent, US20090165833A1.
Khan, S. U. Z., Shovon, T. H., Shawon, J., Zaman, A. S., & Sabyasachi, S. (2013). Smart box: A TV remote controller based programmable home appliance manager. In 2013 international conference on informatics, electronics, and vision, ICIEV 2013, (Iii). https://doi.org/10.1109/ICIEV.2013.6572610.
Kim, J. E., Barth, T., Boulos, G., Yackovich, J., Beckel, C., & Mosse, D. (2017). Seamless integration of heterogeneous devices and access control in smart homes and its evaluation. Intelligent Buildings International,9(1), 23–39.
King, N. (2003). Smart home–a definition. Milton Keynes: Intertek Research & Testing Center.
Kobus, C. B. A., Klaassen, E. A. M., Mugge, R., & Schoormans, J. P. L. (2015). A real-life assessment on the effect of smart appliances for shifting households’ electricity demand. Applied Energy,147, 335–343. https://doi.org/10.1016/j.apenergy.2015.01.073.
Park, H.-J., Kim, K.-T., & Lee, S.-W. (2015): Towards a smart TV control system based on steady-state visual evoked potential. In The 3rd international winter conference on brain-computer interface (pp. 1–2). https://doi.org/10.1109/IWW-BCI.2015.7073039.
Piyare, R. (2013). Internet of things: Ubiquitous home control and monitoring system using android based smart phone. International Journal of Internet of Things,2(1), 5–11. https://doi.org/10.5923/j.ijit.20130201.02.
Rode, J. A., Toye, E. F., & Blackwell, A. F. (2004). The fuzzy felt ethnography-understanding the programming patterns of domestic appliances. Personal and Ubiquitous Computing,8(3–4), 161–176. https://doi.org/10.1007/s00779-004-0272-0.
Schipor, O. A., Vatavu, R. D., & Vanderdonckt, J. (2019). Euphoria: A scalable, event-driven architecture for designing interactions across heterogeneous devices in smart environments. Information and Software Technology,109, 43–59.
Silva, B. N., Khan, M., Lee, K., Yoon, Y., Muhammad, D., Han, J., & Han, K. (2019). Ubiquitous RESTful smart home energy management system. In IEEE international conference on computational science and engineering (CSE) and IEEE international conference on embedded and ubiquitous computing (EUC) (pp. 11–15).
Smart appliances of the future. (2015): Retrieved November 11, 2015, from http://www.appliancereports.com/2015/06/smart-appliances-of-future.html.
Ur, B., McManus, E., Yong Ho, M. P., & Littman, M. L. (2014): Practical trigger-action programming in the smart home. In Proceedings of the 32nd annual ACM conference on human factors in computing systems—CHI’14 (pp. 803–812). https://doi.org/10.1145/2556288.2557420.
Ur, B., Yong Ho, M. P., Brawner, S., Lee, J., Mennicken, S., Picard, N., et al. (2016): Trigger-Action Programming in the Wild : An Analysis of 200, 000 IFTTT Recipes. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 3227–3231). https://doi.org/10.1145/2858036.2858556.
Wenbo, Y., Quanyu, W., & Zhenwei, G: (2015). Smart home implementation based on Internet and WiFi technology. In 34th IEEE Chinese control conference (CCC) (pp. 9072–9077).
Zhang, C. R., Zhang, X. K., & Yu, N. N. (2014). Research and design of touch-screen domestic universal remote controller with self-learning function. Applied Mechanics and Materials,599–601, 1028–1031. https://doi.org/10.4028/www.scientific.net/AMM.599-601.1028.
Zhang, T., Li, Q., & Ma, F. (2013): Remote control system of smart appliances based on the wireless sensor network. In 2013 25th Chinese control and decision conference (CCDC) (pp. 3704–3709). https://doi.org/10.1109/CCDC.2013.6561592.
Zhu, J., Jia, X., & Mei, X. (2015). Smart home control system based on internet of things. Applied Mechanics and Materials,738–739, 233–237. https://doi.org/10.4028/www.scientific.net/AMM.738-739.233.
This work was partly supported by Institute for Information and communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (No. 10043907, Development of high-performance IoT device and Open Platform with Intelligent Software) under the ITRC (Information Technology Research Center) support program (IITP-2015-H8501-15-1017) supervised by the IITP (Institute for Information and Communications Technology Promotion).
Conflict of interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Khan, M.S., Abrar, M.F., Kim, D. et al. Resource-based direct manipulation: a user-centric visual interface for operational customization of future smart appliances. Telecommun Syst (2020). https://doi.org/10.1007/s11235-020-00686-6
- Operational customization
- Smart appliances
- Virtual objects
- Direct manipulation
- User interface, system usability