Keywords

1 Introduction

Providing a hands-free interaction mode to interact with technology is an important feature which may in particular benefit people with severe physical disabilities, such as tetraplegics, who can not use their hands to interact with technology.

Interaction technologies such as gaze-based interaction and head movement have been explored to operate, e.g. a PC [2,3,4]. However, today’s ubiquitous technology interaction scenarios are much more tightly integrated in everyday activities. Prime examples might be reading the news on a smartphone or tablet while eating breakfast, or surfing the web on such a device while at the same time watching TV on a shared big screen. Since people with tetraplegia are often relying on human assistants to interact with technology, this parallel interaction and seamless switching between tasks increases the burden on both assistants and users. From our experience and interviews with people with tetraplegia, we know that they may also be reluctant to have their human assistant constantly switch between such small tasks [1]. On the other hand, technological devices that may support e.g. reading, are also bound to certain locations and positions and do not offer the flexibility that people without such disabilities enjoy. In this demonstration paper, we present SwipeBuddy, a mobile robot with a hands-free Interface enables such flexibility and enhances a human’s opportunities regarding task and attention switching.

2 The SwipeBuddy-System

SwipeBuddy is a physical robotic device that acts as a mobile ebook reader and photo browser and can be controlled by using head movements. Its main tasks are to (a) hold the digital device (e.g. Amazon Kindle) (b) provide an interaction mechanism with the device to swipe pages and (c) flexibly move around so that it does not interfere with parallel activities (e.g. eating). The SwipeBuddy acts as an r-c mobile ebook-reader that can be positioned using head movements.

Fig. 1.
figure 1

Draft concept of the SwipeBuddy. To increase mobility the robot is equipped with a continuous track vehicle propulsion. Furthermore, it features a tilting platform to change the tilt and therefore view-angle of the mobile device.

Full size image

2.1 Hardware Platform

The prototype of SwipeBuddy consists of a mobile robotic platform with a continuous track vehicle propulsion (caterpillar track) for high maneuverability, i.e. turning around its own axis. A mobile device (smartphone, Kindle etc.) is placed on a tilting platform that allows the user to easily manipulate the view-angle of the device. Furthermore, the robot has a mechanical swipe mechanism that helps flipping pages or pressing icons. The swiping mechanism consists of a tip of a stylus for capacitive touch displays and a motorized arm which provides contact pressure for swipe and scroll. Using a mechanical swipe mechanism enables a user to activate a swipe action with any application and with any device. A software controlled swipe would be device dependent and thus less flexible. A visual mock-up of the SwipeBuddy is depicted in Fig. 1.

2.2 Interaction Design

The interaction concept consists of a Magnetic-AngularRate-Gravity (MARG) sensor that is mounted on a headband. This setup enables capturing of head motion data along the roll, pitch and yaw axis of the head when a user wears the headband in order to control the swipe arm to switch pages of the device and furthermore steer the robot itself.

Robustness of the measured orientation depends on the underlying method of sensor data fusion since various disturbances introduce errors. In this work, we use a robust quaternion based filter approach to estimate orientation from MARG-sensors that was introduced by Wöhle and Gebhard [5] and is implemented on a standalone micro-controller. Since the orientation is measured relative to the global quantities (gravity and magnetic field), there are no environmental restrictions that would limit usage to a local use-case. We are using a self-designed sensor system that enables wireless communication via WiFi UDP. The orientation data is mapped on the actuators of the system to achieve the defined tasks.

In our interaction design, the user can switch between different modes to steer the robotic platform, change the tilt angle of the electronic device, and perform a forward or backward swipe action. Additionally, an idle mode is available to block all interactions to put the sensor headband on or off. In particular, the user is switching modes by performing a movement along the yaw-axis, like it is shown in Fig. 2. Movements in roll and pitch axis are used n each mode different i.a. driving, steering, tilting, and swiping.

To provide a visual feedback to the user about which mode is selected and to help to orient, 25 RGB LEDs are installed and used in a way that supports intuitive insight. For Example in driving mode SwipeBuddy is following a car metaphor. White headlamp and red tail lamps are turned on and the continuous track is illuminated green (see Fig. 3). A slightly moving in the roll axis which is used for steering is changing the green continuous track light to a yellow blinker to indicate a change of the steering direction on the respective side. Furthermore, all LEDs are mounted at special positions and on special parts of the robotic system where they could be recognized easily and show which parts are active.

Fig. 2.
figure 2

The modes of SwipeBuddy are arranged in a circular queue and can be selected by movement along the yaw-axis of the head in both directions.

Full size image
Fig. 3.
figure 3

The user is steering SwipeBuddy by his head movements. The white headlamps are turned on. (Color figure online)

Full size image

3 Conclusion

With the SwipeBuddy we presented a software- and device independent robotic assistant system to people with severe physical disabilities. It does not aim to replace human assistants but support them for very specific tasks where users might not feel comfortable constantly asking for help. Our first informal tests with people with tetraplegia showed that the SwipeBuddy could potentially be a very welcome helper device with positive feedback regarding the use-case as well as the overall interaction design. While our scenario focuses on people who are physically limited in the usage of their hands, the SwipeBuddy could also assist people whose hands are currently tied to other activities, e.g. in an industrial context (industry 4.0) where an operator might need their hands to operate a machine interface and at the same time need to read a digital manual.