1 Introduction

Technological advances in sensors, ubiquitous technologies and mobile devices open new horizons for Personal Informatics (PI). Personal Informatics systems have the goal of exploiting technology for gathering data on different aspects of people daily lives. Nowadays, a large variety of data can be somehow collected by means of PI technologies: from the mood or the glucose level in the blood, to performance values, habits and actions. Collecting these data allows users to self-monitor their behaviors in a way inconceivable without such technological means.

The first PI systems were conceived mainly for clinical purposes, to help patients in tracking dysfunctional behaviors. They then started to be used outside the clinical setting by researchers and technology enthusiasts, such as the members of the Quantified Self movement [1]. Today, we assist to their commercial diffusion at consumer level, widening the possibility of their widespread adoption and their integration in everyday life. This nonetheless poses a double problem. First, common users are unfamiliar with PI tools and may have a misperception of their limits and potentialities. Second, we believe that current PI tools are not designed with enough understanding of common users’ needs, desires and problems they may encounter in their everyday lives [2].

In particular, manually collecting data with actual PI tools is cumbersome. Although it is possible to imagine that many behavioral data in the future will be tracked automatically thanks to the advancement of ubiquitous and wearable technologies, many aspects of people’s behavior, such as emotions and mood, will continue to rely on self-reporting, since they involve cognitive and interpretative components that cannot be automatically collected. In fact, even if it can be possible to obtain physiological measurements and objective measures of the behaviors expressed by the emotions, it is not possible to measure the way in which the subject experiences the physiological and behavioral changes other than through self-report [3]. However, the act of self-reporting requires an high compliance to be effective, while common users often fail in reporting their data due to forgetfulness, lack of time and motivation [2].

Tangible User Interfaces (TUIs) is a new type of user interface that exploits physical affordances for connecting the digital world with the physical one [4]. TUIs seem to be more inviting than Graphical User Interfaces (GUIs) when a given task is not appealing [5], providing a more involving experience that may enhance the number of actions performed by users [6]. Using TUIs for self-reporting can make users more physically engaged, providing richer feedback during the interaction [7]. The act of self-reporting, from this point of view become a sort of “physical activity”, in which users “play” with objects giving, at the same time, information about their behaviors.

Thus, we propose a Personal Informatics Tangible Interface able to support users in self-reporting their mood. This solution, built on an Arduino board, allows for self-reporting of emotional states in an amusing, simple and appealing way by means of a physical object. The data gathered by our TUI could be then analyzed, correlated with other personal data and fed back to the user in meaningful visualizations, in order to provide user with a picture of her states. To this aim, we integrated our TUI in an existent framework, Specch.io,Footnote 1 a quantified self platform for seamless data collection, mash-up, visualization and exploration of personal data. The Specch.io’s goal is to raise awareness on individual life patterns and about aspects of the “self” that can be hardly captured from a subjective point of view [8]. In this paper, however, we will focus on describing our solution’s architecture as a stand-alone system, even if the Specch.io’s context is useful to fully understand our solution’s potentialities.

The paper is structured as follow. Section 2 provides the most relevant related work in relation of technologies for self-tracking of mood and emotions. Section 3 provides a usage scenario that describes a possible context of use for our solution. Section 4 describes the general architecture of the system, and delves into the implementation details both from server and client point of view. Finally, Sect. 6 concludes the paper providing the future directions of the work.

2 Related Work

Emotion tracking is a widely-studied topic and several applications, research works and technological tools addressed to this aim, either self-reported or automatic, have been developed so far. In this section, we only provide an overview of the most relevant in comparison with our solution, focusing on self-reporting of emotions.

There are a lot of systems with the aim of collecting user’s emotions for therapeutic and rehabilitation purposes. For example, we can cite, Mobile Mood Diary [9], a mobile and online symptom tracking tool for adolescents with mental problems, and MoodTracker [10], an online tool for manage depression, bipolar disorder, or anxiety.

There are also a lot of commercial applications and devices with the goal of promoting a self-knowledge through a visual exploration of the gathered data. In fact, beside the data collection, they are able to suggest patterns, trends and correlations between emotions changes and habits or occurred events. For example, we can cite T2 Mood Tracker [11], an app designed to help users in tracking emotional experiences over time and sharing these data with an health care provider, and MedHelp Mood Tracker [12], that tracks general mood, as well as symptoms and treatments related to specific mood disorders.

Other apps, like Mood Panda [13], add the social component to the tracking and visualization of data: users can share their mood with friends in order to be supported by them and support them.

All these apps require the user to suspend her current activity to interact with the phone and this makes the tracking burdensome and annoying, with the risk that in the long term the user could give up.

Our solution supports users in self-collecting emotions by providing a tangible interface able to motivate her to report her data. This is different from other systems presented above that make use of a traditional GUI on the screen of mobile device. Our aim is to overcome such limitations in using these tools and facilitate this task for the user, by making it straightforward with a tangible interface.

3 Usage Scenario

In order to better understand the motivation of our work, we provide a usage scenario for our TUI integrated in the Specch.io framework.

Peter is a 35 years old programmer who lives in a big city. He has a sedentary lifestyle, even if he enjoys walking. He uses several personal computers during the day: of course for his job and also at home to be informed and to keep in touch with friends. Generally, he is very anxious and he would like to live a more relaxing life. He decides to use a personal informatics tool in order to gain awareness of some aspects of his everyday life that sometimes prevents him from truly relaxing. The TUI positioned on Peter’s desk (both at home and at workplace) reminds him (with a light jingle or only with its presence) to periodically track his mood. Thus, Peter, rotating the object, reports his data every evening.

To increase his self-awareness, he also decides to install, in his home entrance, the Specch.io system. It identifies Peter when he reflects himself and shows all the information and correlations gathered about his habits and his moods, without the use of a personal computer. Thanks to a tangible/gestural interface, Peter can interact with the mirror and he is able to explore his mood in the last weeks and months and correlations among data. He can see the aggregated data about emotions and other contextual and behavioral data (e.g., other physical parameters collected with other PI devices, such where he has been, what he has done, who he has met, etc.…). This allows him to discover an unexpected correlation among his lunch habits and his mood: every time he does not make sport at lunch, he feels anxious. Thus, he considers to change his habits in order to take a break at lunch and make some sport.

4 Our Solution

The idea is to create a portable, entertaining and, above all, not burdensome platform that can support users in self-collecting their mood states in several moments of the day.

Our solution is based on a client-server architecture (see Fig. 2) where the client is a tangible user interface (TUI), i.e. a physical smart object that the user can manipulate in order to communicate her mood, while the server has the aim to store the received information in a database, to enable other platforms (such as Specch.io) to integrate them. The client and the server communicate each other with a Wi-Fi connection.

We implemented our TUI by means of a wooden cube with each face representing a specific mood state (see Fig. 1). It has been decided to monitor six different mood levels. Users can communicate their mood by moving the cube and positioning it on the table/desk so that the relevant face is the top one. The TUI is able to store these data since it is built on an Arduino board [14]. Arduino is an open-source platform for building digital devices and interactive objects that can sense and control the physical world. When the TUI recognizes the chosen face (Mood Manager), the information about the mood state and the time (Time Manager) are managed by the Data Manager and saved into a storage device, possibly with other information gathered from some sensors such as temperature and barometric pressure. Later, these data are read from the storage device and are wirelessly sent to the remote server by the Communication Manager. On the server side, the data received by the Communication Manager are sent to the Data Manager that saves them into a storage device. Then some checks are performed and if the data are considered valid, they are stored also into a database.

Fig. 1.
figure 1

The TUI Object

Full size image
Fig. 2.
figure 2

System architecture

Full size image

Then the user, through her PC/device or Specch.io, can be notified of specific conditions, patterns and correlations (e.g. every time she meets a particular person, she gets happy).

5 Implementation Details

In this section, we provide a technical description of our solution, both for the client and the server sides.

5.1 Client Side: TUI Implementation

To support users in self-collecting their mood states, we have to recognize the TUI face, matching it with an event or a specific time, gather some data from several sensors, send this information to a remote server and save it in a database. In order to accomplish these tasks, we chose to realize a TUI based on an Arduino platform.

Hardware architecture. We created a first prototype using an Arduino Uno board. Several parts have been added to the platform:

  • an inertial measurement unit (IMU) is used to recognize which face of the TUI the user selects. We used an IMU with 10 degrees of freedom (SEN0140 by dfRobotFootnote 2). It integrates an accelerometer, a magnetometer, a gyro and a barometric pressure and temperature sensor. To communicate with this sensor, we use a dedicated free library, FreeIMU [15];

  • a Real Time Clock is necessary to get the current date/time and allow for associating the emotional state to a particular event or time. It holds a battery;

  • an SD card reader stores data before they are sent to the remote server and some initialization parameters used by the platform, such as Wi-Fi network name and passphrase and server IP and port;

  • a Wi-Fi shield enables the wireless connection to the remote server. We used a WizFi250 shield by SeeedstudioFootnote 3;

  • a buzzer is used to alert the user that the platform is waiting for her input;

  • some RGB LEDs show the state of the platform to the user.

While the Wi-Fi shield is plugged directly into the Arduino Uno board, the remaining parts are mounted on a breadboard and then connected to the board. After some test, we need to move to the Arduino Mega board that provides more memory space for the firmware.

Finally, we designed the physical object to contain the platform. We had to satisfy some constraints: (i) internal dimension should be at least 15 × 10 cm in order to contain the Arduino Mega platformFootnote 4 and all the other components, (ii) metal case is not appropriate due to the wireless activity, and (iii) we had to detect 6 different states. The simplest object that satisfies these constraints is a cube. We decided to build the cube in wooden material with six faces as shown in Fig. 1.

Software modules. We envisaged a scenario where the user tracks her mood at home or in the workplace with our TUI. We can take for granted that, in such contexts, the TUI is always on and placed in an area covered by a Wi-Fi network. The user can interact with the TUI at any time during the day: it selects a face of the cube that represents her mood (the one facing upward), then some parameters are automatically gathered and all the data are stored into the SD card. Then, those data are read and sent to a remote server. Every interaction is temporarily stored on the SD card, marked with a timestamp. The SD card also contains some files used to configure the platform:

  • Wi-fi.txt contains a list of known Wi-Fi networks (SSID, passphrase, authentication type, BSSID, channel);

  • Server.txt contains information related to the remote server: IP address and TCP port where information is to be sent.

  • Request.txt contains the times of the day when the device has to remind the user that it is waiting for an input.

  • Param.txt contains some parameters described below.

The overall behavior of the client is modeled by the finite state machine shown in Fig. 3. In the following, we shortly describe the client side software modules.

Fig. 3.
figure 3

Finite state machine describing the TUI behavior

Full size image

Time Manager. The platform alerts the user that it is waiting for its input (depending on the times stored in the file Request.txt) by activating a buzzer which plays a jingle for one minute: if in the meanwhile the user interacts with the TUI, the song stops; otherwise, after one minute, the buzzer stops anyway not to burden the user.

Mood Manager. The system identifies as “user interaction” any handling of the TUI (detected by the IMU) that lasts for at least X seconds (the value is stored in the file Param.txt). Once a user interaction is detected, the procedure that recognizes the face of the cube selected by the user will start: if the system is not able to recognize the face (if, for example, the TUI is not properly placed on a face), a red LED is turned on intermittently and if it keeps failing to identify a face within a specified timeout (whose value stored in the file param.txt), it aborts the “user interaction” operation and reverts to wait for a new user input. Otherwise, once the face has been identified, a blue LED is turned on intermittently for Y seconds (the value is stored in the file Param.txt) as a feedback for the user that her choice was identified. During this period, the user has the possibility to change the selected face: in this circumstance, the blue LED is turned off, and will be turned on when the device will correctly detect the new selected face. Once the registration process has started (i.e. the detected movements last for more than X seconds), it cannot be stopped and the device, once identified the face, will necessarily save the data.

Data Manager. When the system is ready to store the data, the blue LED is turned off and a green one is turned on, alerting the user that it starts to save the data. To grant privacy and data security, every TUI has a unique identifier (TUI ID, i.e. its Wi-Fi MAC address), and every user has a username and password. This information is stored into the SD card and can be changed by the user. All the data both gathered from the sensors (the cube face, time, etc.) and others (TUI ID, username and password) are saved in a file on the SD card in a JSON string.

Communication Manager. Once the information is stored on the SD card, the device tries to send it to the remote server, using a known Wi-Fi network. The system will attempt to connect to any Wi-Fi network whose parameters are written in the file Wi-fi.txt. Once connected, it tries to establish a TCP connection to the remote server using the parameters stored in the file Server.txt. All data stored into the SD card will be then transferred to the server, including any other data that have not been previously transferred, due, for example, to a temporary Wi-Fi connection loss. If the transmission is successful, the data on the SD card are deleted and the system returns to its initial state (all LEDs off), waiting for the future user input.

5.2 Server Side Implementation

On the server side, there are two modules working together in order to manage the data gathered from the TUI: the Communication Manager and the Data Manager.

Communication Manager. It receives JSON data from the TUI Communication Manager and send it to the Data Manager.

Data Manager. It checks if the identifier, the username and the password contained in the incoming JSON string are correct, valid and exist in the central database.

If so, the data are stored in files on the server and sent to a document database (MongoDB); otherwise, data are only stored as files on the server for logging purposes and future checks but no data will be sent to the database.

6 Conclusion and Future Works

In this paper we propose a TUI which allows users to self report their mood. This tangible interface is intended to make the task of tracking the mood amusing, simple and immediate. We described a possible usage scenario motivating our work integrating it with an existent framework, Specch.io, the technical details of the implementation and the motivations of some design choices.

As future work, we plan to evaluate the platform with a group of users for a long period to collect their feedbacks about the TUI usage with respect to a traditional GUI. We are also planning to optimize our TUI, in order to create a smaller and, consequently, a more portable object. We will start by using Arduino Yun platform that integrates a Wi-Fi shield and an SD card reader.

On the other side, we want to integrate mood data with information taken from other sources: the system could automatically collect the context (place and people) in which the mood state arises, inferring it from e.g. the GPS sensor of user’s smartphone, events gathered from user’s social networks (Facebook, Twitter, WhatsApp, Google +), shared calendars (Google calendar), etc. Moreover, the platform could receive further information from other devices able to automatically detect physiological indicators (such as heart rate, body temperature, etc.).