Combining Generative Art and Physiological Information for New Situation of Garden Restaurant

Abstract

Restaurant consumers often spend most of their time queuing outside garden restaurants during mealtimes, waiting for a table. If restaurant owners and customers can effectively utilize these idle times, both of them will inevitably benefit. This study developed an interactive device that can be used to divert customers’ attention during waiting times and enable them to enjoy the fun of using this device, thereby helping restaurant owners promote their garden restaurants and local postcards.

A literature review on human–computer interaction, wearable devices, and generative art was conducted to propose the design principle of the interactive device. Subsequently, prototyping, user manual, and recommendations for users were developed, and the effectiveness of the prototype was assessed.

This study developed an interactive device with which users can use during their waiting or idling time. The prototype of this interactive device involves combining the concept of generative art and a pulse sensor. Participants can use the garden restaurant’s postcard as an interactive media, which can then be converted into images through image processing. According to rules and motion of a graphics, the prototype can provide unique visual feedback. In addition, the proposed prototype entails a wearable pulse sensor that enhances the graphic rules and motions, enabling users to see their own heartbeat information. Moreover, a novel interactive thinking for wearable devices was proposed.

The contributions of this study included (1) completing the prototyping of a pulse sensor based on the concept of generative arts, and (2) developing a novel wearable interactive device that can be used in garden restaurants.

1 Introduction

As information technology advances, digital media has become an integral part of people’s daily life, in which each information product is developed through a series of design thinking process. During this design process, interactive design is a crucial topic for designing an interactive device that satisfies user needs. The article will design the interactive device for garden restaurant, and arrange the customer at a waiting for idle time can interact with the device.

Restaurant customers often spend most of their time either queuing outside garden restaurants waiting for a table or waiting for their food after placing their orders. Moreover, some restaurants require customers to collect table numbers as a mean of acquiring a table (Fig. 1). If restaurant owners and customers can effectively utilize these idle times, both of them will inevitably benefit. Therefore, this study developed an interactive device for a garden restaurant, where customers can play and interact with it while waiting for a table or their meals, thereby effectively utilizing their idle times.

Fig. 1.

Dining procedure

Generative art is crucial for the development of new media art. Generative art is a unique type of art that combines technology with the concept of aesthetics, enabling computers to autonomously create artworks through a set of rules. Therefore, it is also referred to as an automatic art-creation system. In this study, the concept of generative art was integrated with the interactive system, thus allowing users to appreciate the beauty of generative art and enjoy the fun of interacting with the system.

We developed an interactive device by using a landscape postcard of a garden restaurant as the basis. When getting a table number, the restaurant offers customers landscape postcard and inform that the postcard can interact with the device. Hopes to promote local cultures and landscape through interaction, thereby raising the popularity of the restaurant. Heartbeat sensing is smart watches are equipped with the function. If a wearable device can provide various types of information such as users’ physiological information, or interact with peripheral devices, such device will be interesting application. In this study a wearable device that provides users’ heart rate information and immediately transmits information to the device. The objectives of this study were as follows:

(1) To Design a Generative Art-Based Interactive Device Integrated with a Pulse Sensor. By combining a postcard, image processing, and wearable technology, a prototype of an interactive device was proposed to provide customers with a device that enables them to enjoy the fun of an interactive experience. In addition, a novel concept related to the application of wearable devices was proposed.

(2) To Promote Local Attractions Through the Device and Enhance the Restaurant’s Popularity. Sceneries near the garden restaurant can be promoted through the postcard–device interaction, leaving a deep impression in users of the landscape presented on the postcard and thus to elevate restaurant popularity.

2 Related Work

In this study the interactive device involved image processing, pulse sensor and algorithm. The device can transmit heartbeat signals through a wearable device to a computer, and an automated function for graphical visualization. Therefore, we reviewed literature on human–computer interaction, wearable devices, and generative art.

2.1 Human Computer Interaction

Kantowitz and Sorkin (1983) proposed human factors view of the human operator in a work environment, stating that in human–computer interactions, the brain receives information that stimulates the human body to control the keyboard. Subsequently, the computer provides feedback on the screen, the information on which is transmitted back to the brain through the human eye. The brain continually receives information, which forms the basis for the human to make a decision. The concept of human factors has gradually evolved, forming the concept of human–computer interaction.

An interactive system considers the nature of a design, using user as the basis for system design and determining whether the entire human–computer interaction satisfied people’s needs (Benyon, 2010). The device applied computer vision as the human–computer interface. Computer vision is the capability of a machine to visualize and make judgments like how the human eyes do. The machine acquires images through a video camera, converting the captured objects into image information, which is then subjected to image processing. Image characteristics are obtained by analyzing the pixel distributions, brightness, and color of the processed image. Subsequently, the machine uses the processed information to make a decision and execute an action (Learned-Miller, 2007). In the design of the device comprising a wearable device and computer vision two input interface, to clarify the structure of Human-computer interaction is very important part.

2.2 Wearable Technology

“Wearable computing” is a study of physical devices that comprise microprocessor and sensor. Wearable computing involves converting detected physiological information into usable data. Wearable devices such as smart glasses, watches, clothing, and shoes can be attached to the human body (Mann, 1996). In recent years, smart watches have been widely used in people’s daily lives, and smart watches of various brands are equipped with a pulse sensor. Therefore, the application of smart watches will be one of the focus of future technology developments. This paper adopted a wearable device to detect users heartbeat in real-time. And input heartbeat information into the device, which then processes it through image processing and provides it as feedback to users. Through real-time interaction with field or device feature, the formation of a novel type of wearable device interaction.

2.3 Generative Art

This study employed wearable computing as a means of interacting with the wearable device. Wearable computing entails providing raw physiological data to the device. To achieve real-time visualization, we reviewed literature on generative art. Galanter (2003) defined generative art as follows:

Generative art refers to any art practice where the artist uses a system, such as a set of natural language rules, a computer program, a machine, or other procedural invention, which is set into motion with some degree of autonomy contributing to or resulting in a completed work of art.

In other words, using language programming or mechanical procedures to formulate rules and constraints enables a system to autonomously generate results that feature the following characteristics: chance, unpredictability, surprise, and indeterminacy (Monro, 2008). This design of visual feedback develop to definitions and rules for the graphics, and the two parts of the practice, hoping that through computational design, allowing users to experience a unique presentation.

2.4 Case Study

In this chapter provides a review of case studies on interactive installations and digital display. These cases are based on interactive experience, allowing users to attain pleasure and fun for of an interactive experience. We analyzed and compared human-computer interface, display and feedback for each case. Hoping to generalize the case of design methods, and learn design method from case. To develop design principles of prototype.

(1) Wooden Mirror (Rozin, 1999 ). “Wooden mirror” is an interactive art, designed by Rozin and exhibited in 1999. This work is a mechanical mirror that combining computer vision and a various materials. Device via camera to capture each pixels of grayscale image and each wooden chips corresponding part in image; by motor drive to achieve each wooden chips.

(2) Living Shapes Interactive Wall (Philips, 2012 ). Philips Electronics develop interactive intelligent mirror with OLED (Organic Light-Emitting) in 2012. Device uses infrared cameras to sense the body shape and display on the OLED wall. Each OLED with a unique display brightness, and achieve device illumination. “Living shapes interactive wall” is a set of interactive devices and stylish lighting device.

(3) Sketch Aquarium (teamLab, 2013 ). “Sketch Aquarium” is a digital interactive installation which combining with e-learning experience for children. User can get a draft of ocean creature, and allows users to freely draw. And then put the ocean creature in the scanner, people will see the creature come life on the wall. This work using artificial intelligence and simple human-computer interface to lead children learn graffiti.

(4) Visualizing Perlin Noise (Hingorani, 2014 ).“Visualizing Perlin Noise” is designed by Hingorani and displayed in 2014’s MAT 200C: Pattern Formation. This work is an interactive device of visualized Perlin noise, the device presents life cycle and motion trail of a particle by utilizing Kinect to identify the hand gesture and project the images in an interactive experience space. Besides the strong visual effect, users also can fell Generative art its feature of randomness, unpredictability, surprise and uncertainty.

(5) Time Traveler - The 120th Anniversary Exhibition on Chen Cheng-po (Bright Ideas Design, 2014 ). “Time traveler” is multi-screen projection interactive installation to recode painting of artist Chen Cheng-po each period. Through multi-screen projection will be converted into the artist’s creative period the year and complete record of Chen’s life course. This interactive installation using gesture recognition as human-computer interface, let user can waving right hand or left hand to browse painting.

2.5 Summary

This chapter presents a review of case studies on interactive interfaces and feedback in Table 1. The standing type of interactive installation typically used computer vision as the interactive interface. Specifically, the Sketch Aquarium is an interactive installation where participants can sketch pictures and interact with it through a device. The designer of the Sketch Aquarium considered using intuitive simply interactive interfaces that allow users to assimilate into a scenario. Regarding feedback design, the former two designs involve using a physical structure to achieve digital display, whereas the latter three designs entail using projection to digitally display. This study selected computer vision as the human-computer interface, and used a wearable device to simulate the pulse sensor functions of smart watches, thereby providing a novel thinking approach that enables users to experience the interactive functions of smart watches.

Table 1. Case studies summary

3 Design Method

3.1 Design Concept

Prototype systems are expected to effectively utilize the waiting time to divert the focus of users through fun of an interactive experience, and to promote postcards and restaurants popularity. Customers can interact with the device through the postcard by inserting the card into the device, which will then perform image processing, and display the images of postcard onto the screen as feedback for users. In addition, the pulse sensor function of the wearable device transmits users’ heartbeat information to the device, and the motion and speed of the graphics change according to users’ physiological information to generate unique feedback to users (Fig. 2).

Fig. 2.

Storyboard

In this study, the research method involved first conducting a background and literature review, compiling case studies relevant to interactive installation. Various human-computer interface, display, and feedback approaches were summarized to determine the design principles of various cases, which were then used as the basis for developing the system prototype in Table 2. The human-computer interface method of the prototype involved using computer vision to capture the image information on the postcard and displaying the image according to the concept of generative art. Moreover, the parameters produced when the graph changed according to the information generated from the pulse sensor were also one of the features of this work.

Table 2. Disign principles

3.2 Visual Feedback Design

The prototype displayed digital contents through Processing. Users can view a display that is achieved through design computation and is distinct from conventional image display methods. This section presents the definition and rules for a unit square planar, and how an image can be sketched using the motion of a graphics. The graphics definitions included the x-coordinates, y-coordinates, size, rotational angle, and quantity of squares, and the rules for a square planar comprised the x-axis and y-axis motion, and the colors of display (Tables 3 and 4). According to displacement rules, the system automatically generates unique feedback for users to experience (Fig. 3).

Table 3. Definition of graph

Table 4. Rule of graph

Fig. 3.

Visual feedback

3.3 Production of the Pulse Sensor

According to the Human factors View of the human operator in a work environment, we produced the framework diagram for the prototype (Fig. 4). Regarding the framework for a human–computer interaction, users insert the postcard into a device, subsequently activating the device. Device can use the camera to take pictures. When the computer captures an image, it stores the image to the database; processes the image; computes the graphics; and defines the rules of the graph. Then computed image is then displayed on the screen for users. The wearable device is a simulation of the smart watch; it wirelessly transmits the received heartbeat information to the computer, which converts the physical information into graphics so that the rules correspond to the information, thereby forming an environment where human–computer interactions take place.

Fig. 4.

Human-computer interaction framework (Source: Kantowitz and Sorkin 1983) (This article redraws).

The sensor for the system prototype was developed by integrating the computer vision technology and using the wearable device. The computer vision captures the image information of the postcard, and the wearable device is used to correspond users’ heart rate information to the rules of the graphic, thus leading users into the scenario.

(1) Integrating Computer Vision Technology. Prototype was used computer vision as the input for capturing images. It captures images, stores them, and algorithm according to processing procedures (Fig. 5), which involve three parts: device activation, image process, and data processing.

Fig. 5.

Processing procedures

Regarding device activation, Arduino was used as the interface for hardware and computer communication. Once users insert a postcard into the device, the device will be activated. For image processing, the camera automatically focuses on the image of the postcard and then captures it. The images are stored in the database and subjected to image processing. The device system extracts the processed image for graphics computation and then displays it on the screen for users to view.

(2) Using Wearable Device. A pulse sensor was used as the wearable device. The sensor captures heartbeat information expressed in terms of inter beat interval (IBI) and beats per minute (BPM). According to the developer of Oregon Scientific, the heart rate of an adult resting is approximately 60–80 bpm, and the heart rate of an adult working is approximately 140–180 bpm and a maximum of 190 bpm. Because a pulse sensor generates errors when in use, we adopted a heart rate of 50–200 bpm as the change parameter for the rule of graphics. Fixed graphics were used, and the heartbeat (in BPM and IBI) parameters were quantified and added to the base values (Table 5). The Arduino then wirelessly transmits the detected heartbeat information via Bluetooth to the computer (Fig. 6).

Table 5. Heartbeat correspondence table

Fig. 6.

Prototype of wearable device

4 Research and Analysis

The prototype comprised three portions: visual feedback, wearable device, and tangible user interface. This section describes the steps and interfaces of the interaction process and details the process through users interact with the interface (Fig. 7 and Table 6).

Fig. 7.

A schematic diagram

Table 6. Interactive process

5 Conclusions and Future Directions

(1) Conclusions. The contributions of this study included (1) completing the prototyping of a pulse sensor based on the concept of generative arts, and (2) developing a novel wearable interactive device that can be used in garden restaurants. The results can be provided as reference to future researchers.

(2) Future Directions. In this study, we applied the garden restaurant’s postcard and implemented the prototype. During the implementation process, the following three revisions to the prototype can be made in future:

• (a) Highlighting the application of heartbeat information derived from the wearable device.

  • 1. The heartbeat information did not effectively correspond to the result of the graph computation in the system prototype. Therefore, users could not know the changes in their heartbeat from the interactive device. The design for the graphics computation generated an exceedingly small parameter error and failed to identify the changes in the pulses. Thus, revisions should be made to these aspects of the system prototype.

• (b) Enriching the visualization approach.

  • 2. Users who used the interactive device mentioned that they were attracted to the content of the visual feedback during their first use. This prompted them to play with the device for the second time, but the experience was not as fun as the first time. This indicates that the visual feedback provided accorded with users’ expectation. Thus, future in-depth studies can be conducted according to the graphics computation, and try to create various graphics algorithm..

• (c) Assessing system display and effectiveness.

  • 3. Regarding the on-site exhibition and system evaluation, we intend to exhibit the system in the garden restaurant and conduct on-site observations and questionnaire survey to assess the effectiveness of the system and public participation. From this process, concrete recommendations for system modification can be obtained.