1 Introduction

In order to provide more natural and intuitive user experiences, NUX (Natural User Experience) and NUI (Natural User Interface) Technology has been developed. It becomes more natural and intuitive by using gesture and speech to real-time interact with various sensors. These technologies focus on facilitating the use of future computing environments by keeping the human user in mind. The real-time interactive NUX/NUI technologies have rapidly emerged as an exciting new industry and life paradigm. These research has been the most comprehensive field focused on the various aspects of advances in information technologies, applications, and services. There might be many issues to realize and provide smart services and much effort and enormous attention have been focused on NUX/NUI technologies for the reality media services. Theoretical research contributions presenting new technologies, concepts, or analyses, reports on experiences and experiments of implementation and application of theories, and tutorials on new trends should be needed on IT research fields. For the aforementioned reasons, those research area intends to give an overview of the state-of-the-art of issues and solution guidelines for the Advanced Technologies and Applications for Interactive NUX/NUI. In addition, it provides completing the panorama of current research effort, which is widely inherent to topics of high interest for the advanced future technologies and applications for real-time reality media service with NUX/NUI. We decided the topics that have been more aggressively covered by journals in the advanced technologies and application for Interactive NUX/NUI.

2 Related works

Daxing Jin et al. proposed an approach to generate virtual agents that can support users for NUI-based applications through human-robot interaction (HRI) learning in a virtual environment [2]. The method was implemented in a virtual environment, which improves the learning speed, efficiency and safety of the learning procedure. The new method in which a virtual agent recognizes human goals with the goal mapping table and learns interaction using the reactive Q-learning was proposed. The proposed method was applied to a model of a typical house in virtual environment with virtual human performing daily-life activities such as washing, eating, and watching TV, and results show that the virtual agent can predict a human’s intent, identify actions that are helpful to the human, and can provide services 16 % faster than a virtual agent trained using traditional Q-learning.

Jong H. Park et al. implemented the touch screen-based smart multimedia devices that are utilized by many different people due to their portability and many inbuilt functions. Vulnerable security points of smart multimedia devices are complemented and the locking system for enhanced security (LSES), in which intuitive user interface provides convenience, is proposed. LSES reduces exposure risk factors with various input methods for the lock pattern.

Min-Feng Lee et al. presented an idea for data mining in emotion color with affective computing. This research applies an innovative way to measure and identify user’s emotion with different ingredient color. This detecting emotion framework found the relations between the color and emotion while uses the FCM model. The emotion responsive color was affected by a lot of factors, because every single individual has the unique emotion color pattern. This research shows the significant result in training and simulation convergence, and the emotion forecast accuracy is during in a well interval. Some of emotion word in different culture will present entirely different.

Dongkyoo Shin designed and implemented a multimodal bio-signal-based interface system based on EEG and ECG. They used EEG for detecting the concentration degree and ECG for the stress status. The system can diagnose both of the user’s degree of concentration and the stress status in real time, and they also demonstrated a sensor device that acquires continuous brainwave signals from the frontal lobe. This study can be applied to develop an easy-to-use next-generation interface device by human biometric signals. Since the real status of human body can be detected from bio signals [4].

Jung M. Oh et al. shows an idea for gaze behavior data profiling and analysis system platform based on visual content representation. The proposed system captures the gaze information from multiple users and provides the ability to measure the degree of visual content perception by the users through statistical analysis. Visual content representation scheme is presented for capturing and annotating the gaze behavior effectively. Information correlation property among multiple image frames is defined for providing the ability to analyze the pattern of perception of user based on complex visual contents [1].

Gwang S. Hong et al. presented the fast multi-feature pedestrian detection algorithm based on discrete wavelet transform for interactive driver assistance system. This research shows a fast scheme of pedestrian detection based on multi-feature combined of the HOG feature and DWT. They reduced the computational complexity comparing to other pedestrian detection algorithms.

Su Y. Lee et al. focuses on a new method to visualize weathering effects due to the physical changes of stone artifacts. Major parameters concerned with both the breakdown process and changes of properties of the particles that comprise the stone artifact are considered. They pay attention to the fact that the common and main cause of weathering is granular disintegration and that such disintegration depends on the porosity. In order to predict physical property changes under unpredictable natural environments, probability variables were applied. For the 3D expression of weathering phenomenon, the concept of voxels was used. For that purpose, surface modeling data expressed in polygons were converted into a voxelized solid model which is the exfoliation phenomenon of rock particles by granular disintegration was visualized in each voxel.

Gwanghyung Lee et al. showed a study which is to recognize intuitive hand gestures and map them on a monitor to allow people of all ages to easily use the NUI/NUX framework. The difference between this study and previous studies was the users who did not use markers and the NUI/NUX framework that was implemented with bare hands. In addition, the mouse area was implemented dynamically to allow people of all ages to use it, and intuitive and accurate mouse events were implemented using a one-hand mouse and two-hand mouse. The advantages of Kinect SDK and OpenNI were also merged.

Jin H. Lee et al. proposed a wearable system that helps visually impaired persons to walk to their destination. After a destination is selected, our system computes an optimal path and guides the user with a marker position and its identifier detected by a camera (indoors) or positioning data from a GPS receiver (outdoors). The main role of a guidance system is to quickly obtain the data using various sensors and map the extracted and processed content onto available user interfaces in the most efficient manner. They implemented a guidance system for visually impaired persons.

Svetlana Kim et al. developed the recommendation system for sharing economy based on multidimensional trust model. This system is widely adopted in today’s mainstream online sharing services, providing useful prediction of user’s rating or user’s preferences of sharing items. A key challenge of recommendation systems in sharing economy is to employ prediction algorithms to estimate the matching items with considering their interests and needs. Thus, they proposed a Multidimensional Trust (MT) model based on tensor factorization and proposed a predictive algorithm using Skyline algorithm. The matrix factorization is one of the most favored approaches to collaborative filtering but the model is not flexible enough to add contextual dimensions in a good manner. For solving the limitation, we presented an extension of the model to N-dimensions through the use of tensors. The result data of rating holds the N-dimensional relationship between user’s contextual information, users, and items.

Hak H. Choi researched the theoretical background of augmented reality techniques and Popcode, as well as actual cases where these technologies are used together. Based on this research, they established a new experimental research model to enhance the expressive techniques of promotional videos. Currently, augmented reality techniques and Popcode are receiving significant attention. But these technologies are not widely utilized, especially in cultural arts organizations. Combining these techniques using virtual reality provides less reality to consumers. But by using the proposed PARPV model, consumer interest tends to increase because they are immersed in the promotional content using augmented reality [3]. Promotional videos using these techniques provide an interactive experience to consumers, expanding the content’s space by combining it with virtual reality, and increasing the reality of the viewing experience.