Keywords

1 Introduction

The complexity of automated productions systems increases constantly. Currently the introduction of Industry 4.0 combined with the Cyber-Physical Systems and techniques from the field of information technologies lead to new structures and higher degrees of automation. A result is the so-called Smart Factory [1].

The increased complexity of the factories leads to new requirements to human-machine interaction [2]. Regarding the lifecycle of automated production systems (Fig. 1) two phases with three user groups can be pointed out:

Fig. 1.
figure 1

Lifecycle of automated production systems [2]

Full size image
  1. 1.

    During the engineering phase the user group of system designer develop and design the whole production system. Different craft groups (e.g. mechanical engineers, electrical engineers, PLC software engineers etc.) work with their specific software tools sequential on the designing, configuring and building up the production systems.

  2. 2.

    During the operating phase two user groups play an important role:

    1. a.

      The operators supervise and control the assigned production line.

    2. b.

      The service and maintenance technicians keep the production ticking over.

  3. 3.

    The last phase “scrapping” is not considered.

In the engineering phase the “Digital Factory” approach becomes more and more relevant. In this approach, parts of the activities are performed in a virtual environment instead on real site. A so-called “digital twin” has to be modelled and used for the virtual commissioning. Figure 2 shows the difference between the “classical” and the “digital factory” approach.

Fig. 2.
figure 2

“Classical” vs. “Digital Factory” approach [2]

Full size image

In parallel, the field of mobile technologies increased. Nowadays mobile devices from web pads [3] over data glasses [4] to Virtual Reality (VR) devices are powerful and affordable. As a basis for further research an online user analysis is performed to collect user requirements and opinion regarding the usage of data glasses and VR devices especially in the domain of robotics.

2 Background and Related Work

The following subsections describes basic information on the Augmented Reality and the Virtual Reality similar to the explanation the participants of the analysis got.

2.1 Background on Augmented Reality (AR)

Augmented Reality (AR) is a visual extension of the reality. It combines digital information such as videos, pictures, texts etc. with data that exists in reality, for example the picture of a digital camera. The supplementary data can help the user to accomplish several tasks. The virtual extension occurs in real time so that relevant digital data can be retrieved by the user. Especially big companies such as Google and Apple are working on integrating AR into everyday life and developing new and more improved AR-Systems [5, 6].

The most known example for such a product are data glasses. These smart glasses are wearable AR-Systems that combine virtual reality with physical reality [7].

2.2 Background on Virtual Reality (VR)

Virtual Reality (VR) is a computer technology that creates a simulated environment. VR places the user inside a virtual word unlike other traditional user interfaces. The VR simulates several senses such as vision, hearing, touch and smell [8]. This creates a world without real objects that can be controlled by head and hand movements, the voice or the sense of touch [9].

Well known VR products are VR devices, e.g. the Oculus Rift, HTC Vive, PlayStation VR, FOVE, etc. These VR devices create a whole new virtual environment.

2.3 Augmented Reality vs. Virtual Reality

VR creates an imaginary reality and AR only upgrades the “real” reality.

3 The Carried-Out Survey

Based on a previous analysis [10] and to find out in which areas the research activities regarding the use of AR and the VR will be carried forward, 28 specialists participated in this survey. The survey was conducted as an online survey [11]. The robotic specialists were invited to participate in a robotic forum [12]. The participants were at the age of 20 to 60 years. Most of the participants are working in research and development, mostly in fields such as robotics, automation and engineering. The average professional experience of the participants is 7,5 years. Additional to the questions also explanatory figures were shown to the participants. The Figs. 3 and 5 are examples for such explanatory figures.

Fig. 3.
figure 3

Visualization of the robot working space

Full size image

3.1 Augmented Reality in Terms of Industrial Robots

The first part of the survey covers the use of AR in terms of industrial robots. The question to the participants was the participants if it is useful to simulate the following situations and data for example with smart glasses.

Visualization of the working space of an industrial robot.

The participants were asked, if it is useful to visualize the working space of an industrial robot. The Fig. 3 shows an illustration of such an Augmented Reality visualization.

78% of the participants rated the visualization of the working space of an industrial robot with good or very good, only 7% think the visualization is inadequate (Fig. 4). Certainly, this result is according to expectations, because the visualization of the working space (and the danger zone) is a useful feature. Interesting will be the implementation of this information in an Augmented Reality system.

Fig. 4.
figure 4

Rating of the visualization of the working space

Full size image

Visualization of the programmed movement profile of the robot.

Furthermore, the participants were asked if it is helpful to visualize the programmed movement profile of an industrial robot in the working space. For better understanding this scenario is visualized in the Fig. 5. This feature may reasonable for the onsite programming of the robot, e.g. for checking if the program fits the desired movement profile.

Fig. 5.
figure 5

Visualization of the programmed robot movement

Full size image

The positive result of this question is shown in the diagram below (Fig. 6). This diagram above shows that 79% of the participants think that the visualization of the programmed movement of an industrial robot is a good idea (sum of very good and good). Seven percent think the idea is inadequate.

Fig. 6.
figure 6

Rating of the visualization of the programmed robot movement

Full size image

Further questions regarding AR in terms of industrial robots.

The visualization of the working space and the visualization of the programmed movement were not the only questions asked in terms of this subject. The following table shows the results of the other questions (Table 1).

Table 1. Further results regarding the use of AR in the field of robotics
Full size table

The results of these questions are fairly positive. The only exception is the question if it is useful to show small animations of common tasks e.g., tool measurement. In this case, only 50% of the participants think it is a good idea to show these small animations via AR. The idea behind this question was to give service and maintenance technician animated instructions. In a previous analysis and research work this item was rated quite better [3, 10].

Which further situations and data can be visualized through AR.

The participants were asked to think of further situations and data that could be visualized with AR. The following answers were given:

  • In mobile robotics, it could improve the man-machine interaction and make it more secure.

  • The AR could visualize collisions risks in the working space of the robot within given movements.

  • Visualization of undefined objects in the working space of the industrial robot.

  • AR could show the current operating mode of the industrial robot.

  • Visualization of the axis positions. Especially for the axis four and six it is interesting to know how close they are to their limits. During the programming of the robot you often notice too late that they reached their limits.

  • Maintenance and repair of the robot: Visualization of the working steps for the maintainer.

  • Installation of the robots: Visualization of the correct position for the robots and all the other process equipment in the robot cell.

  • Visualization of the current positions of the different coordinate systems.

Is Augmented Reality in terms of industrial robots useful?

In the end of the first part of the survey the participants were asked if they think it is helpful to use AR in terms of industrial robots. 75% of them answered that the idea of the use of AR in terms of industrial robots is good. Only seven percent think it is insufficient (Fig. 7).

Fig. 7.
figure 7

Rating of the question “How useful is AR in terms of industrial robots?”

Full size image

Taken all together an Augmented Reality extension is rated by the participants very positive. The challenge is now to develop and implement sensible and usable AR Human-Machine Interfaces that are suitable for industrial usage.

3.2 Virtual Reality for Industrial Use

In the second part of the survey focused the opinion of the participants about a few exemplary situations regarding the reasonable use of VR. This part is not just focused on the field of robotics.

Visualization of large production facilities (Engineering phase).

One question for the participants was if it is useful to visualize large production facilities to make it possible to work through them. This would give for example customers a better description of the product. It could be possible to show several procedures within the production facilities. Another usage could be during the virtual commissioning in the digital factory approach. The majority of the participants thinks it is a good idea to visualize large production facilities. But 14% of them think it is a bad idea (Fig. 8).

Fig. 8.
figure 8

Rating of the visualization of large production facilities

Full size image

Visualization of workstations.

The next question for the participants was if it makes sense to visualize workstations to train new workers before they work for example at a production line in the automobile industry. Another aspect to visualize workstations is for their better planning during the engineering phase. 79% of the participants thinks it is useful to visualize workstations before they are build or to train workers for their forthcoming tasks (Fig. 9).

Fig. 9.
figure 9

Rating of the visualization of workstations

Full size image

Which further situations and data can be visualized with Virtual Reality systems?

The participants were requested to suggest further situations and data that can be visualized with VR. The following enumeration shows the answers of the participants:

  • 3D construction

  • Safety commissioning (safety areas, interfering contours)

  • To test control concept and accessibility of production facilities in their original size

  • To teach complex manufacturing processes

  • Support in equipping, service and diagnose of machines

  • Remote maintenance, optimization

  • To train the employees

What risks are associated with Augmented Reality and Virtual Reality devices?

The participants were asked if they could imagine risks of using VR and AR in the industry. The following risks are named:

  • security concerns in terms of industrial espionage.

  • risk of eye injury.

  • limitation of the visual field.

  • In case of failure of the VR or AR glasses the working process cannot be continued because of lack of abilities and knowhow.

  • Directs and leads to carelessness.

Some of the mentioned concerns are still in actual the discussion about Industry 4.0 – topics (e.g. industrial espionage/it security). Some concerns may result from the lack of knowledge about the devices (e.g. risk of eye injury). A further analysis could be useful for the interpretation of the mentioned risks of AR and VR. AR/VR application should avoid these concerns.

4 Actual and Future Activities

In a first step a prototype for the use of data glasses for the communication with a PLC is developed and evaluated [4]. Necessary and situation-related information can be displayed in the normal visual field of the user. For the user inputs a specific touchpad is available (Fig. 10).

Fig. 10.
figure 10

Augmented reality via data glasses for interaction with a PLC

Full size image

Main purpose of this first step is to realize a stable data communication between the data glass and the PLC. Beside this the developed application include a menu to navigate to the single PLC modules. Via the touchpad of the data glass the user can monitor the process variables (also the historical run of the variables is presented) and can change the binary process values.

The next planned step will be the transfer to VR application. Applying an HTC Vive an application for the virtual commissioning of a simple production system (Fig. 11) will be developed at the PLC Laboratory of Heilbronn University. The PLCs of this production system are connected via OPC UA to a cloud server. The HTC Vive system exchange the necessary data via this cloud server.

Fig. 11.
figure 11

Automated production system with different types of PLCs

Full size image

Based on these first experiences the transfer to the fields of robotics is planned for further research steps. One project will focus the use of VR application based on a HTC Vive for the stationary robotics, a second project is planned with AR for mobile robotics.