Keywords

1 Introduction

Visual inspection is an important process in an industry to recognize defective parts, to assure quality conformity of a product and fulfill customer demands [1, 2]. In assembly and manufacturing activities, product and process inspection are usually performed by human inspectors, but due fatigue, small parts, small details, hazardous inspection conditions and process complexity this task may not achieve the desired quality or be almost impossible to detect some types product non-conformities. In this cases a machine vision solution is recommended [1, 3, 4].

Machine Vision System (MVS) consist in applying computer vision to industrial solutions [8]. MVS can be used to perform visual inspection and fulfill industrial and factory performance, consequently improving product quality outcomes. To meet industrial expectations, MVS has been used to reduce product quality problems through improved inspections. Inspection system must be adapted to a scenario which have a wide variety of product features and high production speed assembly lines with complex environment variables from the MVS perspective [5, 6].

The purpose of this article is to provide an overview of MVS concepts and theirs current status for industrial inspection, identifying the main Artificial Intelligence (AI) concepts and application to quality in the automotive industry.

2 Theoretical Foundation

2.1 Machine Vision with Artificial Intelligence Techniques Machine Vision System Concepts and Applied Technologies

MVS systems have become imperative in many modern manufacturing facilities as forms of automatic quality inspection. These systems are integrated with manufacturing process where all products must pass through. Most systems consist of a camera (or cameras), PC and usually a controlled lighting environment within an enclosure [7].

2.2 Machine Vision with Artificial Intelligence Techniques

Machine learning and deep learning are data-driven artificial intelligence techniques, which may be applied to MVS. Both techniques uses neural network architecture concepts. which transforms raw data into representative information for decision making. Machine learning can be applied to feature extraction and classification where each step is constructed separately, and it may be highly dependent of an expert knowledge. In the other hand deep learning is applied in both feature extraction and classification steps as an unified neural network solution with requires minimum human interference. Figure 1 shows a comparison between machine learning and deep learning [8].

Fig. 1.
figure 1

Comparison between machine learning and deep learning. Adapted from [8].

Full size image

One major drawback of MVS system without artificial intelligence techniques is that it cannot learn with all the processed images. Every image detail or new fault detection that may occur after the MVS initial setup may not be detected, which can lead to an incorrect information output while MVS with learning features has the potential to learn with new images incomes.

Golbani and Asadpour [4] proposed a block diagram for a typical vision system operation when artificial intelligence techniques were still being developed for MVS, as shown in Fig. 2. A new diagram was built containing the original block together with new AI techniques identified in recent literature. In this new framework is important to emphasize the need of an image knowledge database, which contains object features and quality criteria definitions, which may be used to assist and improve the current learning methods [1, 8].

Fig. 2.
figure 2

A block diagram for a typical vision system operation and applicable AI techniques. Adapted from [1, 4, 8].

Full size image

2.3 1-D and 2-D MVS Industrial Applications

One (1-D) and two dimensional (2-D) MVS have a wide range of applications such as measurement, surface and depth inspection, thermal inspection and robot vision. Each kind of application has its own characteristic equipment with different image gathering source, such as photoelectric sensor, lasers, cameras and so on. Some MVS source types and applications are shown in Fig. 3.

Fig. 3.
figure 3

Adapted from [1, 9].

Conventional MVS types and detection modes.

Full size image

Camera based MVS for industrial applications are commonly used to verify presence or absence of components, verify if the components are in their correct position and orientation, verify if components has the desired colors, analyze and recognize image content such as code bars and inspect size and measure of parts and assembly components [4].

Photoelectric and laser-based MVS inspections can also be used to presence or absence of components, check component positioning and verify desired colors, mainly to measure parts.

2.4 3-D MVS Industrial Applications

Optical non-contact 3-D measurement technique has been used to measure an image of an object and extract its geometrical information. It can be divided in passive and active 3-D sensing systems, where passive works with natural lighting from the scene without controlling the light that goes to the inspected object. While active sensing systems uses an external light, such as laser or a known projected light, by measuring speed of light, laser coherence or applying triangulation techniques [10].

Structured light is an active 3-D sensing system, which illuminates the object with predefined patterns and analyses how these patterns are deformed by the object when observed from a different angle of the projection. Some systems adopt non-visible structured light to avoid interfering with other computer vision [11].

Stereo vision profilometry techniques simulates human vision through two camera setups angled with each other, which aims to identifying and match common features of an object images from multiple allowing it to be reconstructed through triangulation techniques [11]. Stereo vision normally is a passive 3-D sensing system but there are new camera setups which also uses a projected structured light in the object that turns them into active stereo vision [7].

Another active vision techniques are through time of flight light measurement. This technique uses light pulses with a known camera range so the time for the emitted light to travel from the camera and hits the object and it reflected to the camera is measured, based a fixed and known light speed the distance can be calculated [7].

Light coding imaging is also an active 3-D sensing system but instead of using light pulses it keeps light source constantly turned on. It also uses an infrared spectrum emitter and receiver, which analyses lens distortion, the emitted light patter and the distance between object, emitter and receiver and the deformation of the light over the inspected object [7].

2.5 MVS Industrial Evaluation

Pérez et al. [7] compared several 3-D machine vision techniques applied to industrial environments emphasizing which factors need to be considered in order to select the most adequate vision, considering, system accuracy, working distance, image output, system advantages and limitations. An overview of this evaluation is shown in Fig. 4 along with the current status of machine vision system in the automotive industry.

Fig. 4.
figure 4

Adapted from [7].

MVS system assessment.

Full size image

Environment light influence is one of the major problem to MVS, only a few systems are not subject to external light conditions. Another constraint that may affect your MVS selection is the necessity of both the camera and the object to remain static which, in some cases, are an exception in the industry. Accuracy and working distances are variables that must be taken in account, because depending on the precision needed the MVS solutions options narrows down.

2.6 MVS Applications to Industry 4.0

The fourth industrial revolution or Industry 4.0 aims to develop intelligent factories with upgraded manufacturing technologies through new features such as cyber-physical systems (CPSs), the Internet of Things (IoT), Big Data and cloud computing. New manufacturing systems propose simultaneous monitoring of physical processes with being controlled by digital technologies, being able to make smart decision through real-time communication and interaction between humans, machines, or any smart device [12].

Figure 5 contains a simplified 4.0 Industry diagram, adapted from more complex diagrams available in the literature. Machine vision is located in an IoT layer. Its function is to provide image data through a connected network to a big data cloud server. This data will be subject to mining and cleaning procedures, removing unnecessary information. This information can be used as an input for machine learning techniques, allowing an integrated system to detect and describe what happened to the product, determining why that happened, predict what may happen and prescribe which actions must be taken.

Fig. 5.
figure 5

Adapted from [8, 13].

MVS integrated with a simplified 4.0 Industry diagram.

Full size image

MVS future solutions, such as knowledge-driven decision-making, real time control, online advanced analytics and artificial intelligence in CPS, are considered as challenging implementation process. It may take from 3 to 10 years for industries to achieve a concrete degree of maturity and obtain a fully operational system with these functionalities [14].

3 Methodological Approach

In order obtain a detailed comprehension of the theme and to direct future studies, a systematic review is proposed. Based on the keywords identified in the theoretical foundation, they were grouped in in three fields of study: Industrial Quality Control (IQC), AI and Machine Vision (MV). Papers available through Scopus database were chosen, covering the period between 2007 to 2017.

Figure 6 shows each keyword used for each group. Keywords combinations from same group were not used in this paper, in order to increase the likelihood of significant search results.

Fig. 6.
figure 6

Proposed field groups with each composing keyword.

Full size image

The primary search results totalized 60.176 articles but they contained duplicate values. Before removing duplicate values, results which belonged of at least more than one search group were labelled as ‘AI + MV + IQC’. After that duplicates were removed, 42.858 articles remained.

Figure 7 shows these non duplicated publications distributed over the analysed years. The graphic results indicates an increased number of publications for each group. Artificial intelligence and machine vision keywords combination presented the most relevant increase, mainly after 2012. It can be observed that vision systems applied to industrial quality through AI technologies tend to be revelant in the following years.

Fig. 7.
figure 7

Publications in each field of application over the years.

Full size image

We filtered our search results to article labelled by SCOPUS as Journals, reducing to 16760 articles. Through ISSN information of each journal and the database extracted from SCIMAGO, a link between database could be established with the remaining search results. With SCIMAGO information another filter was applyied, selecting journals that has been evaluated by SCIMAGO, totalizing 8549 articles.

A content analysis with detailed keyword identification was made using the remaing articles. Titles and abstracts with keywords related to medicine, human features identification and non industrial applications were used to create an exclusion filter, because they were not related to this paper, resulting in 3350 articles.

New relevant keyword were also identified and included in initial keywords used to create the first database. With the updated groups, a filter with articles considering at least one keyword of each group was made and 289 results were found.

Table 1 contains the keywords used in the last filter and Table 2 summarizes all filtering steps database results.

Table 1. Search results for machine vision applications and trends.
Full size table
Table 2. Search results for machine vision applications and AI and IQC trends.
Full size table

3.1 Initial Results

During abstract reading step, solutions which integrates AI and IQC applied to MVS were found. One proposed solution used genetic programming with machine learning to develop and modify preprocessing programs. It has the ability to adapt to new production parameters and light conditions changes with an automated preprocessing framework. This solution has a potential to solve common MVS problems [15].

One common aspect of most of the articles is that they only provide a solutions according to a specific type of industry, such as welding systems, PCB fault detection, and sometimes in does not have the potential to be applyied to other scenarios. One important factor detected in the articles is the absence of a common key performance indicators (KPI) in order to evaluate how accurate is each MVS solution. Another point identified is the absence of a method for selecting the best AI solution framework for each kind of solution. Figure 8 shows that from 289 results, only a few articles provides a framework or method as an important part of the abstract.

Fig. 8.
figure 8

Publications in each field of application over the years.

Full size image

4 Conclusion

MVS solutions applied to quality inspection for the industry can be improved thorugh existing and new technologies such as AI, increase inspection detection processing speed and the capacity detect new types of defects.

There are many project restrictions and technical constraints in MVS implementation, which must be consider during selection, implementation and validation steps for industrial solutions, otherwise the system may not perform accordingly.

Despite of existing MVS solutions with AI applied to IQC, they tend to be exclusive for each scenario and they lack of a common KPI to evaluate its performance or the ability the replicated in other industrial scenarios. A method to select the most appropriate type of AI technique and how to integrate with existing or new MVS systems it is still lacking.

5 Future Work

A detailed review initial of the current research is necessary, in order to indentify which are main adopted AI frameworks and techniques to MVS and IQC, which are the KPI’s to evaluate MVS performance, how much MVS with AI are improving industrial KPIs and which are the main researches.