Keywords

1 Introduction

In the recent years, composite laminates have become the preferred materials for building large components in transportation industry because of their properties of lightweight, fatigue and corrosion resistance, capability to mold large complex shapes, high stability in space environment and so on. The possibility to have internal defects as a consequence of anomalies during the production process imposes rigid controls to the final structures by Non-Destructive Tests and Evaluations. Recently, in order to detect internal defects many signal processing algorithms [1,2,3,4,5] have been applied to temperature signals obtained by both lock-in (LT) and pulsed thermography (PT), and to ultrasound signals. However, the costs of repair processes can be excessive and in some cases the repair itself can be not applicable and lead to the rejection of the whole structure. For this reason the introduction of a visual system able to monitor the production process and support human operators in their checking tasks can greatly reduce the risks of internal defects, and increase the quality standard of the final components.

The reconstruction of three-dimensional surfaces of observed objects has been already used to highlight defects. In [6] a phase measuring profilometry has been applied to reconstruct the surface shape of wheels and detect wheel abrasions. An on-line quality detection system has been proposed in [7] to detect thick defect blocks found on the surface of rectangular steels observed by smart sensors based on red laser optics technologies. Fringe projection profilometry is one of the methods used to reconstruct surfaces, models, and inspect the integrity of structures. In [8] a digital fringe projection technique along with a look up table based gamma correction method has been used to detect surface damages such as dents, cracks and corrosion of aircraft structures. A photometric stereo 3D measurement system in [9] scans steel plate/strip and identify and locate 3D defects such as roll marks, cracks, and indentations. The authors of [10] propose the use of a combination of laser slice panoramic images and texture panoramic images to analyze simultaneously texture information and surface deformation and solve the problem of the pipeline defect detection. A triangulation-based laser scanner is used in [11,12,13] to extract a three-dimensional model of drilling tools and allow the fast detection and characterization of surface defects. The system, by using an innovative acquisition procedure, overcomes the occlusion problems due to the presence of grooves on the object surface and provides at the same time high precisions and fast measurements.

In this paper we propose the use of a non invasive and accurate experimental setup: a combination of camera and a laser light that models the 3D surface as the appearance of the laser spot changes according to the distance between the light source and the object surface. Quality controls are performed during the production of composite materials. After the stratification of every layer of the stratification, point clouds are acquired and thus used to build the 3D model of the surface. Then, by using a pipeline of signal processing methodologies, the differences between the expected model and the actual one are processed to detect the anomalies that could be occurred during the stratification, i.e. the presence of gap out of ranges or overlap between adjacent plies. The main novelty of the proposed methodology lies in the extraction of the reference model, which performed directly from the acquired surfaces and, consequently, is independent of the shape (e.g. curvature) of the laminate to be tested. The paper is organized as follows: Sect. 2 describes the experimental setup for laser triangulation and the proposed pipeline for signal processing; Sect. 3 reports the experimental results and the relative comments; final remarks and conclusions are in Sect. 4.

2 The Proposed Approach

The detection of anomalies during the stratification process of composite materials can be early done by analyzing, after each layer, the position of stripes and measuring the relative orientations and distances between adjacent ones. In Fig. 1 the experimental setup devised to solve this problem is shown. It is a laser profilometer placed on a motorized and encoded linear axis which scans all the production area (\(800 \times 150\) mm) and generates a point cloud of the surface. The camera is the Dalsa Falcon 4m60, which is able to produce frames having resolution of \(2352 \times 1728\) pixels at a frame rate (fullframe) of 60 fps. The camera-lens set is able to produce an out-of-plane resolution of the final 3D model, i.e. the minimum detectable alteration of laminate thickness, is equal to 0.07 mm. On the other hand, the in-plane spatial resolution, which is only ascribable to the resolution of the movement of the linear axis, is equal to 0.1 mm at a scan speed of 8 mm/s. Under these conditions, the whole scan of the production area is performed in 100 s.

Fig. 1.
figure 1

(a) Laser profilometer for 3D modeling of the plies of composite taper; (b) example of a layer of composite to be tested. The arrows indicate the direction of the laser scan.

Full size image

The production process of composite laminates is subjected to strict geometrical rules. Specifically, the positioning of each layer has to be done by respecting both some orientation constraints and distance values between adjacent plies (no overlaps and gaps in a fixed range). For instance, in the challenging field of aeronautics, the geometrical tolerance for gaps between adjacent plies is equal to 2.5 mm, whereas overlaps are always forbidden. In Fig. 2 examples of the 3D point clouds resulting from the proposed inspection setup are reported. In Fig. 2(a) an “in tolerance gap” is highlighted on the left and an “out of tolerance gap” on the right. In Fig. 2(b) an overlap between adjacent plies is reported.

Fig. 2.
figure 2

3D point clouds resulting from the proposed inspection. (a) Examples of in- and out-of-tolerance gaps and (b) overlap between adjacent plies.

Full size image

As stated in the previous lines, the aim of this paper is the definition of a complete methodology for the in-line detection and characterization of this kind of anomalies. In Fig. 3 the whole processing pipeline is described. The input disparity image, representing the variations of the laser line with respect a fixed reference system, is processed in three steps:

  • a model of the expected surface, obtained from the acquired data, is generated and the differences between the model and the input disparity are used to generate a corrected disparity map (the idea is quite similar to an unsharp masking procedure, ie. fine details are highlighted and noise is reduced);

  • a binarization via image statistics is performed;

  • a dedicated morphological processing is applied to filter binary images according to the domain, in order to generate images containing only significant information.

Final disparity images are then converted in 3D models in world coordinates through the application of the results of a preliminary calibration phase. Edge analysis on 3D models is thus able to give quantitative measures of the orientation and of the distance parameters of the inspected plies.

Fig. 3.
figure 3

Representation of the processing pipeline. Image processing (blue blocks) is performed on input disparity, whereas the final geometric characterization is completed in actual world coordinates. (Color figure online)

Full size image
Fig. 4.
figure 4

Results of disparity correction by model comparison: (a) input disparity, (b) reference model due to median filtering and (c) final corrected disparity

Full size image

The input disparity image (see Fig. 4(a)) contains many information (the darker are the higher region, the brighter are the deeper). First, the overall surface appearance has some dark regions corresponding to not planar areas, which are due, for example, to the presence of air under the plies. In addition, the signs of previous stratifications generates further textures to the surface appearance (see the horizontal directions of the plies). Finally, the separations between adjacent plies during the last stratification produce the alteration of interest for the current analysis (see the extended oblique dark stripes of Fig. 4(a)). In order to extract only the valuable information for the overlap/gap detection, the following steps are carried out: after an initial smoothing, each point of the disparity image is treated by a median filter with an extended kernel, whose in-plane size, projected to the surface, is higher than \(20 \times 20\) mm. In this way, a surface model, taking into account the global shape of the laminate, can be defined from the acquisition itself. The resulting reference model is shown in Fig. 4(b). Then a corrected disparity map is thus obtained by a differential analysis between the reference model and the initial disparity map (Fig. 4(c)). It is worth noticing that this processing is mandatory to correct the input disparity to rectify the scan plane to the shape of the laminate. In this way, relative alterations of depth detected by the laser profilometer due to the intrinsic shape of the laminate under testing, or to the misalignement of the scan plane with respect to the laminate surface, are automatically compensated and do not produce false positives in the analysis of 3D model.

The histogram of the resulting corrected disparity is thus centered on a specific value, directly linked to the distance between the laser profilomenter and the specimen surface. Its shape resembles on a Gaussian-shaped function, whose mean and standard deviation values, \((\mu ,\sigma )\), can be estimated by the application of a curve fitting in the least square sense. All disparity values which fall in the range \((\mu - \sigma , \mu + \sigma )\) correspond to surface points, whose depth is within a range defined by \(\sigma \), which takes into account for surface roughness and measurement uncertainty. The remaining disparity values under or above this range are informative, since they represent dips and peaks over the laminate. Gaps and overlaps belong to these two regions, respectively. The binarized images reported in Fig. 5(a) and (b) represents the low disparity regions (under the threshold \((\mu - \sigma )\)) and the high disparity regions (above the \((\mu - \sigma )\)).

The morphological filtering applies the domain knowledge: the areas between adjacent plies produce long connected regions crossing the image from one side to another. Connected regions which do not fulfill this simple hypothesis are rejected from the analysis. It is applied to both low disparity (LD) and high disparity (HD) images as the ones in Fig. 5(a) and (b) to detect gaps or overlaps, respectively. In Fig. 5(c) the results of the morphological filtering of the binarized HD image is reported. On the contrary, the results of morphological analysis on LD image is not shown as it has not produces any significant result, sing gaps are not present on the inspected surface.

Although this image contains a clear evidence of the presence of overlaps, it is not ready to allow the estimation of the values of the overlap (or gap) parameters. A transformation of the disparity values in world coordinates is applied by using the calibration parameters of the profilometry setup. Each point (i, j, d) is transformed in a world reference system having coordinates (x, y, z). At this point, with simple geometrical consideration the ply orientation, the gap measures, and the presence of overlaps can be estimated.

Fig. 5.
figure 5

Binarization of corrected disparity, according to image statistics. Binary images of (a) high and (b) low disparity values (HD and LD, respectively), and (c) results of morphological filtering on the HD binary image. Morphological filtering on the LD binary image produces no results, since there are no gaps on the laminate surface.

Full size image

3 Experimental Results

The proposed setup and processing approach have been tested during a real experiment of stratification of a composite material carried out by a human operator. During this production process 8 layers were placed on the working area. The plies were positioned under different orientations and present some anomalies voluntarily produced at specific layers in order to test the reliability of the proposed detection systems. These anomalies were certified by the human expert operators, who annotated their sizes and orientations. After each stratification the laser profilometer scanned the surface and all the 3D point clouds were processed to highlight anomalies.

Fig. 6.
figure 6

Input disparity images of three layers having (a) two overlaps, (b) an in-tolerance gap and (c) two out-of-tolerance gaps between adjacent plies. (d), (e) and (f) show the corresponding outcomes of the geometrical characterization, including the computation of edge orientations.

Full size image

In Fig. 6 the results obtained by application of the proposed approach to the inspection of defective laminate surfaces having overlaps and out-of-tolerance gaps are reported. In Fig. 6(a) an input disparity map containing two overlaps is shown; in Figs. 6(b) and (c) the disparity maps of in-tolerance gaps (spacing below 2.5 mm) and out-of-tolerance gaps are respectively displayed. The corresponding results of the geometric characterization in terms of edge orientation are displayed in Fig. 6(d), (e) and (f). These values are in good agreement with those measured by the human operator who performed the manual stratification, i.e. \(-45^\circ \), \(90^\circ \) and \(45^\circ \) for Fig. 6(a), (b) and (c), respectively.

Table 1. Measures of estimated gaps and overlaps of Fig. 6.
Full size table

In Table 1, in fact, all these measures are listed and compared. In particular, the first column reports the reference to the corresponding figure. Then the anomaly type is described in the second column, whereas the expected measurements, as evaluated by the human operator, are reported in the third column. The estimated measurements of both min and max values of the thickness of the detected overlaps/gaps, and the final decision of acceptance are reported in the last three columns.

As can be observed, all the measurements were correctly estimated, with improved resolution with respect to the one given by the human operatos. As a consequence, the real experiments confirm the accuracy and the effectiveness of the proposed system which provide the operators a twofold contribution: by one hand the system verifies the correct positioning of the plies in terms of orientations; by the other hand it detects the anomalies which can occur during the stratification phase alerting promptly the human operators in case of not allowed measurements.

4 Conclusion

The routine inspections during production processes of composite materials relies on the human ability of skilled inspectors who visually check the production line and detect the presence of anomalies. However, this is a laborious task and sometimes maybe ineffective, or too subjective. This raises the need for a system to automate the process in order to make it faster, efficient and reliable.

In this paper we propose a laser profilometer able to scan with high resolution the surfaces of composite materials as they are stratified, and to produce corresponding disparity images. Then, an image analysis pipeline is applied to highlight targets of interest, such as gaps and overlaps between adjacent plies. It is obtained by constructing the expected model of the surface, which becomes the reference for a differential analysis. Finally, the dimensions of gaps and overlaps occurred during each stratification are determined in world coordinates, in order to state a final decision of acceptance. Experimental results obtained during a real stratification session demonstrate the reliability of the proposed system to support human inspectors in their tasks of quality control during production processes.

Future work will be addressed to test the system and the processing methodology on more complex surfaces and on different anomalies that could be found in industrial contexts.