A knowledge based infrared camera system for invisible gas detection utilizing image processing techniques
- 65 Downloads
An infrared camera system for real-time monitoring of invisible methane gas leakages is proposed, which can be used in production, transportation, and gas plants. Instead of using conventional lasers, a medium wavelength light-emitting diode (LED) with a center wavelength of approximately 3300 nm is used for the light source of the gas detector. We achieve compact cooling with low power consumption by using a small electronic part with a Peltier element instead of a larger cooling unit. We also propose an algorithm based on adaptive average method and adaptive histogram equalization process to extract the background efficiently. This knowledge based infrared camera system effectively detects the leakage of invisible gas. Preliminary experimental results confirm the potential and effectiveness of the developed equipment to be commercialized for use in real-time gas monitoring systems. We aim to eventually develop an image analysis system that combines image processing and knowledge engineering techniques.
KeywordsInfrared camera Gas detection Adaptive background extraction Plateau equalization Contrast enhancement
In this paper, we propose an infrared camera system that analyzes image sequence from a camera to detect invisible leakage of gas. Methane gas, the main ingredient of natural gas, is combustible and explosive, so safety management is essential for chemical installations and gas facilities. Gas leakage cannot be visually detected but can be verified by other means (Vollmer and Möllmann 2018), such as detection with soapy water, frame-ionization detection, and laser detection. By conventional methods, rapid response to leakage and accurate location of leaks is not possible. However, the proposed infrared camera equipment makes gas detection possible by means of digital image processing. One can monitor and prevent leakage of gas by using a camera instead of monitoring it directly at the spot.
For decades, infrared cameras have revolutionized the maintenance of many industries, and have proved to be the best technology to detect flaws before and after electrical and mechanical faults arise. Infrared cameras can play an important role in helping to reduce environmental impact, as well as improving workers’ safety and the quality of processes. They also can be used successfully to detect environmentally hazardous gas leakages. Not only does gas leakage adversely affect the environment, but also there is considerable economic loss associated with leakages. In light of this, we have developed a gas detection camera that can detect many kinds of gases based on particular center wavelengths, including methane gas. The developed product can be relatively small and inexpensive by combining the advantages of laser and infrared sensors. It is also highly likely to be utilized in the field because of increased portability and reduced failure rates. The developed camera is a revolutionary infrared camera capable of detecting methane gas in all areas of production, transportation, and distribution of oil and natural gas. This camera can scan a wide range of areas and visualize potential gas leaks in real time, allowing multiple components to be checked with one scan. Instead of using the laser of conventional methane gas detectors, a medium wavelength LED with a center wavelength of approximately 3300 nm is used for the light source of the gas detector. Furthermore, a compact cooling unit with low power consumption can also be achieved by applying a small electronic part using a Peltier element instead of a larger cooling unit. Currently, a popular gas detection device with a camera image is produced by FLIR company, but it is expensive and cooling gas needs to be replenished. In response we have developed a durable low-cost gas detection imaging device. A comparison of preliminary tests under various conditions proves that the proposed camera system can afford fast and efficient gas detection.
There have been many studies on how to obtain clear infrared images from sensors. Zuo et al. presented a technique based on separating the acquired image into base and detail parts using a bilateral filter, then processing each part independently (Zuo et al. 2011). The technique effectively maps the raw acquired infrared image to an 8-bit domain based on a bilateral filter and dynamic range partitioning method. It has prospects for application, but may not work for some particular conditions. Peng et al. suggested detail enhancement for the infrared image (Peng et al. 2016). The raw infrared image is decomposed into the base and detail layers by using a propagated filter. This produces results similar to that of the preceding theory. Yoo et al. proposed a novel digital binning algorithm of image enhancement for low-light scenes in an environment with insufficient illusion (Yoo et al. 2015). It can easily be built into an existing image signal processor since the proposed algorithm does not use an iterative computation. However, its computational complexity depends on the sorting algorithm. Here, we propose a log function to achieve 8-bit infrared images similar to those of the bilateral filter. Recently, a logarithmic histogram modification has been proposed to improve image contrast with natural conservation (Bhandari 2019). The proposed method can uniformly enhance image contrast with fewer parameters without losing the basic features. In addition, safety-related services based on a single depth camera installed in the user’s home were designed and evaluated for safety management using a camera (Mettel et al. 2019). The goal of ARKTOS project is to build an intelligent knowledge-based system to classify satellite sea ice images (Soh and Tsatsoulis 2002). We aim to eventually develop an image analysis system that combines such image processing and knowledge engineering techniques.
This paper is presented in the following order: Sect. 2 describes the overall developed camera system. Section 3 explains the non-uniformity correction with automatic gain control. Section 4 presents the algorithm of the contrast limited adaptive histogram equalization. Section 5 presents the adaptive algorithm for background extraction, and compares it with other methods. Section 6 explains comparative experimental results. Finally, Sect. 7 presents the conclusions of this research.
2 The infrared camera system
Input images are poor in quality. Therefore, it is almost impossible to identify gas leakage from an unenhanced image. We apply the contrast limited adaptive histogram equalization (CLAHE) algorithm to improve the quality of the image (Zuiderveld 1994). We then apply an adaptive background extraction method to extract the background from the image. Because the camera for gas detection is stationary, we use the adaptive average method to detect background images. Once the background of the image is detected, the gas image can be extracted by subtracting the background from the input image. Because of the nature of gas images, the resolution of the output images is typically poor, thus requiring additional image processing to enhance the quality of the image. The following sections provide detailed descriptions of the algorithms mentioned here.
3 Non-uniformity correction
We perform a uniformity test to ensure that all elements within the sensor produce the same results. A problem associated with this test is that there is no standardized test for performing uniformity, which results in a difference in reported values. Here we deal with two common ways of specifying uniformity: 2-point non-uniformity calibration (NUC) and 1-point offset calibration. The offset uniformity is calculated by capturing a single image and determining the values reported for all columns and rows within the sensor or sensor area. It is entirely based on measurements of all pixels in the sensor or region. The response uniformity method uses images taken from two different temperatures to analyze the reactions of all pixels, and then calculates the temperature changes for all pixels in the sensor (Ratliff et al. 2002). The 2-point non-uniformity calibration, also known as a calibration-based correction, uses a correction table computed by all pixels in order to compensate for the non-uniformity between pixels. This method is commonly referred to as 2-point NUC because it uses tables calculated from two different temperature points (hence 2-point) of a blackbody source, measured at different temperature points under optimal working conditions. If a thermal image is invoked and a correction table is applied, the compensation table becomes calibrated. When the pixels differ between different sensitivity levels and are offset by different sensitivity factors, they are calibrated based on a blackbody source and then the same output is exported under the same conditions (Tendero et al. 2010).
The concept of a 1-point offset correction, also known as a scene-based correction, is not significantly different from a 2-point NUC for calibration purposes. Because only one absolute reference value is used, we leave the gain value alone and adjust the offset values only. Scene-based correction techniques are optimized methods to adjust images effectively using numerical analysis algorithms instead of using a blackbody or heat source under optimal conditions. This technique is applied to cases where thermal images are stored continuously or when movement is traced in the event of thermal images. In this case, the scene-based correction technique applies the reference values for various temperature values that each detector can display, and then they are normalized so as to be expressed as optimal thermal data in the thermal image.
The detector data is streamed directly from the sensor to the 14-bit values for each pixel in the array. Almost all commercially available devices display analog images using an 8-bit value. In other words, the video is displayed in a 256 scale rather than a full 16,384 resolution of the sensor. Therefore, data must be imported in a format that allows data to be displayed. Vickers used the Plateau histogram equalization for these transformations (Vickers 1996). The Plateau histogram equalization algorithm maximizes the dynamic range available for the scene content. It uses a transfer function for conversion based on the number of pixels in each bin and allocates more 8-bit ranges for that bin. When the plateau value is small, the automatic gain control approaches linear algorithms that maintain linear mapping between 12 bits and 8 bits of data. The objective of the automatic algorithm is to make each of the 256 levels identical to the same number of pixels, which provide the best contrast for the given scene. The higher the plateau values, the better the algorithm can redistribute data to achieve this goal. Therefore, it is clear that the histogram does not occur in areas where there are no contents in the scene, and that the peak is much smoother and the data is much more uniformly distributed than the original data. Note that higher plateau value distorts relationships between the physical temperature of the scene and the gray levels of the image, which are well preserved in a linear case (Raju et al. 2013). The gain and level values have been adjusted properly in order to point out special low contrast gas.
4 Contrast limited adaptive histogram equalization
To create an image that is bright and decodable, the gray level needs to be distributed properly. The most common way to resolve this kind of problem is histogram equalization (Raju et al. 2013). The common histogram equalization uses the same transformation to convert all pixels in an image. This method is effective if the distribution of pixel values across an image is similar. However, for images that contain significantly brighter or darker areas than their surroundings, the contrast between these areas does not improve sufficiently. Adaptive histogram equalization (AHE) solves these problems by converting each pixel into a transform function derived from a nearby area (Pizer et al. 1987). This is one of the useful image processing techniques used to improve the local contrast of images rather than the entire image. It differs from ordinary histogram equalization in terms of how the adaptation method calculates multiple histograms. Each of the multiple histograms corresponds to a specific portion of the image and reconfigures the brightness of the image. This makes it suitable for improving local contrast and clarifying edges in each local area of the image. However, adaptive histogram equalization method tends to amplify noise in relatively homogeneous areas of images. Contrast limited adaptive histogram equalization (CLAHE) method, which is a variant of adaptive histogram equalization, improves the contrast enhancement performance by inhibiting amplification according to certain conditions (Zuiderveld 1994). It divides the image into small areas (known as tiling), and applies local histogram equalization to each tile to achieve uniformity across the entire image.
To prevent the excess amplification of noise, the contrast limiting procedure must be applied to areas where each conversion function has been created. The contrast amplification near the given pixel values is given by the slope of the cumulative distribution function (CDF). CLAHE limits the histogram by clipping the histogram (the so called the clip-limit) to a predefined value before calculating the CDF. As a result, the slope of the CDF is limited and therefore the conversion function is limited. The numerical value of the clip-limit varies according to the normalization of the histogram and therefore depends on the size of the surrounding area. The sizes of the neighborhood are properly selected to ensure that the background is suitably extracted. One must adjust the tile size and function parameters to maximize contrast and structural visibility whenever images are acquired. Adjacent tiles are merged using bilinear interpolation to eliminate artificially induced boundaries (Hong et al. 2017).
5 Adaptive background extraction
We have compared methods from simple techniques such as the conventional average method, to more complex probabilistic modeling techniques such as mixture of Gaussians (MoG) (Piccardi 2004; Sobral and Vacavant 2014). The MoG is one of the most popular recursive modelling techniques if there is no prior information about the circumstances (Bouwmans et al. 2008; Stauffer and Grimson 2000). The average method is one of the most frequently used background modeling techniques (Cucchiara et al. 2003). Multiple images can be obtained from the same location if the camera is at the same point several times. At this point, the simplest approach to finding a background image is basically to take the average of all images. The background estimates are defined as average values in each pixel position of every frame in the buffer. The supposition is that the pixel remains in the background in more than half of the frames in the buffer. The means of obtaining background values as a result of the average or median are simple and fast, but they require a lot of memory. Increasing the number of frames increases the amount of memory required.
6 Experiments and discussions
In this paper, we proposed an infrared camera system that analyzes image sequences to detect invisible gas leaks. To increase the mobility of infrared cameras, instead of using laser as in conventional methane gas detectors, a medium wavelength LED with a center wavelength of approximately 3300 nm is used as the light source of the gas detector. In addition, a compact cooling unit with low power consumption is made possible by applying a small electronic part using a Peltier element instead of a larger cooling unit. Some cooling gas needs to be replenished, which is a very expensive and cumbersome process. Rather than using a complex plateau algorithm, we have proposed a log-based automatic gain control process after the histogram equalization to improve the performance of images. The developed prototype has not yet reached a satisfactory level of size and weight. However, progress in underway to make it more compact and lighter, making it easier to carry. Instead of monitoring directly on-site, the camera can monitor gas leakage extensively from a distance.
The leakage of invisible gas can be detected by the proposed infrared camera system. The results show that the proposed system is a simple but efficient system for monitoring gas leaks. To overcome the complicated computational process of the mixture of Gaussians (MoG) method, we have presented an algorithm based on adaptive average method to efficiently process and to detect the background in real time. Comparison of prequalified testing using various methods demonstrates that the proposed system can provide rapid and efficient background extraction. Such simplified algorithms appear to exhibit better performance with lower memory requirements compared to established techniques. Preliminary experimental results confirm the potential and effectiveness of the developed equipment to be commercialized for use in real-time gas monitoring systems. Further research is expected to lead to more precise background extraction processes and a lightweight camera for portable gas detection devices.
The problem of expressing knowledge within a computer, or making deductions with a computer using knowledge, is a problem in the field of knowledge engineering. In the branch of computer vision, deep learning techniques are becoming very important and highly developed tools for understanding images, and they outperform other techniques in terms of results. It is vitally important to extract necessary information by image processing in order to develop an application of artificial intelligence based on computer science. In this paper, we have presented video processing techniques for background extraction and gas detection, which can later lead to the study of automatic classification using machine learning. It is beyond question that teaching a computer using background and foreground images extracted from these algorithms can produce better recognition results. We continue to experiment in using Deep Convolution Neural Networks for background image extraction and detection of gas leakages. Since a large amount of image database is required to do this, we have only performed preliminary experiments with potential for this paper and will produce results based on Convolution Neural Networks after conducting more experiments. The goal of the future research is to develop learning knowledge algorithms that allow computers to increase their recognition rate based on these images. In the ambient service environment, we share information on the status of all things in real time, and since all places and objects have sensors, we perform intelligent activities such as making decisions on our own. When the proposed camera system has a built-in detection sensor and is connected to a high-performance computer network, people are provided with gas leakage information in real time.
The author is grateful to the Young Kook Electronics Company for providing assistance with the experiment.
- Hong S, Kim K, Moon N (2017) Sector based multiple camera collaboration for active tracking applications. J Inform Process Syst 13(5):1299–1319Google Scholar
- Peng Y, Yan Y, Zhao J (2016) Detail enhancement for infrared images based on propagated image filter. Math Probl Eng 2016:1–12Google Scholar
- Piccardi M (2004) Background subtraction techniques: a review. In: 2004 IEEE international conference on systems, man and cybernetics, pp 3099–3104Google Scholar
- Raju A, Dwarakish GS, Reddy DV (2013) A comparative analysis of histogram equalization based techniques for contrast enhancement and brightness preserving. Int J Signal Process Image Process Pattern Recognit 6(5):353–366Google Scholar
- Tendero Y, Gilles J, Landeau S, Morel J (2010) Efficient single image non-uniformity correction algorithm. In: Proceedings of SPIE—the International Society for Optical Engineering, 7834Google Scholar
- Vollmer M, Möllmann K (2018) Infrared thermal imaging. Wiley-VCH, GermanyGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.