Automatic Surveillance Methods
Automatic surveillance methods are based on the systems used to monitor of wildlands or wildland urban interfaces (WUI) for the purpose of detecting wildfires in their initial phase (early wildfire detection). They are usually based on the advanced analysis of data received from sensors suitable for detecting various wildfire features, particularly those characteristic of the early wildfire stage, such as wildfire smoke and/or flames. In most cases, these sensors are video cameras that are sensitive to various electromagnetic spectra.
In the firefighting community, the necessity of detecting wildfires in their initial stage is well known. Wildfire fighting efforts and potential wildfire damage are proportional to the time passed between wildfire ignition and detection (Kourtz 1987). Therefore, enormous efforts have focused on developing methods and systems capable of detecting wildfires as soon as they are ignited.
There are few advantages of using a video camera-based system. The human observer is capable of monitoring a wider area covered by several video monitoring field units. Also cameras are usually equipped with power zoom, so the human observer can easily inspect areas that are suspected to contain a wildfire. Another advantage is that an observer or firefighter commander could, in the case of real fire, virtually monitor a distant wildfire on video. He or she could monitor the fire spread and behavior on video screens and manage firefighting actions from a distant location. This is particularly useful if the area under observation is covered by several video cameras. Finally, such systems usually have video storing capabilities that can be quite useful for postfire analyses and for the preparation of wildfire reports.
One limitation of a video-based system is that wildfire detection depends entirely on the human observer. The human observer is responsible for spotting the fire and activating the fire alarm. He or she has to carefully watch computer screens around the clock, so problems like fatigue, boredom, and loss of concentration are often encountered. Therefore, great efforts have been made to make this job easier for the human operator. The results have led to innovative wildfire surveillance techniques where routine wildfire detection is left to an advanced computer detection algorithm, and the task of the human observer is only to perform a final check and decide whether or not a wildfire is real. This system is usually called automatic wildfire surveillance, although it is not entirely automatic but rather a semiautomatic system. The main task of such a technical system is to preselect images with possible wildfire traces, yet the final decision is still left to a human operator.
Various types of sensors can be used for automatic wildfire detection. In most cases, particularly in commercial systems, detection is based on video cameras that are sensitive to visible and/or infrared (IR) spectra. Systems based on optical spectrometry, light detection and ranging (LIDAR), Doppler weather radar, radio acoustic sounding (RASS), and acoustic volumetric scanning (VAS) sensor networks have also been proposed as viable options (Stipaničev et al. 2010). Even so, human wildfire observers primarily use their sense of sight to detect wildfires; therefore, video-based systems are perhaps the most appropriate devices for replacing the human observer and for mimicking human wildfire detection methods. This is one of the reasons why video-based systems are the most widely used systems today. The other reason is that video cameras, particularly those sensitive to the visible spectra, enable a virtual presence at the location of potential wildfires.
Video-based systems are further divided into ground-based (terrestrial) systems in which cameras are installed on monitoring locations on the ground; air-based systems that use cameras installed on various flying vehicles like drones, unmanned aerial vehicles (UAVs), airplanes, helicopters, and balloons; and satellite-based systems wherein cameras are installed on satellites.
Future monitoring will likely be performed using air- or satellite-based systems. However, the current challenge in implementing the wider use of such systems for early wildfire detection is due to the low spatial/temporal resolution of many of these cameras or the resulting cost of high-quality equipment. Therefore, most automatic wildfire surveillance systems currently use terrestrial video-based systems. The majority of them are equipped with pan-tilt-zoom (PTZ) high-definition (HD) cameras that are sensitive to the visible light spectrum, although the use of dual systems equipped with one HD camera sensitive to the visible spectrum and another thermal camera sensitive to the infrared (IR) spectrum on the same PTZ unit is also encountered.
Automatic Video-Based Wildfire Surveillance Systems
Automatic video-based wildfire surveillance systems are technical devices designed to mimic the surveillance tasks of a human wildfire observer. Generally, the observer is a person or a technical device capable of perceive or becoming aware of a wildfire in a monitored area. A potential wildfire is detected through perception, which can be described as a process in which sensory stimulations are translated into organized and meaningful experiences (Lindsa and Norman 1977). In other words, perception is a process of acquiring, selecting, organizing, and interpreting sensory information.
Each conceptualization has two spaces: the configuration space and the observation space. The configuration space of the low-level observerX is the real, three-dimensional (3D) space where various natural phenomena, like storms, lighting, fog, rain, and wildfires, occur. Subset E is our phenomenon of interest: wildfire. This subset E can be labeled as the configuration event of the low-level observer. Vision sensors map the 3D phenomena from the real world into a series of 2D images (video sequence). All video sequences generated by vision sensors form the observation space of the low-level observer, which is called Y in Fig. 2. Only a limited number of these video sequences correspond to an actual wildfire in the real world. They form subspace V, or the observation event of the low-level observer.
Space Y is the starting point for the high-level observer, or the configuration space of the high-level observer, while subset V is the configuration event of the high-level observer. The task of the high-level observer is to recognize wildfires from video sequences. The observation space called Z is a collection of single images, each corresponding to one input video sequence. In the case that a wildfire is detected, the observation corresponds to high-level observer event A. The final output images of observation event A are usually called alarm images.
In human wildfire surveillance, a human operator located at an observation location can simultaneously function as a low-level and a high-level observer. Humans use stereovision as low-level observers to map the real world in a human 3D representation. A human wildfire recognition process based on certain visual characteristics of wildfires realizes the task of high-level observer.
In video-based human wildfire surveillance, the human operator acts as a high-level observer. The low-level observer is the technical system composed of video cameras, communication devices, and presentation devices (monitors, computers, etc.). The video camera, transmission, and presentation systems have certain limitations, so real wildfires are not always visible on the computer screen. Human operators can only recognize those wildfires that are well presented on computer screens. For the formal perception model, the human operator who recognizes a wildfire based on the presentation of a wildfire on a computer screen can be referred to as a reference observer; this person is usually called the “ground-truth” observer in computer science. The capacity of a reference observer to correctly recognize wildfires is often used to evaluate the performance of an automatic wildfire surveillance system.
Meanwhile, in automatic video-based wildfire surveillance, both the low-level and the high-level observers are reliant on technical systems. The low-level observer is the same as that in video camera-based human wildfire surveillance systems, although the presentation devices are distinct. Video sequences form space Y form the configuration space of the high-level observer. Video sequences with wildfires recognized by reference human observers are part of configuration event V of space Y. Automatic wildfire recognition is modeled by mapping configuration space Y on the observation space of the high-level observerZ. The observation space Z contains all images obtained from processed input video sequences from Y. Images with positively detected wildfires form observation events of the high-level observerA. A is the subset of space Z.
Wildfires are a natural phenomenon and constitute a complex spatial and temporal event with specific physical and chemical characteristics. For wildfire detection, characteristics that can be easily captured by sensors are relevant. For human- and video-based wildfire observers, these characteristics are primarily static or dynamic visual characteristics that are typical of wildfires in their incipient stages. Wildfires are often detected in open spaces where smoke is usually visible before flames. Therefore, for early wildfire detection, the visual characteristics of smoke generated by wildfires are of primary importance.
Visually, wildfire smoke can appear as a nontransparent phenomenon of characteristic color or as a semitransparent phenomenon that greatly impacts the color of the background in front of which it propagates. Wildfire smoke is gray yet varies in color from white to black depending on the type of burned material and its moisture content. Smoke also has a specific texture and morphological characteristics. Color, texture, and morphology are static or spatial visual characteristics. Wildfire smoke also has very specific dynamics and temporal characteristics that are characterized by, for example, growth rate, boundary flickering, self-similarity (Xu and Xu 2007), and specific motion in addition to the velocity distribution of smoke plumes and temporal variations in gray levels (Vicente and Guillemant 2002). The visual characteristics of both spatial and temporal wildfire smoke are used in the wildfire recognition process.
In comparing the detection results of an automatic wildfire detection system and a reference observer (human), there are four possible outcomes: correct detection (CD), when both have correctly detected the wildfire; missed detection (MD), when the automatic detection system has not detected the wildfire but the human observer has; false detection (FD), when the automatic detection system has detected the wildfire and human observer has not; and correct rejection (CR), when neither have detected the wildfire.
Technical Realization of Automatic Wildfire Surveillance Systems
The camera units are the sensors and the sources of data for automatic wildfire detection and also are used to enable distant video presence. Automatic wildfire surveillance systems usually have three working modes.
The automatic wildfire detection mode is based on advanced algorithms for digital image analysis and recognition using typical spatial and temporal visual characteristics of wildfires.
The manual mode can be used to take advantage of distant video presence and an advanced camera’s pan-tilt-zoom (PTZ) controls. The manual mode is used during the detection phase for distant inspection of suspicious phenomena but also in the firefighting phase for distant monitoring of fire behavior and spread.
The archive retrieval mode can be employed for post-alarm analysis and report generation.
Currently, all three modes are generally supported by advanced geographic information system (GIS) applications used to locate and to determine alarms in automatic and archive modes and for advanced pan-tilt-zoom (PTZ) cameras control in manual mode.
Automatic wildfire detection is based on advanced digital image processing and analysis algorithms. Usually, several algorithms analyze input video sequence images in order to extract and to recognize the visual characteristics of wildfire smoke during the day and of wildfire flames during the night. Visual smoke or flame characteristics can be extracted from a single image, based on spatial characteristics (intra-frame, static), or from a sequence of images, based on temporal characteristics (inter-frame, dynamic). The spatial characteristics of wildfire smoke or flames in images are mostly determined by chromatic, texture, frequency, or shape characteristics, whereas temporal characteristics are determined by motion dynamic analysis as well as temporal variations in the color and shape of detected regions, wavelet energy, or motion orientation (Stipaničev et al. 2012).
The final step is usually to use a certain type of classifier or device responsible for making the final decision. The inputs to the classifier are the detection results from various spatial and temporal detection algorithms, and the outputs from the classifier are the images with detected or non-detected wildfires. The classifier is usually a smart one that marks areas suspected to contain wildfires on the output images (see Fig. 3). Figure 4 schematically shows the organization of an automatic wildfire detection system. If a wildfire is detected, a human operator is alarmed (flashing border on image, audio alarm signal). His or her job is to make final inspection and to decide whether or not the detected wildfire is real. Therefore, we can emphasize that today’s automatic wildfire systems are largely semiautomatic systems in which well-trained human operators are still of prime importance.
Limitations and Future of Automatic Wildfire Surveillance Systems
There are several limitations of today’s automatic wildfire surveillance systems. First of all, there are various sources of substantial uncertainty in detecting wildfires. Video-based systems sensitive in visible spectra detect possible wildfires by smoke and/or flames and systems sensitive in infrared (IR) detect possible wildfires by hotspots. Flames and hotspots must be directly visible. On a hilly terrain, there are lot of places not directly visible by camera, so there are lot of hidden views and places where flames or hotspots could not be detected. Therefore, on a hilly terrain, smoke detection is a better solution, but as smoke in these cases appears in the sky above the hills, it can easily be replaced with clouds.
Another limitation of today’s video-based is a critical detection size. In automatic mode, typical detection demand is the system ability to detect a wildfire smoke of 10 × 10 m at the distance of 10 km. Contemporary cameras are usually HD cameras with power zoom and quite wide initial field of view (typically at least 50° horizontally for zoom = 1). In that case for zoom = 1, the smoke size we want to detect takes up only a few picture elements (pixels), not enough for any serious detection algorithm. Minimal detection size is typically 50 pixels. For this reason, higher zoom should be used, but higher zoom means the narrower field of view of the camera, resulting in a considerably increased number of preset positions required to cover the entire 360° angle.
A special problem is detection of smoldering wildfires (Watts and Kobziar 2013). Smoldering fire is combustion without flames. Smoke from smoldering fires often dissipates slowly, more like a mist then like a typical smoke cloud, so most smoke detection algorithms have difficulties to detect it. Techniques based on wireless sensor networks have been purposed (Yan et al. 2016), but their commercial use was largely absent. As the burning temperature in combustion fires is usually between 500 and 700 °C (Watts and Kobziar 2013), the greatest success in detection smoldering fires has cameras sensitive in infrared (IR) spectra. Today firefighters are increasingly using drones equipped with IR cameras, particularly after large fires in order to detect hotspots and possible smoldering fires.
In wildland urban interfaces (WUI), automatic fire detection is not so significant because today in the era of mobile communication, every occurrence of fire is usually reported several times by citizens using mobile phones, either to firefighters or emergency services. However, as firefighters point out, the problem of reporting fires by citizens is that often a description of the situation usually does not correspond to the actual state. In these cases, particularly visible video-based surveillance systems are quite useful in their manual mode that allows remote video presence at the site of a potential fire. A professional firefighter will quickly, using images from video camera, figure out the potential fire hazard and how to prepare the intervention. In addition, archived camera images can be of great help in investigating the causes of fire ignition and help detect an eventual culprit for its occurrence. If the entire area is covered by cameras, the firefighter’s commander can parallelly manage firefighting of multiple fires directly from the monitoring center.
Automatic wildfire surveillance systems are widely used today, and it is expected that more and more will be used in the future. The future system will surely overcome lot of today’s drawbacks and problems. Maybe today, the biggest detection problem is still the significant high rate of false alarms. Therefore, great efforts are made to develop new, more advanced detection algorithms. Computational intelligence methods, particularly new technologies using deep learning techniques, are one of them.
Other issues to be addressed in future automatic wildfire surveillance systems are the fusion of various electromagnetic ranges (multispectral and hyper-spectral imaging) in video detection systems as well as the fusion of visual surveillance systems with advanced sensory networks.
Lastly, but not least, future work will likely focus on remote sensing and various aerial- or satellite-based wildfire surveillance systems as well as the synergy between these and ground-based wildfire surveillance.
- Benett BM, Hoffman DD, Prakashi C (1989) Observer mechanics – a formal theory of perception. Academic, New YorkGoogle Scholar
- Jakovčević T, Stipaničev D, Krstinić D (2009) False alarm reduction in forest fire video monitoring systems. MIPRO 2009 – 32nd international convention on information and communication technology, electronics and microelectronics, Opatija, Croatia, 25–29 May 2009, pp 264–269Google Scholar
- Jakovčević T, Šerić Lj, Stipaničev D, Krstinić D (2010) Wildfire smoke-detection algorithms evaluation. VI international conference on forest fire research, Coimbra, Portugal, 15–18 Nov 2010, 053Google Scholar
- Kourtz P (1987) The need for improved forest fire detection. For Chron 63(4):272–277Google Scholar
- Lindsa P, Norman DA (1977) Human information processing: an introduction to psychology, 2nd edn. Academic, New YorkGoogle Scholar
- Stipaničev D, Štula M, Krstinić D, Šerić LJ, Jakovčević T, Bugarić M (2010) Advanced automatic wildfire surveillance and monitoring network. Proceedings of VI international conference on forest fire research, Coimbra, Portugal, 15-18 Nov 2010, 052Google Scholar
- Stipaničev D, Šerić LJ, Braović M, Krstinić D, Jakovčević T, Štula M, Bugarić M, Maras J (2012) Vision based wildfire and natural risk observers. Proceedings of 3rd international conference on image processing theory, tools and applications, OS1: special session on Image Processing for Natural Risks, Istanbul. Turkey, 15–18 Oct 2012, p. 271Google Scholar
- Taylor JR (1997) An introduction to error analysis: the study of uncertainties in physical measurements, 2nd edn. University Science Books, SausalitoGoogle Scholar
- Xu Z, Xu J (2007) Automatic fire smoke detection based on image visual features. International conference on computational intelligence and security workshops (CISW 2007). Harbin, Heilongjiang, China, 15-19 Dec 2007, pp 316–319Google Scholar