ABSTRACT
The goal of this study is to develop a smoke detecting algorithm using digital image processing techniques on multi-spectral (visible & infrared) video. By utilizing principal component analysis (PCA) followed by spatial filtering of principal component images the location of smoke can be accurately identified over a period of exposure time with a given frame capture rate.
This result can be further analyzed with consideration of wind factor and fire detection range to determine if a fire is present within a scene. Infrared spectral data is shown to contribute little information concerning the smoke signature. Moreover, finalized processing techniques are focused on the blue spectral band as it is furthest away from the infrared spectral bands and because it experimentally yields the largest footprint in the processed principal component images in comparison to other spectral bands. A frame rate of .5 images/sec (1 image every 2 seconds) is determined to be the maximum such that temporal variance of smoke can be captured.
The study also shows eigenvectors corresponding to the principal components that best represent smoke and are valuable indications of smoke temporal signature. Raw video data is taken through rigorous pre-processing schemes to align frames from respective spectral band both spatially and temporally. A multi-paradigm numerical computing program, MATLAB, is used to match the field of view across five spectral bands: Red, Green, Blue, Long-Wave Infrared, and Mid-Wave Infrared. Extracted frames are aligned temporally from key frames throughout the data capture.
This alignment allows for more accurate digital processing for smoke signature. Clustering analysis on RGB and HSV value systems reveal that color alone is not helpful to segment smoke. The feature values of trees and other false positives are shown to be too closely related to features of smoke for in solely one instance in time. A temporal principal component transform on the blue spectral band eliminates static false positives and emphasizes the temporal variance of moving smoke in images with higher order.
A threshold adjustment is applied to a blurred blue principal component of non-unity principal component order and smoke results can be finalized using median filtering. These same processing techniques are applied to difference images as a more simple and traditional technique for identifying temporal variance and results are compared.
THEORY AND ALGORITHMS
Figure 1: Visible Image of Active Fire Scene with Identified Smoke
Figure 2: RGB Feature Space of Fire Scene
Each pixel location in the image space has 3 coordinate values picked up from the respective red, green, and blue image. Throughout this study, fire scenes will be analyzed in both the feature space and the image space in order to successfully identify smoke. Figures 1 and 2 offer a comparison of the red, green, and blue channels plotted as a visible image (image space) and plotted in the feature space.
DATA ACQUISITION PREPROCESSING
Figure 9 : Multi-Spectral Frame Extraction Process
Figure 9 helps illustrate part of the data acquisition process. Visible data stored as an audio video interleave (AVI) file is extracted to frames saved as bitmaps (bmp). The Cooled MWIR and Cooled LWIR are stored as one as one dual band raw (RAW) bit stream and are extracted appropriately into separate raw images.
Figure 10 : Unregistered FOV
The visible camera and the IR camera have different field of views such that initially there is no direct spatial correspondence between images from each camera. Although the visible camera has a larger resolution than that of the IR camera, it has a much narrower field of view. The IR images will be cropped and mapped to the FOV and resolution of the visible images using an affine transformation. Twenty hand selected correlation coordinates used for the alignment are displayed in Figure 10.
FEATURE SPACE ANALYSIS
Figure 13 : Visible and Infrared Comparison
Figure 14 : Visible and Infrared Comparison
The infrared channels are investigated to see their contribution to smoke information. Below in figures 13 and 14, a visible image is compared to two respective infrared images:
DIFFERENCE IMAGING SCHEME
Figure 21 : Image and Histogram Comparison of Fire Scene with Visible Bands
If one of the visible channels yields a more pronounced smoke signature, future processing can be simplified by only viewing data from that channel. Below, aligned images from the three visible channels and their respective histograms can be observed. Qualitatively, it can be argued that the smoke in the center of the frame is most explicit in the blue channel. Using the manually identified smoke in section 4.2, the feature location of the smoke can be identified in each individual channel in figure 21.
Figure 23 : Block Diagram of Difference Image Threshold Scheme
After difference images have been summed together, blurring is used to smooth out the intensities of the image’s features. A selective threshold adjustment tool in Photoshop is then used to separate areas with high intensities from areas with low intensities. Median filtering is used to identify large blobs of variance and eliminate small and irrelevant locations of variance. The block diagram in figure 23 below shows the processing scheme on four difference images.
SELECTIVE THRESHOLD ADJUSTMENT ON PRINCIPAL COMPONENT IMAGES
Figure 27 : 5 Temporal Blue Images and Principal Component Images
Just as in chapter 5, five temporal images with a given frame rate are selected for temporal processing. A principal component transformation is applied to the temporal images and the resulting principal component images can be seen in figure 27. In the resulting principal component images, areas with large temporal variance have intensity values outside of the mean value of the image.
Figure 34: Comparison of Selective Threshold Adjustment on Visible Channels
Using five temporal images with 2 seconds in between each temporal image, results can be compared using different channels from the visible spectrum. Below in figure 34, the effects of using the red, green, and blue channels can be compared to one another. The following results reinforce that the blue channel provides the strongest spectral and temporal smoke signature.
CONCLUSION
Throughout this study, there have been many strong conclusions. In chapter 4, the infrared channels are shown to contain little spectral features associated with smoke. Chapter 4 and chapter 6 also show that the strongest smoke signature results from the use of the blue spectral band. These two results are in conjunction with each other as the blue spectral band is the furthest away from the infrared spectral band in frequency.
The feature space location of the smoke is determined to be unique; however, many other objects in the frame also occupy and overlap through this feature space. From this, it is determined that smoke can only be identified up to a certain confidence based on its spectral features. Difference image analysis in chapter 5 suggests at the ability to successfully segment the temporal variance of the smoke. However, this requires long periods of time between successive temporal frames (5 seconds minimum) and this process is susceptible to many false alarm rates as is shown in chapter 5.
An autonomous selective threshold adjustment scheme is shown to successfully identify smoke in chapter 6. The process requires 5 temporal images with a minimum period of 2 seconds between each image. The eigenvector associated with principal component images that greatly accentuate the smoke is treated as the temporal smoke signature, and can be used as a recipe to combine temporal images and reduce processing time by avoiding an entire principal component transform.
Source: California Polytechnic State University
Author: David C. Garges
>> 50+ Matlab projects for Digital Image Processing for Final Year Students
>> More Matlab Projects on Video Processing for Final Year Students
>> More Matlab Projects on Audio Processing for Final Year Students