A Robust Flame Detection Method Base for Shaky Real-Time Video

A Robust Flame Detection Method Base for Shaky Real-Time Video

Our proposed was accepted to the engineering technology application conference in 2019. The conference place is in the College of Engineering of the Taipei City University of Science & Technology (TPCU) in Taiwan (R.O.C.), and Conference Date on 12th April, 2019.

We provide the files of our study that the part of the source code of the HEFD-MCS (A High Efficiency Flame Detection Mobile Camera System), and experiment files. These files are in the GitHub, and the hyperlink is https://github.com/laitaiyu/ComputerVision__HEFD-MCS .

Two key issues of flame detection are high accuracy and low false alarm rate, and this study focuses the latter. In the past, static video from surveillance camer-as was used to analyze and find the flames. However, these studies were conducted us-ing the background method, which cannot process dynamic videos such as the video taken with a mobile phone. In order to achieve full automated flame detection for both static and dynamic video, two high-efficiency strategies are utilized in this study. Firstly, Feature Extraction: The Strong Sobel edge and the Flame Texture are established and then, the image edge is enhanced. The Flame Color Filter rules are helping filter the flame candidate regions. Secondly, High Performance Analysis Method: To filter flame or non-flame though by the motion vector method and the Fill rate, and then use groups to establish the contours of the flame. Our experimental results show that the results for static, dy-namic (including shaky) and real-time vid-eo is 92.83% (TP), 9.76% (FP), 1.77% (FN).

This study aims to detect flames from unstable fire scene footage or footage shot from different angles. This section discusses the feasibility and reliability of the proposed flame detection method. A High Efficiency Flame Detection Mobile Camera System (HEFD-MCS) was constructed using the proposed method. The experimental results prove that the HEFD-MCS can correctly catch the flames.

Figure 1. The flow chart of flame detection system.
Figure 1 shows the process of the proposed system. Flame detection has two main steps. Firstly, Feature Extraction: The Strong Sobel edge and the Flame Texture is established and then, the image edge is enhanced. The Flame Color Filter rules are helping filter the flame candidate regions. Secondly, High Performance Analysis Method: To filter fire or non-flame though by the motion vector method and the Fill rate, and then use groups to establish the fire detection contours.

Most shaky videos come from the I.M.C. database (Intelligent Media Computing Laboratory, Sun Yat-Sen University). As per I.M.C. fire dataset website as of January 19, 2015, http://vision.sysu.edu.cn/systems/fire-detection/). Forest fire 1 and forest fire 2 videos both come from A. E. Cetin [2]. (As per Computer Vision based fire detection software as of November 12, 2018. http://signal.ee.bilkent.edu.tr/VisiFire). In the popular standard test video, a Christmas tree can burn in 2 seconds from National Institute of Standards and Technology in the U.S.

Video Type	Static Video: 10 videos Video Source: A. E. Cetin [2], National Institute of Standards and Technology, Intelligent Media Computing Laboratory, Sun Yat-Sen University						Shaky (Dynamic) Video: 5 videos Video Source: Intelligent Media Computing Laboratory, Sun Yat-Sen University
Authors	True Positive	False Positive	True Negative	Precision	Accuracy	F-Measure e	True Positive	False Positive	True Negative	Precision	Accuracy	F-Measure e
[1]	732.3	114.2	114.2	26.15	27.2	34.3	1242.4	754.4	0	69.19	67.85	80.02
[14]	720.1	114.2	114.2	26.16	27.04	34.11	1242.4	576.6	0	73.56	73.56	83.09
[18]	1421.8	174.3	174.3	63.6	65.01	72.62	1242.4	153.6	0	89.49	89.49	93.07
HEFD-MCS	1574.6	238.55	318.36	90.6	88.7	90.5	1240	150.6	0	89.89	89.55	93.11

	Video Type				Static Video: 2 videos Video Source: A. E. Cetin [2] , UltimateChase.com
Authors	Title	Frame Per Second (FPS)	Resolution (Pixel)	Total Frames	True Positive	False Positive	False Negative	Average Time Per Frame
[5]	Forest fire 2	15	400×256	178	24.7%	No provided	No provided	2 Second
[17]	Pos Video 11	30	352×288	505	19.7%	No provided	No provided	‪50 Milli-Second
HEFD-MCS	Forest fire 2	15	400×256	178	100%	0%	0%	80 Milli-Second
	Pos Video 11	30	352×288	505	78.2%	8.7%	0%	40 Milli-Second
	Video Type				The Negative Sample of the Static Video: 5 videos Video Source: Intelligent Media Computing Laboratory, Sun Yat-Sen University
	5 Sample Videos				98%	2%	0%	40 Milli-Second
[1]	5 Sample Videos				0%	100%	0%	30 Milli-Second
[14]	5 Sample Videos				0%	100%	0%	30 Milli-Second
[18]	5 Sample Videos				67%	33%	0%	30 Milli-Second

	Shaky (Dynamic) Video: 1 video Video Source: UltimateChase.com
Authors	True Positive	False Positive	False Negative	Average Time Per Frame
[5]	The study [5] uses a background model. Therefore, it is not suitable the shaky video.
[17]	83%	No provided	No provided	50 Milli-Second
HEFD-MCS	Shaky (Dynamic) Video: 5 videos Video Source: Intelligent Media Computing Laboratory, Sun Yat-Sen University
	90%	12%	0.19%	150 Milli-Second
[1][14][18]	These studies [1][14][18] use their background model. Therefore, they are not suitable the shaky video.

This paper proposed a high efficiency flame detection approach for a real-time video surveillance system. Although false alarms (FP) and missing rate (FN) are not completely eliminated, they are all within acceptable ranges. This study yields high accuracy and differs from the other studies with shaky (dynamic) video analysis. Most studies are characterized by motion vectors for background or static background rules that are inapplicable to shaky (dynamic) video and heavy computational workload that is not suitable for mobile devices. This study requires low computational workload, making it perfect for mobile devices. Given to the prevalence of mobile devices, reporting fire incidents via them is a real-time and reliable application. We propose a highly efficient flame detection algorithm that is applicable to static, dynamic, indoor, outdoor and real-time video. This study is an open source research that hyperlink is https://github.com/laitaiyu/ComputerVision__HEFD-MCS/ .

[1] A. Chenebert, T. P. Breckon, and A. Gaszczak, "A Non-Temporal Texture Driven Approach to Real-Time Fire Detection," In IEEE International Conference on Image Processing, Brussels, 2011, pp. 1741-1744.
[2] A. E. Cetin. (2018, Nov. 12). Computer Vision Based Fire Detection Software. [Online]. Available: http://signal.ee.bilkent.edu.tr/VisiFire/
[3] A. Stadler, T. Windisch, and K. Diepold, "Comparison of Intensity Flickering Features for Video Based Flame Detection Algorithms," Fire Safety Journal, vol. 66, May 2014, pp. 1-7.
[4] B. C. Ko, S. J. Ham, and J. Y. Nam, "Modeling and Formalization of Fuzzy Finite Automata for Detection of Irregular Fire Flames, "IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 12, Dec. 2011, pp. 1903-1912.
[5] B. Jiang, Y. Lu, X. Li, and L. Lin, "Towards A Solid Solution of Real-Time Fire and Flame Detection," Multimedia Tools and Applications, vol. 74, no. 3, Jul. 2014, pp. 689-705.
[6] B. M. N. de Souza, J. Facon, D. Menotti, "Colorness Index Strategy for Pixel Fire Segmentation," 2017 International Joint Conference on Neural Networks, Anchorage, AK, USA, 2017, pp. 1057-1063.
[7] B. U. Toreyin, Y. Dedeoglu, U. Gdkbay, and A. E. Cetin, "Computer Vision Based Method for Real-Time Fire and Flame Detection," Pattern Recognition Letters, vol. 27, no. 1, Jan. 2006, pp. 49–58.
[8] J. Y. Kuo, T. Y. Lai, Y. Y. Fanjing, F. C. Huang, Y. H. Lao, "A Behavior-Based Flame Detection Method on Real-Time Video Surveillance System," Journal of the Chinese Institute of Engineers, vol. 38, no. 7, Jun. 2015, pp. 947-958.
[9] K. Souza, S. Guimaraes, Z. Patrocinio, A. de A Araujo, and J. Cousty, "A Simple Hierarchical Clustering Method for Improving Flame Pixel Classification," 23rd International Conference on Tools with Artificial Intelligence, Boca Raton, FL, 2011, pp. 110-117.
[10] M. Mueller, P. Karasev, I. Kolesov, and A. Tannenbaum, "Optical Flow Estimation for Flame Detection in Videos," IEEE Transactions on Image Processing, vol. 22, no. 7, pp. 2786-2797, Apr. 2013.
[11] National Fire Protection Association. NFPA 72. Accessed August 10 2013, http://www.nfpa.org/codes-and-standards/document-information-pages?mode=code&code=72.
[12] S. Wang, Y. He, J. Zou, B. Duan and J. Wang, "A Flame Detection Synthesis Algorithm," Fire Technology, vol. 50, no. 4, pp. 959–975, Jul. 2014.
[13] T. Celik, H. Ozkaramanli, and H. Demirel, "Fire Pixel Classification Using Fuzzy Logic and Statistical Color Model," In IEEE International Conference on Speech and Signal Processing, Honolulu, HI, 2007, pp. I-1205-I-1208.
[14] T. Çelik, H. Demirel, and H. Ozkaramanli, "Automatic Fire Detection in Video Sequences," in Proc. of the 14th European Signal Processing Conference, Florence, Italy, Sep. 4-8, 2006.
[15] T. Çelik, H. Özkaramanl, and H. Demirel, "Fire and Smoke Detection without Sensors," in Proc. of the 14th European Signal Processing Conference, Florence, Italy, 2007, pp. 1794–1798.
[16] W. T. A. Budi, and I. S. Suwardi, "Fire Alarm System Based-On Video Processing," In International Conference on Electrical Engineering and Informatics, Bandung, 2011, pp. 1-7.
[17] Y. Habiboglu, O. Günay, and A. E. Cetin, "Covariance Matrix-Based Fire and Flame Detection Method in Video," Machine Vision and Applications, vol. 23, no. 6, Sep. 2012, pp. 1103–1113.
[18] Y. Qiang, B. Pei, and J. J. Zhao, "Forest Fire Image Intelligent Recognition Based on the Neural Network," Journal of Multimedia, vol. 9, no. 3, Mar. 2014, pp. 449-455.