PE&RS May 2003

Direct Georeferencing

Real-Time Direct Georeferencing of Thermal Video for
Forest Fire Hot-Spot Detection

Time is of the essence when it comes to fighting forest fires; that and limited resources, both in equipment and manpower. Any uncontrolled fires need to be identified and located as quickly as possible to allow forestry personnel to evaluate the risk factors associated with that fire, and then direct suppression operations appropriately. Delays of hours can potentially cause millions of dollars in damages and economic losses. Our goal has been to develop a system to quickly and automatically identify and locate small forest fire hot spots. A key enabling technology for this is direct geo-referencing hardware. The other key technology is the thermal imaging camera, which has been available and used by forestry personnel in forest fire fighting for several years. The combination of the direct georeferencing system, a thermal imaging camera, and current computer processing muscle opens the door to the development of such a system.

From field observations of post suppression surveys, and detailed analysis of thermal videos collected during forest fires, the following operational requirement was defined for this project. Step 1 is the identification and segmentation of forest fire hotspots from thermal video at 30 frames per second. Next, WADGPS at 5 Hz and IMU data at 100 Hz is used for direct geo-referencing of each video frame. Finally, the system reports the date, time, and position of the aircraft, approximate area covered on a continuous basis, and also the position of any hot spots identified within 30 seconds of detection via a low speed satellite internet data link. The high video capture data rate will be explained shortly.

There is a lot going on in the background to make this system work. A commercial GPS/IMU unit has been used as the direct georeferencing component. An Electro-Physics 10 bit 2-14 µm imager has been used as the thermal imaging component. This was combined with a Dual PIII computer to give us the hardware platform that was used. Then there are the real time programming aspects of the system to make it all work. Figure 1 shows the software flow.

The real time capability of the system is dictated by the slowest module in the system. The entire software architecture carefully utilizes multithreaded programming and real time principles to maintain a specific data flow through the system. This all comes down to inter thread signaling, control thread switching, appropriate memory allocation and hardware control, and accurate timing and synchronization. The two key bottlenecks are in the thermal image processing module to identify the hot spots and to store the data.

Hot-spot identification requires complex image processing. This implies solar gain issues, reflections, and object emissivities. In addition, it implies application-specific issues that dictate additional processing requirements. One critical application issue is that small hot spots may be at the base of a tree. In that case, hot spots are often obscured by the surrounding foliage, causing the hot spot to flicker in and out of sight. This is what dictated the need to work at the higher video frame rate for the system, and also why motion estimation and tracking is utilized to consistently label hot spots from frame to frame.

Contrary to what Hollywood movies may lead people to believe, thermal imagers do not see through objects. What is seen by a thermal imager is the radiation in the infrared (IR) band that a particular object is radiating. However, anything behind the object is effectively blocked. It is only when objects are in contact that one can see the heat conducted to the other object, making it appear that one can see through it. In some cases, a material is effectively transparent to the thermal radiation, but this is not the case for most common materials.

Even with the data flowing consistently in the system, several other parameters needed to be determined. Every optical imaging system has some distortion that needs to be corrected for by calibrating the optics. Additionally, the camera center, GPS antenna, and IMU center positional offsets are determined. This is called a lever arm calibration, which can be determined by explicitly measuring, by post-adjustment averaging using bundle adjustment, or a combination of measurements and adjustment. Furthermore, IMU/camera boresight is also calibrated using bundle adjustment. Boresight calibration is an extremely critical calibration item and in reality a non-trivial operation in our case.

Currently, the real time data acquisition with thermal video and WADGPS /IMU Synchronization has been tested and proved functional. Certain stability issues are currently under investigation. Also, preliminary testing and evaluation proved that real-time filtering, feature extraction and tracking, and the triangulation of extracted features for final position identification are all in good order.

In conclusion, the system is feasible using today’s technology, and our research and development efforts are dedicated to making this system a key component in the arsenal of equipment that a forestry department has at its disposal to fight fires.

D. Bruce Wright and Dr. Naser El-Sheimy are with Mobile Multi-Sensor Group, Department of Geomatics Engineering, University of Calgary.

Edited by Dr. Mohamed M.R. Mostafa, Applanix Corporation.

Top

Home