PE&RS May 2002

Direct Georeferencing

Direct Georeferencing In Digital Imaging Practice

This month's column is intended to address the questions related to how good direct georeferencing technology is in practice. This question is answered by presenting experiences with orthorectified imagery production at Emerge. Recent columns in PE&RS have addressed the basic concepts and theory.

As is always the case in the engineering profession, accepting a new technology in place of traditional methods requires proof. The best evidence of proof comes from empirical results with favorable outcome that fits with the accepted theory. This article discusses the experiences from Emerge's orthorectified imagery collection and production business. A brief description of the system is introduced. Then, the results that build the empirical case for the validity of this technology are discussed in some detail.

At Emerge, the primary product is the orthorectified image mosaic. Therefore, it is critical that the product, which is generated from a large number of individual images, be seamless. To achieve this result on a production basis, several steps are taken as a matter of common practice. First, systems are calibrated and test flown over ground calibration fields before being put into operation. The primary reason is to validate the camera model and obtain best estimates of the camera to inertial measurement unit (IMU) boresight; a critical parameter set required for transforming the IMU-provided attitude angles into photogrammetric angles with respect to some local mapping frame of reference. The calibration field has three-dimensional, geodetically referenced, ground control points. The absolute positioning accuracy is at the meter level, while the relative accuracy is at the centimeter level. These calibration fields are re-flown as part of a maintenance program whenever the system is modified in any way that would impact the IMU to camera boresight or whenever the periodic maintenance schedule calls for it. This calibration work has built an important knowledge base that helps with the design and operation of current and future systems.

System Description
Emerge has been collecting and processing direct georeferencing airborne missions since 1996. Since 1998, Emerge has been processing digital imagery collections into orthorectified image products with this technology. The current rate of collection and processing is approximately 1000 missions per year. The sensors are essentially comprised of a digital imager with a resolution of approximately 2000 x 3000 pixels, a Novatel dual frequency Global Positioning System (GPS) receiver, and a Litton LN-200 IMU.

The imager has several focal length options. These options are used throughout a variety of flying heights that allow for collections with ground sample distance (GSD) ranging from approximately 1 ft. to 3 ft. or 0.3 m to 0.9 m. The 2000 pixel dimension is typically oriented along the flight path, allowing the largest imaging dimension to be across the flight path. This digital imager is small in format compared to a film-based mapping camera, which has the equivalent resolution of approximately 20k x 20k pixels. Although the format and typical lenses make the base to height ratios somewhat unattractive for elevation extraction, this imager has excellent characteristics for the production of orthorectified imagery, which is its primary usage. The digital imaging system has been calibrated for focal length, principal point, and lens distortion. The calibration was initially done at the USGS facility under special arrangement with USGS. The results were subsequently used to validate an airborne procedure that is performed using one of several calibration fields throughout the U.S. This body of experience demonstrates that the focal length is well behaved. Tangential lens distortion for these small format lenses is negligible; a small fraction of a pixel with standard errors of the same magnitude. Therefore, these tangential distortion terms do not contribute materially. Although the radial lens, is substantial, it is well behaved and able to be calibrated. Finally, for these particular camera bodies, the principal point is not as stable as one would find in a camera specifically designed for metric use. However, this has been stabilized with mechanical modifications and is checked with self-calibration techniques for each mission. The net result is a camera system that has an error budget for metric use of approximately 1 pixel.

From a navigational point of view, it is important to acquire continuous information about both the attitude (pointing direction of the camera principal axis) and the position of the aircraft. The IMU and GPS provide this function. This IMU has six outputs: three accelerometers, and three gyros (accelerometers describe the change in velocity in the x, y and z direction, while gyros measure any rotation along these axes), as shown in Figure 1.

There are two navigation data sets that are recorded during flight in the aircraft. The first is GPS. Every second, GPS issues one pulse per second (PPS) data. This PPS, which is the precise timing pulse associated with a GPS epoch, is recorded with the interval clock. This PPS and all other events are correlated using this interval clock. It is critical that the timing of all events in the aircraft be known precisely, so the interval clock on a counter timer card is employed by the data recording system. This is an interval clock that is stable for many hours at a time and records with an accuracy of one milli-second. The clock also monitors IMU measurements, and the camera events. However, the interval clock must be set to an absolute time reference such as that of the GPS. The GPS clock is highly stable for many years and the PPS times are known precisely. However, the clock ticks only once per second, making the GPS data a low frequency data source. Therefore, the system uses the interval clock to add the required time precision to all other measurements that occur in that one-second interval.

The recorded GPS data permit comparison of the data with a ground-based GPS receiver. The ground-based GPS antenna has been surveyed against national ground control networks such that it has its absolute geodetic location to sub meter accuracy. This allows calculation of the GPS position with greater precision than with directly broadcast signals alone. The method of calculating position is used by Emerge in its image product to employ "kinematic" GPS. This uses the phase measurements of the signal. Kinematic GPS calculations are accomplished by tracking the same satellites continuously with both the base station and the aircraft. Calculating a solution without cycle slips will provide typical accuracy at the decimeter level. One last consideration for GPS collection is the distance between the base station and the aircraft. This distance is referred to as the "baseline." As the baseline increases, the accuracy will decrease. With the dual-band receivers that are used on Emerge aircraft, baselines have been shown to hold excellent accuracy when less than 200 miles and will be good for baselines less than 400 miles. The error budget for GPS is to maintain GPS positional accuracy of better than one meter, with decimeter accuracy being the desired level.

The combination of these two independent information sets, GPS and IMU, with a Kalman filter allows the accurate determination of both attitude (roll, pitch, heading) and position (x, y, z) of the camera at the time of exposure. This results in the same information that is normally obtained in photogrammetric block or bundle adjustments used in film collections. The primary operational difference is that the Emerge approach does not require ground control to obtain the solution of the attitude and position data. This saves large amounts of labor and schedule time, resulting in a low cost, georeferenced image product. Most importantly, the GPS and IMU data are independently collected measurements. One provides sensor locations based on time measurements from a spacecraft constellation, while the other provides locations from the integration of accelerometer measurements and angles from highly correlated gyro measurements. The Kalman filter is used to reconcile the errors in these two systems and reports what those error magnitudes are. The output from the Kalman filter includes the error estimates associated with the position and attitude of the camera.

Traditional photogrammetric techniques derive the camera attitude and positions by performing a space resection using ground control, tie points, and camera model geometry. One of the strengths of this approach is its ability to adapt to the ground control data. However, there is only one primary measurement tool in the system, the camera model. This is reconciled with the ground control, but the control can be local in nature. This traditional technique therefore does not see biases in the data. The advent of GPS has helped this issue by providing efficient methods of global reference for both the aircraft collection and ground control. However, it is difficult to overstate the value of combining GPS with IMU to provide two independent, global references. If these are subsequently compared with a bundle adjustment, the user has three independent measurements to compare. Emerge does that as a matter of procedure when processing the calibration field data. First, GPS is processed, then the IMU and GPS is combined with Kalman filtering, finally a photogrammetric bundle adjustment is performed incorporating ground control. The two objectives are to acquire calibration of the IMU-to-camera boresight and to check the camera calibration parameters. In addition, this test range process also has illustrated how good the direct georeferencing data are.

A Note on Navigation Data Accuracy
An important metric to understand is how accurate the positional and attitude information is that comes directly off the aircraft. Filtering and smoothing are used to estimate the resulting errors of two distinct and separate measurements of the aircraft location. Namely, GPS that has been processed through a kinematic solution and the IMU data that compares its integrated accelerometer data with the GPS positional (and/or velocity data). The analysis from many missions suggest that the navigation solution provides positional accuracy on the order of decimeters and attitude accuracy of approximately 20 arc seconds for pointing down (roll and pitch) and approximately 30 arc seconds for heading. This is all directly a result of navigation processing alone.

Within the photogrammetric community, the determination of position and attitude is typically done through aerotriangulation by measuring ground control on the imagery and interpolating between the imagery using tie points. Here too, accuracy is estimated. One very good test of the system accuracy however is the data set that results from orthorectifying imagery that came out of a block adjustment. Two independent measures will testify to the accuracy of the process, the size of tears along mosaic seams and the placement accuracy of checkpoints, etc.

Emerge performs an adjustment of attitudes and positions derived by the navigation solution by measuring tie or pass points and using those data to adjust the imagery block. This has the effect of reducing small tears that are present in the Quality Check (QC) mosaics. At the end of the adjustment process the tears for points that lie on the ground (as opposed to those on building roofs etc. where the DEM is not representative) is typically one or two pixels. Table 1 represents the differences between the GPS/IMU-provided solution and the final adjusted attitudes and positions. The result comes from a data set that produced a mosaic with tears of less than 2 pixels and an accuracy compared to check points of approximately one meter. These data came from a test mission over the control range at the Utica/Rome airport.

Table 1. Difference Between Camera Exterior Orientation Provided by GPS/IMU
and Aerotriangulation (Position in meters & Angles in Arc Seconds)

	Lattitude	Longitude	Attitude	Roll	Pitch	Heading
Average Error	0.006	0.014	-0.005	-0.450	0.000	0.000
Std. Error	0.121	0.061	0.224	19.581	16.783	32.146

Note that the errors are consistent with the estimates that are available from the Kalman filter. That is, the positions are good to a decimeter, and the angles are good to 20 arc seconds in roll and pitch and to 30 arc seconds in heading. This appears to indicate that these errors reported from the Kalman filter are representative of absolute accuracy estimates.

These data have mean adjustment values that are essentially zero. This illustrates that there is no apparent bias in the GPS/IMU direct georeferencing data. Note that, the GPS adjustments are of the same order of magnitude as the anticipated GPS accuracy. The roll and pitch adjustments are approximately 20 arc seconds which is the error of magnitude reported by the Kalman filter. The yaw estimate is 32 arc seconds vs. the reported 30 arc second error.

Summary
The experience gained from the many missions produced by Emerge over the past six years, and over the test ranges provides evidence that the error estimates provided with direct georeferencing techniques using GPS/IMU are representative of absolute errors for camera position and attitude. This is intuitively pleasing since the IMU and GPS measurements used in the process are independent measurement techniques.

Gerry Kinn is the Senior Advisor with Emerge, USA.
Edited by Dr. Mohamed M R Mostafa, Applanix Corporation