Mapping Matters
By Qassim A. Abdullah Ph.D. PLS CP
Your Questions Answered
The layman’s perspective on technical theory
and practical applications of mapping and GIS
Please send your question to Mapping_Matters@asprs.org and indicate whether you want your name to be blocked from publishing.
Answers for all questions that are not published in PE&RS can be found on line at www.asprs.org/mapping_matters.
Dr. Abdullah is the Chief Scientist at Fugro EarthData Inc. Frederick MD.
The contents of this column reflect the views of the author who is responsible for the facts and accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the American Society for Photogrammetry and Remote Sensing and/or Fugro EarthData Inc.
2010
[March 2010] [April 2010] [July 2010] [August 2010] [September 2010] [October 2010]
October 2010 (Download a PDF 443Kb)
Question: I noticed that the vertical accuracy is more stringent than the horizontal accuracy according to both ASPRS and NSSDA standards. For example if I produce orthophoto products from 15 cm (6 in) digital imagery the stated horizontal accuracy using the ASPRS standard is 30 cm (1 ft) while the expected vertical accuracy is 20 cm (0.67 ft). We always believed that the vertical accuracy of any mapping product is less stringent than the horizontal accuracy. Why is that? Evgenia Brodyagina Frederick Maryland – USA
This answer contains graphics and tables. Please see the PDF
September 2010 (Download a PDF 385Kb)
Question: I noticed that according to both ASPRS and NSSDA standards the vertical accuracy is more stringent than the horizontal accuracy. For example if I produce orthophoto products from 15 cm (6 in.) digital imagery the stated ASPRS standard for horizontal accuracy is 30 cm (1 ft) while the expected vertical accuracy is 20 cm (0.67 ft). We always believed that the vertical accuracy of any mapping product is less stringent than the horizontal accuracy. Why is that?
Dr. Abdullah: PART II: In Part I of my answer (PE&RS August 2010) I addressed the issues that resulted in the contradictory accuracy figures that came into question. I explained that many of the map accuracy standards used today particularly here in the United States were derived from the use of film sensors and paper maps. At the conclusion of Part I I called on all concerned agencies and organizations in the United States to develop a new national standard that can be applied to modern geospatial data products. In Part II I would like to introduce some high level thoughts and ideas to generate discussions on how to create such a standard and I hope these ideas may even prove useful in the development of such a standard.
1. The new standard should be useful on a national level:
The standard should be accepted and endorsed by all those agencies and organizations that historically publish and maintain map standards in the United States such as ASPRS FGDC USACE FEMA and others. In addition the new standard should appeal to users from different sectors of the mapping and GIS community through its transparency and ease of use. When it comes to geospatial products a single standard can be used if it is drafted carefully and in a way that satisfies varying user requirements. Different agencies or users should be able to apply different accuracy figures to the same standard and still achieve results that are specific to their unique suite of products. This is easily achievable by matching products to specific accuracies based on product resolution or map class. I will elaborate further on this concept later in this article. Currently different agencies have already established or are in the process of establishing their own individual standards. For instance agencies such as FEMA ASPRS and the USGS have all published their own standards or guidelines for lidar data accuracy. Since lidar systems are based on the same fundamental laser technology raw products from different lidar systems all possess more or less the same quality and accuracy. Quality and accuracy are essentially determined by the methods used to post-process and handle the data; therefore users should have a single standard they can use to calculate accuracies that are specific to the methods being applied.
2. The new standard should be modular:
The old concept of “one sensor multiple products” no longer applies to today’s modern map-making practices. The diverse range of technologies currently used in map-making dictates the need for new standards that can be applied to new sensor technologies such as lidar (topographic lidar and bathymetric lidar) interferometric synthetic aperture radar (IFSAR and InSAR) digital cameras underwater survey by sonar etc. Therefore the standard should be modular in the sense that it should encompass a set of sub-standards that can be individually applied to different technologies. For instance one sub-standard may be used to define accuracies and specifications of products derived from imaging sensors. As a result this group of products (e.g. orthophoto compiled map and elevation data) would share the same vertical and horizontal accuracy requirements.
Another sub-standard might address the specifications and accuracy of lidar and IFSAR data and would define products such as elevation data and ortho-like intensity images; and additional substandards could be defined for hydro survey or acoustic survey using sonar technologies to map sea river and lake floors.
By developing a single standard that simply and uniquely addresses each sensor-type this modular approach eliminates the user’s confusion when trying to interpret multiple unrelated standards from multiple unrelated agencies. Modularity also lends well to change and expansion over time. Rather than becoming outdated and inapplicable over time this modular standard will change and adapt as new sensor technologies and products are added by the geospatial mapping community.
3. The new standard should apply one of the following two measures to classify the accuracies of final products:
a) Accuracy according to the resolution of the final delivered products
For example an orthophoto produced with 15 cm GSD should have a horizontal accuracy of RMSEX = RMSEY = 1.25*GSD (of the final delivered product) or 18.75 cm regardless of the sensor used or the flying altitude. The proposed accuracy figure is a little aggressive when compared with the current practice of assigning an ASPRS Class 1 accuracy of RMSEX = RMSEY = 30 cm for such a product. Vertical accuracy can be derived using a similar measure of RMSEV = 1.25*GSD (of the final delivered product) or 18.75 cm versus the current practice of labeling such products with an ASPRS Class 1 accuracy of RMSEv = 20 cm for 2 ft contour intervals.
The standard should not allow for the production of orthophotos with a GSD that is smaller than the raw imagery GSD (the GSD during acquisition). However the standard should allow for re-sampling of the raw imagery for the production of coarser orthophoto GSDs as long as the final accuracy figures are derived from the re-sampled GSD and not the native raw imagery GSD. Using the resolution or GSD of the imagery in referencing the final product accuracy introduces a more scientific and acceptable approach since a product’s accuracy is no longer based on the paper scale of a map.
One may argue that some users (e.g. a soldier on a battlefield) may need hard copy maps for field investigations. This is a valid concern. The new standard should allow users the option to produce paper maps using any scale they choose as long as the map accuracy is stated on the paper map and the scale is represented by a scale bar that automatically adjusts to the map scale.
b) Accuracy according to national map classes In this case the standard can specify multiple map categories for all users and the standard will provide specifications and accuracy figures to support each of these classes. The following proposed categories represent reasonable classes that should fit the needs of most if not all users:
1. Engineering class-I grade maps that require a horizontal accuracy of RMSEX = RMSEY = 10 cm or better and vertical accuracy of RMSEv = 10 cm
2. Engineering class-II grade maps that require a horizontal accuracy of RMSEX = RMSEY = 20 cm or better and vertical accuracy of RMSEv = 20 cm
3. Planning class-I grade maps that require a horizontal accuracy of RMSEX = RMSEY = 30 cm or better and vertical accuracy of RMSEv = 30 cm
4. Planning class-II grade maps that require a horizontal accuracy of RMSEX = RMSEY = 50 cm or better and vertical accuracy of RMSEv = 50 cm
5. General purpose grade maps that require a horizontal accuracy of RMSEX = RMSEY = 75 cm or better and vertical accuracy of RMSEv = 75 cm
6. User defined grade maps that do not fit into any of the previous five categories.
This concept provides more flexibility for data providers in designing and executing the project. However it may be problematic for users who are not well educated in relating map classes to product spatial resolution (GSD). Keep in mind that due to the fact that digital sensors are manufactured with different lenses and CCD array sizes different scenarios for image resolution and post spacing may result in the same final product accuracies and therefore it is important that users clearly define their required GSD or work with the vendor to determine the optimal GSD for their needs.
4. The new standard should address aerial triangulation sensor position and orientation accuracies:
Currently there is no national standard that addresses the accuracy of sensor position and orientation. As a result the subject has been left open to interpretation by users and data providers. The accuracy of direct or indirect sensor positioning and orientation (whether derived from aerial triangulation IMU or even lidar bore-sighting parameters) is a good measure to consider in determining the final accuracy of the derived products. Furthermore issues can be detected and mitigated prior to product delivery if the standard defines and helps govern sensor performance. In the past we adopted the rule that says aerial triangulation accuracy must be equal to RMSE = 1/10,000 of the flying altitude for Easting and Northing and 1/9,000 of the flying altitude for height. Obviously the preceding criteria were based on the then-popular large format film cameras that were equipped with 150 mm focal length lenses. Today’s digital sensors come with different lenses and are flown from different altitudes to achieve the same ground sampling distance (GSD) so relying only on the flying altitude to determine accuracy is no longer scientific or practical and new criteria needs to be developed.
When examining the 1/9,000 and 1/10,000 criteria the following accuracy figures apply for 1:7,200 scale imagery that is flown using a large format film metric camera. such as Leica RC-30 or Zeiss RMK to produce a 1:1,200 scale map:
RMSEX = RMSEY = 1/10,000*H = 1/10,000*1,100 = 0.11 m
RMSEZ = 1/9,000*H = 1/9,000*1,100 = 0.12 m
When using the current ASPRS class 1 standard the following accuracy figures would be expected for a map derived from the same imagery:
RMSEX = RMSEY = 0.30 m
RMSEZ = 0.20 m (assuming 0.60 m [2 ft] contours were generated from the imagery)
The previous accuracy figures call for aerial triangulation results that are 270% more accurate than the final map accuracy. Old photogrammetric processes and technologies required stringent accuracy requirements for aerial triangulation in order to guarantee the final map accuracy and past map production methods have transitioned through many different manual operations that ultimately resulted in the loss of accuracy.
Today’s map-making techniques have been replaced with all-digital processes that minimize the loss of accuracy throughout the entire map production cycle. In my opinion the new standard should support accuracy measurements for aerial triangulation based on the resulting GSD. Considering all of the advances we are witnessing in today’s map making processes aerial triangulation horizontal and vertical accuracy of 200% of the final map accuracy should be sufficient to meet the proposed map accuracy. Accordingly the aerial triangulation accuracy required to produce a map product with a final GSD of 0.15 m regardless of the flying height is shown below:
RMSEX = RMSEY = RMSEZ = 0.625*GSD = 0.625*0.15 = 0.09 m
(if the final map accuracy is based on RMSEX = RMSEY = RMSEZ = 1.25*GSD = 0.1875 m)\
Similar calculations can determine the required accuracy for direct orientation (no aerial triangulation required) using systems such as IMUs. To derive the required accuracy for raw pitch heading and position the previous aerial triangulation error budget of 0.09 m can be used to mathematically derive the acceptable errors in the IMUderived sensor position and orientation.
Lastly I feel that a new approach should be developed to calculate lidar orientation and bore sighting accuracies. Since the sensor’s geopositioning and not the laser ranging is the main contributor to the geometrical accuracy of lidar data this calculation should link lidar final accuracy to sensor orientation and positioning accuracies. In the forthcoming issue of PE&RS I will introduce the final part (Part III) of my answer which focuses on the importance for the new standard to deal with data derived from non-conventional modern mapping sensors such as lidar IFSAR and under water topographic survey using acoustic devices such as active SONAR (SOund Navigation And Ranging). In addition Part III will provide recommendations on the statistical methodology and confidence level to be used in the standard.
August 2010 (Download a PDF 183Kb)
Question: I noticed that according to both ASPRS and NSSDA standards the vertical accuracy is more stringent than the horizontal accuracy. For example if I produce orthophoto products from 15 cm (6 in.) digital imagery the stated ASPRS standard for horizontal accuracy is 30 cm (1 ft) while the expected vertical accuracy is 20 cm (0.67 ft). We always believed that the vertical accuracy of any mapping product is less stringent than the horizontal accuracy. Why is that?
Dr. Abdullah: I am glad you brought up this important issue concerning existing mapping standards and how they apply differently to imagery acquired by the new digital sensors. I would like to correct your understanding of the ASPRS and National Standard for Spatial Data Accuracy (NSSDA) standards as they relate to the example you’ve provided. The horizontal and vertical accuracies figures in the example are contradictory not because the ASPRS standard is stated incorrectly but because of the way we associate the image resolution or the Ground Sampling Distance (GSD) with the standard’s defined map scale or contour intervals.
When softcopy photogrammetry was introduced in the early 1990s it was standard practice to scan the film or the dispositive with 21 micron resolution or 1200 dpi (dots per inch). Therefore for a negative film scale of 1:7,200 (1”=600’) which is designed to support a map scale of 1:1,200 (1”=100’) according to 6x enlargement ratio the resulting Ground Sampling Distance (GSD) after scanning is 15 cm (6 in.). When we transitioned to digital aerial sensors which essentially replaced film cameras we maintained the same standards and conventions that we used for film products. As a result digital imagery flown with 15 cm GSD are routinely used for the production of 1:1,200 (1”=100’) scale maps or orthophotos and 2 ft contours. So the confusion actually originated when we adopted the old conventions for the new mapping products from digital cameras.
The ASPRS standard did not specify a certain GSD for a certain map scale but it did state that for class 1 mapping products a 1=1,200 scale map should meet a Root Mean Squares Error (RMSE) of 30 cm horizontally. Also the standard did not specify that imagery with 15 cm GSD had to be used for the production of 2 ft contours. The ASPRS standard states that the class 1 vertical accuracy for elevation data with 2 ft contour intervals must meet an RMSE of 20 cm; however when we extract accuracy figures for 15 cm imagery we use the above mentioned association of map scale and GSD to apply the ASPRS accuracy standard for evaluating the new digital sensor data products.
This is clearly a confusing situation that we created ourselves due to the lack of concise mapping standards for the highly accurate products produced from modern digital sensors. Immediate needs forced the mapping community to adapt conventions and measures that were originally designed for film cameras and paper-based products. The well known “enlargement ratio” which had been used in the past to determine how much film or dispositive could be enlarged to produce a final map with minimum or no degradation in quality is no longer applicable in today’s digital world of geospatial data production. An enlargement ratio of 6 was widely accepted and used in the mapping industry when dealing with film-based mapping products; however some of the modern digital sensors are built with diiferent CCD size (i.e. 6 microns versus the 14 or 21 microns of scanned films) and a variety of lenses and therefore the enlargement ratio becomes irrelevant when compared to film scanned at 21 microns. In fact the application of scale to digital imagery is not valid and only adds to the confusion particularly since the concepts of paper scale and enlargement ratio are based on film or paper-based maps. Again the contradicting accuracies represented in our original example are not derived from the ASPRS standard but result from our misconception that digital imagery with a GSD of 15 cm is only suitable to produce a 1=1,200 (1”=100’) scale map with 2 ft. contours.
The ASPRS mapping standard however is problematic when applied to data from digital sensors. The ASPRS standard materialized in the 1980s and was approved in the 1990s before digital sensors were used (or even existed) for mapping purposes. When we consider our level of achievement using today’s mapping processes the ASPRS standard is outdated and no longer suitable for further advancement of digital passive and active sensors and to support technologies such as GPS and IMU especially when the standard is based on mapping scale. Modern standards that are more suitable for digital maps and current and future technologies such as digital cameras lidar and IFSAR are needed to replace both the National Map Accuracy Standard (NMAS) and the ASPRS standard. A new set of standards should be developed based on the GSD of the digital data and the resolving power of the imaging sensor and not on scale since digital scale can vary from one user to another based on the zoom ratio used to evaluate the data. These same arguments are valid for the more modern standard published by the Federal Geographic Data Committee which is called the National Standard for Spatial Data Accuracy (NSSDA). The phrase “Accuracy Standard” in the NSSDA title is misleading and should be called “Testing Guidelines”. The term “standard test method” is defined by Wikipedia as follows: “to describe a definitive procedure which produces a test result. It may involve making a careful personal observation or conducting a highly technical measurement”. This definition does not apply to NSSDA since it does not quantify the testing threshold. To determine the final accuracies the NSSDA provided a statistical acceptance formula based on 95% confidence level without addressing the threshold (in this case the “RMSE”). Users typically derive an RMSE value in order to use the NSSDA. When users address the NSSDA we find they are often confused by these guidelines and misrepresent the standard in some way such as mislabeling requirements (i.e. 2 ft RMSE at 95%). This example statistically makes no sense since the term RMSE always refers to test results with a confidence level around 68% and not 95%. In my opinion the industry desperately needs to reform and consolidate all three standards – NMAS and ASPRS and NSSDA – into one single unambiguous national standard that clearly defines procedures and acceptance or rejection thresholds for the different mapping products. This effort requires constructive and focused cooperation between the ASPRS and the FGDC (which represents almost all federal agencies) to draft a standard that’s based on today’s knowledge practices and vision for the future. This effort should focus on developing sets of standards that will remain applicable over time and will not quickly become obsolete as today’s innovations and technologies rapidly progress. In the next issue of this column I will further discuss my ideas and thoughts on developing this standard as well as the different conditions and parameters on which it should be based.
July 2010 (Download a PDF 476Kb)
Question: What is a “bias” in mapping processing? Where does it come from? How is it calculated? How would one deal with it at different stages of the process?
This answer contains graphics and tables. Please see the PDF
April 2010 (Download a PDF 431Kb)
Question: Due to plate tectonics the Earth’s crust is moving at a rate of 5cm per year. What impact does this have on our GPS solutions and the accuracy of jobs that requires very high coordinate measurements?
This answer contains tables. Please see the PDF
March 2010 (Download a PDF 186Kb)
Question: My questions are about accuracy degradation of horizontal and vertical data during the photogrammetric process for airplane based platforms. I know that there are many variables involved but is there a relative constant multiplier that determines the loss of accuracy between ground survey and AT results as well as between AT results and final vector data and contours? Also can I assume digital and film cameras will result in different multipliers? Finally should the flying height be the sole determinant of the data accuracy?
This answer contains tables. Please see the PDF