Divisions and Committees
PAD, PDAD AND LIDAR JOINT COMMITTEE FOR
MAP ACCURACY STANDARD UPDATES
UPDATED DRAFT FOR COMMENT AND REVIEW
An initial review by ASPRS members took place in October 2013. Based on those comments, a revised draft was developed and published in the December issue of PE&RS to encourage wide dissemination and comment. That draft is posted below. The comment period on the revised draft closed February 1, 2014. All comments are being reviewed with the intent of finalizing a revised document for ASPRS Board review and feedback at the 2014 ASPRS Spring Conference in Louissville. The new version will be posted after the internal committee review is complete.
PDF Format: Draft_ASPRS_Accuracy_Standards_for_Digital_Geospatial_Data_PE&RS.pdf
Word Format: Draft_ASPRS_Accuracy_Standards_for_Digital_Geospatial_Data_V15_PE&RS.docx
(*NOTE: Word .docx format has track changes enabled)
NOTE: REVISIONS ARE PENDING AND WILL BE POSTED PRIOR TO THE 2014 SPRING CONFERENCE
Submit questions to: AccuracyStandard@asprs.org
The public comment period has closed. However, if you have issues you wish to discuss, or input you with to provide, please contact the committee chair at the above e-mail for an update on the current status.
In the summer of 2011, the Photogrammetric Applications Division (PAD) and Primary Data Acquisition Divisions (PDAD) held a series conference calls with the intent of forming a committee to update and revise the existing ASPRS Map Accuracy Standards for Large Scale Maps.The existing standard is primarily intended for published maps and has several shortcomings when applied to new digital technologies. Currently, there is no consistent and appropriate accuracy standard that applies specifically to new technologies for digital geospatial data.
At the November 2011 ASPRS Pecora Conference in Washington DC a meeting was held to present a draft concept and initiate the effort to update and revise the existing accuracy standard. The initial draft concept outline for the standard was presented by Dr. Qassim Abdullah. This outline was based on discussions during the initial teleconferences as well as Dr. Abdullah's extensive past work on the subject through his PE&RS Mapping Matters column. Several conference calls were held to assimilate more information, identify, discuss and resolve key issues.A Hot Topic session was presented in Sacramento (Spring, 2012) to solicit additional feedback from the membership. During this session, an updated accuracy table was presented by Dr. Dave Maune, based on the concepts outlined in the initial draft concept and additional work Dr. Maune was doing related to an update of the US Army Corps of Engineers engineering manual sections on mapping guidelines and standards.This table was further revised over the summer and an updated version was presented at the 2012 Fall conference in Tampa Bay. After the 2012 fall conference, a subcomittee chaired by Dr. Maune and including Dr. Abdullah and Karl Heidemann, worked to develop a complete working draft of a new standard. The initial draft was reviewed by the overall committee. Additional comments, modifications and contributions were incorporated into the current version, which is now being submitted for review and comment by the overall ASPRS membership.
- Initial e-mail comment and review by ASPRS membership (Comment period closed Oct. 14, 2013)
- Revised draft published in PE&RS for full public review: Dec. 2013
- Comments requested by February 1, 2014 to allow development of a final version prior to the 2014 Spring Conference.
- Special Session presentation followed by discussion: ASPRS Conference in Louisville, Kentucky; Tuesday, March 25, 2014 3:30 p.m. to 5:00 p.m.
- Final document for Board Approval: Spring/Summer 2014, pending final comments and feedback received at the Spring Conference.
Current Comments and Background Materials
COMMENTS, NOTES AND BACKGROUND MATERIALS
(Page Last Updated: 10/14/2013)
Existing Standard: ASPRS Accuracy Standards for Large-Scale Maps, 1990
Existing Standard: ASPRS Guidelines, Vertical Accuracy Reporting for Lidar Data, 2004
FGDC Reporting Standard: National Standard for Spatial Data Accuracy (NSSDA), 1998
Initial Draft Concept Outline for New Standard (Pecora 2011)
CURRENT COMMENTS (REVIEW PERIOD ENDING 10/14/2013)
1) General comments
Add a glossary defining acronyms and key terms
Distinguish between thematic accuracy and spatial accuracy; possibly address this in the title
Be consistent in usage of 1.96 and 2.0 for computing the 95% range of errors (Update 12/2013: This was addressed in the revised draft)
Add examples showing conversion of horizontal accuracy to other units.
2) Detailed Technical Comments (attachments)
KEY ISSUES FROM PAST MEETINGS AND DISCUSSIONS
(This is not a comprehensive list of all issues discussed. The document provides detailed background information and discussion in support the assumptions made and final approaches selected therein. This discussion is only intended to provide background clarification for a few key issues that received significant discussion and debate in the course of developing the draft standard and for any additional comments that have not yet been addressed).
- Map scale, ground sample distance (GSD) and pixel size: Ground sample distance (or the distance on the ground represented by one pixel in the digital image) is used to establish accuracy thresholds for digital orthophotography. In the case of an orthophoto, the GSD, or pixel size, is a representation similar to the final published scale of the map and as such is an appropriate metric to use for establishing orthophoto accuracy thresholds. This is not true for planimetric data. In the case of planimetric data, there are numerous variables that affect the final accuracy. These include the quality of the aerotriangulation solution, the type of camera used, the method used for digitizing as well as the point spacing for collection used based on the intended final published scale of the map. As such, planimetric data accuracies cannot be directly characterized by the GSD of the source imagery. While it is true that digital data can be displayed or printed at any scale, in the development of the data, the degree of generalization and other factors dictate the design or target map scale for which the data is intended to be used. The data can certainly be viewed or printed at scales that exceed that, but doing so does not improve the accuracy or level of generalization. For this reason the accuracies in this standard are based on a map scale factor which is related to the design map scale, ie. the scale for which the data was designed to be viewed and printed. The GSD that would be appropriate given current, large scale photogrammetric mapping cameras is provided for general reference only.
- NSSDA equations in relating RMSE to 95% confidence levels: The NSSDA equations that use RMSE to compute the error range at a 95% confidence level are only statistically correct under the very restrictive condition that the mean error equals zero. This condition rarely occurs. Further, the RMSE statistics are only valid for normally distributed data. However, in many common applications, empirical results indicate that if the mean error is small, and the data is normally distributed, the NSSDA equations can represent a reasonable approximation of the error range at a 95% confidence level. As the mean error increases, the approximation is less accurate and tends to overestimate the range of errors. This is not well documented in the NSSDA standard. In addition, RMSE by itself does not fully characterize other aspects of the data set accuracies. As such, RMSE is applicable only for normally distributed data sets with all systematic errors removed. Ensuring that the data set meets these requirements requires careful evaluation of the other statistical parameters.
- Approach to Using RMSE in New Standard: RMSE is used in the existing standard and is a long established, well understood and widely used parameter for estimating geospatial data accuracies for normally distributed data sets. Further, as a simple to use, single parameter threshold, RMSE appropriately characterizes the absolute accuracy of the data set (as opposed to the relative accuracy or precision about the mean). For these reasons, the new standard continues to use RMSE as the accuracy threshold for normally distributed data sets. 95% values that correlate to the NSSDA reporting standard are listed in the table as reference. Clarifying text was added to indicate the limitations of that relationship and to explain the necessity of evaluating other statistical parameters to ensure that the data set has had all systematic errors removed and meets conditions for normally distributed data. Further, for lidar accuracies in vegetated areas (which are known to be biased and not normally distributed), thresholds are based on the 95th percentile accuracies and do not use RMSE values. The current standard was intended to provide a simple to use and straight-forward threshold for the most common data sets. It does not preclude future modules or addendums that address the more complex case of data sets that do not meet these criteria.
- Modular Standard: Several comments to date have indicated that other modules may be needed. These include: Assessment of linear data; rigorous total propagated uncertainty (TPU) models for our products (as opposed to ground truthing against independent data sources); more detailed statistics that do not rely on the assumption of normally distributed data; and image quality factors (such as edge definition and other characteristics). The current standard is intended to be the base standard needed to replace the existing standard for Large Scale Maps and to meet the immediate need of better addressing current digital technologies. Additional modules should be pursued and can be added by subject matter experts in these fields as they are developed.
- Published data set vs. source data points: The standard is replacing the existing "map accuracy standard" and as such applies to values interpolated from the final data sets. The standard is not necessarily evaluating the system accuracy at discrete source points such as lidar returns or digitized points. Elevation accuracies are assessed as interpolated from a TIN generated from the final digital elevation model. Planimetric accuracies are measured at well-defined and readily identifiable features.
- Spot Elevations: Higher accuracy spot elevation points are not specified by the new standard. Spot elevations were primarily used on cartographic contour maps, published at a fixed map scale, to aid in the accurate interpolation of elevations at key locations between drawn or interpolated contours. With current GIS, DTM and lidar technologies spot elevation points tied to a specific contour interval are less relevant. The new standard uses a single accuracy threshold to specify the accuracy of elevations interpolated from the source terrain model and moves away from specifying accuracies in terms of contour intervals.
More stringent that past standards:The new accuracy classes represent a more stringent standard than the existing 1990 standard as newer technologies can support the higher accuracies. The primary focus was on higher accuracies. Some comments indicated that particularly for orthophoto imagery, additional classes may need to be added, or some of the higher classes may need to be relaxed to address applications where less stringent accuracies are required. (Update 12/2013: The V15 draft as published in PE&RS addressed earlier comments regarding the accuracy classes by relaxing the standards for Class II, Class III and subsquent classes. Class I accuracies remained at the higher engineering grade or best accuracy standard while Class II became the "normal" standard typical of most projects and consistent with the prior standard).