Divisions and Committees
PAD, PDAD AND LIDAR JOINT COMMITTEE FOR
MAP ACCURACY STANDARD UPDATES
CURRENT DRAFT FOR COMMENT AND REVIEW (OCTOBER 15, 2014)
The comment period for Revision 5 closed September 15, 2014.
Revision 6 of the proposed ASPRS Positional Accuracy Standards for Digital Geospatial data as published below is currently available for review. This version incorporates comments received during the prior review period and is being provided for additional review prior to submital to the ASPRS Board of Directors at their Fall Meeting (Pecora conference) in November, 2014. The most significant changes included revising horizontal accuracy classes to be independent of scale and pixel size; additional details on how to choose swath to swath accuracy check points; revisions to discussion on interpolation of elevations from elevation data sets.
Please refer to the Comments and Background for a summary of key issues and discussion to date.
Word Format: ASPRS_Positional_Accuracy_Standards_for_Digital_Geospatial_Data_Draft_Rev6_V1.docx
Word Format (includes highlights of changes since Revision 5): ASPRS_Positional_Accuracy_Standards_for_Digital_Geospatial_Data_Draft_Rev6_V1_Highlighted.docx
Please submit feedback and questions to: AccuracyStandard@asprs.org
Comments are requested no later than Oct. 31st, 2014
PRIOR VERSION FOR REFERENCE ONLY (VERSION 5, REV. 1, JULY 2014)
PDF Format: ASPRS_Positional_Accuracy_Standards_for_Digital_Geospatial_Data_Draft_Rev5_V1.pdf
Word Format (.docx with track changes enabled): ASPRS_Positional_Accuracy_Standards_for_Digital_Geospatial_Data_Draft_Rev5_V1.docx
COMMENT AND REVIEW HISTORY
The current draft is the sixth revision of the standard. An initial draft was reviewed by ASPRS members in October 2013. This was followed by a revised draft which addressed some, though not all of the comments received during the October review. That revision was published in the December issue of PE&RS to encourage wide dissemination and comment. The comment period for review of the PE&RS version closed February 1, 2014. A third revision was presented at the ASPRS 2014 Spring conference in Louisville. Revision 4 incorporated Board of Director's and ASPRS member feedback from the spring conference and was then submitted for internal review. Revision 5, addressed remaining issues and was intended to be a final draft submitted for final public review prior to Board Approval. While in line with all prior comment, Revision 6 included a significant revision to how horizontal accuracies are presented. Numerical associations did not change. Revision 6 was primarily intended for internal review by those who had provided prior feedback and concerns; however, anyone is welcome to provide comments.
In the summer of 2011, the Photogrammetric Applications Division (PAD) and Primary Data Acquisition Divisions (PDAD) held a series conference calls with the intent of forming a committee to update and revise the existing ASPRS Map Accuracy Standards for Large Scale Maps. The existing standard is primarily intended for published maps and has several shortcomings when applied to new digital technologies. Currently, there is no consistent and appropriate accuracy standard that applies specifically to new technologies for digital geospatial data.
At the November 2011 ASPRS Pecora Conference in Washington DC a meeting was held to present a draft concept and initiate the effort to update and revise the existing accuracy standard. The initial draft concept outline for the standard was presented by Dr. Qassim Abdullah. This outline was based on discussions during the initial teleconferences as well as Dr. Abdullah's extensive past work on the subject through his PE&RS Mapping Matters column. Several conference calls were held to assimilate more information, identify, discuss and resolve key issues.A Hot Topic session was presented in Sacramento (Spring, 2012) to solicit additional feedback from the membership. During this session, an updated accuracy table was presented by Dr. Dave Maune, based on the concepts outlined in the initial draft concept and additional work Dr. Maune was doing related to an update of the US Army Corps of Engineers engineering manual sections on mapping guidelines and standards.This table was further revised over the summer and an updated version was presented at the 2012 Fall conference in Tampa Bay. After the 2012 fall conference, a subcommittee chaired by Dr. Maune and including Dr. Abdullah and Karl Heidemann, worked to develop a complete working draft of a new standard. The initial draft was reviewed by the overall committee. Additional comments, modifications and contributions were incorporated into the current version, which is now being submitted for review and comment by the overall ASPRS membership.
ADDITIONAL BACKGROUND MATERIALS
The version of the standard initially published in PE&RS in December 2013 is provided here as a reference. While several significant updates and revisions have been made since, this version is useful as a reference as it provides more extentisive background information and represents the core effort on which the final versions are based.
- August, 2011 - Initial effort initiated
- November, 2011 (Pecora Conference) - Initial outline proposed
- Spring/Fall 2012 - Meetings were held in Sacramento and Tampa Bay; further discussion ensued
- Winter 2012/Spring 2013 - A drafting committee was formed and a preliminary draft developed to address what were viewed as the core elements of the standard, recognizing that some of the needs identified in earlier meetings would need to be completed as future modules or additions.
- September/October, 2013 - Initial e-mail comment and review by ASPRS membership (Comment period closed Oct. 14, 2013)
- December, 2013 - Revised draft published in PE&RS for full public review (comment period closed February 1, 2014)
- February/March 2014 - Revised and restructured draft to meet ASPRS standards templates requirements, address all comments and incorporate additional work related to horizontal accuracies of elevation data and low confidence area development, published March 21, 2014
- March 25, 2014 - Special Session presentation followed by discussion: ASPRS Conference in Louisville, Kentucky; Tuesday, March 25, 2014 3:30 p.m. to 5:00 p.m.
- June 30, 2014 - Final revisions made and approved through internal committee review.
- September/October 2014 - Target dates for final public review, adjudication of comments and final submital to the Board for approval
Current Comments and Background Materials
COMMENTS, NOTES AND BACKGROUND MATERIALS
(Page Last Updated: 9/8/2014)
Existing Standard: ASPRS Accuracy Standards for Large-Scale Maps, 1990
Existing Standard: ASPRS Guidelines, Vertical Accuracy Reporting for Lidar Data, 2004
FGDC Reporting Standard: National Standard for Spatial Data Accuracy (NSSDA), 1998
Initial Draft Concept Outline for New Standard (Pecora 2011)
1) DETAILED TECHNICAL COMMENTS (ATTACHMENTS)
Comments 2013-12-04 HCS.docx
TomAsbeck_Low Confidence Area Polygons.docx
Note: Document markups received during the current review period will be posted at the close of the comment period.
2) COMMENT SUMMARY
ReviewerComments_Updated_1015_2014_Anonymous.xls (Excel spreadsheet summarizing comments received and actions taken)
3) KEY ISSUES FROM PAST MEETINGS, DISCUSSIONS AND DELIBERATION
This is not a comprehensive list of all issues discussed. The document provides detailed background information and discussion in support the assumptions made and final approaches selected therein. Additional background detail is also included in the first draft narrative version of the standard, as published in PE&RS in December 2013. The link to that document is posted on this web site. The discussion here is intended to provide background clarification for those selected key issues that received significant discussion and debate in the course of developing the draft standard.
Map scale, ground sample distance (GSD) and pixel size:There has been continued debate regarding how to assign accuracy classes to digital orthophotography and planimetric data,
Revision 6 presents accuracies that are independent of scale and pixel size. Associations to scale and pixel size, as related to both legacy data and current technologies, are now presented as guidelines in the Annex tables and not presented as standards.
Initial proposals utilized the GSD of the source imagery (ie. the distance on the ground represented by one pixel in the digital image). This was problematic since the accuracy of the final data depends on many factors other than the acquisition GSD. These include the sensor itself, ground control accuracies, direct georeferencing or aerotriangulation methods used, and compilation or rectification methods.
For digital orthophotography, a distinction was made between the GSD of the source image and the GSD of the orthophoto. GSD was used to refer to the raw image resolution; “pixel size” is used to refer to the resolution of the rectified orthophoto.
For planimetric data, the issue was more complicated. With digital data, there is no “published” map scale as the data can be printed or viewed at any zoom resolution. However, all planimetric data does have a specific resolution or target scale for which it was designed to support. This is determined by the resolution of the source imagery, compilation methodologies (ie. point spacing and the level of detail digizited) and other factors. While the data can certainly be viewed or printed at scales that exceed the intended target scale, doing so does not improve the accuracy or level of generalization. For this reason prior revisions based accuracies on a map scale factor which is defined as the design map scale, or the scale for which the data was designed to be viewed and printed. The Map Scale Factor was introduced to facilitate a simple formula for computing accuracy thresholds at any scale. This solution was workable for projects requiring scaled planimetric maps (whether digital or plotted), though still problematic for horizontal data not tied to scale or pixel size.
NSSDA equations in relating RMSE to 95% confidence levels:The NSSDA equations that use RMSE to compute the error range at a 95% confidence level are only statistically correct under the very restrictive condition that the mean error equals zero. This condition rarely occurs. Further, the RMSE statistics are only valid for normally distributed data. However, in many common applications, empirical results indicate that if the mean error is small, and the data is normally distributed, the NSSDA equations can represent a reasonable approximation of the error range at a 95% confidence level. As the mean error increases, the approximation is less accurate and tends to overestimate the range of errors. This is not well documented in the NSSDA standard. In addition, RMSE by itself does not fully characterize other aspects of the data set accuracies. As such, RMSE is applicable only for normally distributed data sets with all systematic errors removed. Ensuring that the data set meets these requirements requires careful evaluation of the other statistical parameters.
Approach to Using RMSE in New Standard: RMSE is used in the existing standard and is a long established, well understood and widely used parameter for estimating geospatial data accuracies for normally distributed data sets. Further, as a simple to use, single parameter threshold, RMSE appropriately characterizes the absolute accuracy of the data set (as opposed to the relative accuracy or precision about the mean). For these reasons, the new standard continues to use RMSE as the accuracy threshold for normally distributed data sets. 95% values that correlate to the NSSDA reporting standard are listed in the table as reference. Clarifying text was added to indicate the limitations of that relationship and to explain the necessity of evaluating other statistical parameters to ensure that the data set has had all systematic errors removed and meets conditions for normally distributed data. Further, for lidar accuracies in vegetated areas (which are known to be biased and not normally distributed), thresholds are based on the 95th percentile accuracies and do not use RMSE values. The current standard was intended to provide simple to use and straight-forward thresholds for the most common data sets. It does not preclude future modules or addendum that address the more complex case of data sets that do not meet these criteria. In fact all efforts were made to facilitate incorporating this work when/if it is pursued.
Modular Standard: Several comments to date have indicated that other modules may be needed. These include: Assessment of linear data; rigorous total propagated uncertainty (TPU) models for our products (as opposed to ground truthing against independent data sources); more detailed statistics that do not rely on the assumption of normally distributed data; and image quality factors (such as edge definition and other characteristics). The current standard is intended to be the base standard needed to replace the existing standard for Large Scale Maps and to meet the immediate need of better addressing current digital technologies. Additional modules should be pursued and can be added by subject matter experts in these fields as they are developed.
Published data set vs. source data points:The standard is replacing the existing "map accuracy standard" and as such applies to values interpolated from the final data sets. The standard is not necessarily evaluating the system accuracy at discrete source points such as lidar returns or digitized points. Elevation accuracies are assessed as interpolated from a TIN generated from the final digital elevation model. Planimetric accuracies are measured at well-defined and readily identifiable features.
Spot Elevations: Higher accuracy spot elevation points are not specified by the new standard. Spot elevations were primarily used on cartographic contour maps, published at a fixed map scale, to aid in the accurate interpolation of elevations at key locations between drawn or interpolated contours. With current GIS, DTM and lidar technologies spot elevation points tied to a specific contour interval are less relevant. The new standard uses a single accuracy threshold to specify the accuracy of elevations interpolated from the source terrain model and moves away from specifying accuracies in terms of contour intervals.
More stringent that past standards: New technologies can achieve higher accuracies than required by prior standards. This standard faciliates more stringent accuracies than the existing 1990 standard.
Early comments indicated that, particularly for orthophoto imagery, additional classes may need to be added, or some of the higher classes may need to be relaxed to address applications where less stringent accuracies are required. Significant objections and concerns were raised with regards to proposed structure used in Revision 3 of the standard (March 2014) presented at the Spring Conference. That version listed Class 1 as the highest accuracy class appropriate for new technologies and very stringent project design; Class 2 corresponded to a typical high accuracy mapping project and corresponded to Class 1 in the old standard. The concern was that agencies and users would have difficulty adjusting to specifying Class 2 for the majority of their work, when Class 1 was the standard for so many years. To address this, Revision 5 introduced a Class 0 accuracy. In that version (Revision 5, July 2014), Class 0 represents the highest accuracy class; Class 1 corresponds to the Class 1 standard from the prior standard which is in widespread use and is still applicable for most high accuracy mapping projects.
Revision 6 solves this issue by abandoning discrete classes and referencing accuracy classes to the RMSEx and RMSEy thresholds they represent. This provides 100% flexibility for users and data providers to establish the most appropriate accuracy class for the application, given the very wide variation that now occurs in sensors, approaches and other critical factors. Annex B tables provide guidelines to choose the appropriate RMSE accurcy class as associated to both legacy data and prior standards as well as for what can be achieved with new technology.