Marine & Environmental Sciences Faculty Reports

Document Type

Article

Publication Date

4-23-2009

Abstract

This report describes the methodologies, analyses, and results for an independent accuracy assessment of a thematic benthic habitat map produced by NOAA for the Florida Keys. It is an analysis of four regional accuracy assessments. Over the course of the Florida Keys mapping project, NOAA amended part of the classification scheme. The original scheme for mapping benthic cover was a tiered approach where certain benthic cover categories were given priority over others (e.g. coral was most important). Recently, this was modified to a dominant benthic cover scheme where the habitat is characterized by the single most dominant cover type and all habitats are characterized for percent cover of coral. The data and data analyses from Walker and Foster (2009 and 2010) were used to evaluate the accuracy of the reclassified map for Regions Of Interest (ROI) 1 and 2. New data were collected for ROIs 3 and 4 as part of this report. All four regions were combined and analyzed to determine total map accuracy.

Data were collected in January 2009 at ROI 1 (eastern Lower Keys), in June 2009 at ROI 2 (western Lower Keys), in September 2012 and February, March, and May 2013 at ROI 3 (back country), and in May 2013 at ROI 4 (Key Largo) (Figure 1). A total of 2029 sampling stations were visited, of which 1969 were used in the accuracy assessment. The sites were selected using a stratified random sampling protocol that equally distributed sampling points amongst the detailed structure categories. Most sites were sampled by deploying a weighted drop camera with the vessel drifting in idle and recording 30-120 seconds of dGPS-referenced video. The shallowest sites were sampled by snorkel, waverunner, or kayak, using a hand-held dGPS for navigation and a housed camera to record video. Each sampling station was given a Detailed Structure, Biological, and Coral Cover assignment in the field. These field classifications were reevaluated post-survey during a systematic review of video and photographic data, designed to ensure consistency within classifications. The efficacy of the benthic habitat map was assessed by a number of classification metrics derived from error matrices of the Major and Detailed levels of Geomorphological Structure and Biological Cover.

The overall, producer’s, and user’s accuracies were computed directly from the error matrices. The analyses of the combined ROIs 1 – 4 gave an overall accuracy of the benthic habitat map of 90.4% and 84.6% at the Major and Detailed levels of Structure respectively, and 85.1% and 76.5% at the Major and Detailed levels of cover. The known map proportions, i.e. relative areas of mapped classes, were used to remove the bias introduced to the producer’s and user’s accuracies by differential sampling intensity (points per unit area). The overall accuracy at the Major and Detailed levels of Structure changed to 92.3% and 85.9%. The overall accuracy at the Major and Detailed levels of cover changed to 84.3% and 79%. The overall accuracies were also adjusted to the number of map categories using the Tau coefficient. Tau is a measure of the improvement of the classification scheme over a random assignment of polygons to categories, bounded between -1 (0% overall accuracy for 2 map categories) and 1 (100% accuracy for any number of categories). The Tau coefficients were 0.807 ± 0.026 and 0.829 ± 0.018 at the Major and Detailed levels of Structure, and 0.814 ± 0.020 and 0.745 ± 0.020 at the Major and Detailed levels of cover.

Percent coral cover was classified for every polygon, thus coral cover was evaluated separately. Total accuracy for Coral in all habitats for all ROIs was 89.6% and 93.4% after adjusting for map marginal proportions. This calculation, however, was not realistic because it evaluated coral cover in non-coral habitat which inflated the number of correct sites. To account for this, coral cover was also evaluated at only those sites found to be Coral Reef and Hardbottom habitats. Total map accuracy for mapping coral cover on Coral Reef and Hardbottom habitats was 79.8%, and 82.7% after adjusting for habitat proportions. The accuracy varied greatly between the two coral categories present. User’s and Producer’s accuracies for Coral 0% - <10% were near or equal to 90%. Conversely, Coral 10% - <50% user’s and producer’s accuracies were 54.3% and 66.5% respectively. Adjusted producer’s accuracy was reduced to 55.2%. The adjustment for map proportions was very relevant here due to the large disparity of area between the two classes. The map contained 658.5 km² of Coral 0% - <10% and 39.8 km² of Coral 10% -<50%. Further 583 of AA points on Coral Reef and Hardbottom habitat were in Coral 0% - <10% and 219 were in Coral 10% - <50%. Interestingly, there were no mapped polygons of Coral 50% - <90% and 90% - 100%. There was confusion between coral classes where 88 locations mapped as Coral 10% - <50% were actually Coral 0% - <10% and 60 locations mapped as Coral 0% - <10% were found to be Coral 10% - <50%. Confusion between 11 locations that were mapped as Coral 10% - <50% were actually Coral 50% - <90% and 1 location mapped as Coral 10% - <50% was found to be Coral 90% - 100%. These sites were all located in the patch reefs of Hawk Channel. It is unknown if these sites met the minimum mapping unit criteria, but the field data indicated high coral cover at these locations. The relatively low adjusted producer’s accuracy for Coral 10% - <50% (55.2%) suggests that not all higher coral cover areas were captured in the map. Furthermore the relatively low user’s accuracy (54.3%) indicates that the areas of Coral 10% - <50% portrayed in the map are highly variable.

Combining all the results into a total map accuracy assessment gave a sense of how the overall map portrays the seascape. However, it should be noted that large gaps in map coverage exist, especially between Marathon and Key Largo, a 137 km stretch. The results given in the appendices are more representative of their specific regions. ROIs 1 and 2 covered most of the lower Keys and their results are a good representation of map accuracy for that region. ROI 3 covered the Backcountry which had higher accuracies, presumably due to a reduced diversity of habitats and lack of coral cover. ROI 4 is a good representation of the upper Keys map accuracy. It is difficult to know which assessment best represents the middle Keys. The landscape is more similar to the upper Keys, but Hawk Channel becomes deeper and more turbid.

Report Number

Final Report

Publication Title

Prepared for: Steven O. Rohmann, Ph.D., Office of National Marine Sanctuaries NOS/NOAA

Download

COinS