Tool validation

Wetland Prioritization Study Main Page

 

Systematic field confirmation

  • Montana Natural Heritage Program
  • Michigan Tech Research Institute
  • USACE Sunrise River Watershed-Based Mitigation Pilot
  • USGS Forest Breeding Bird Decision Support Tool
  • Wisconsin Department of Natural Resources

 

Correlation analysis

  • Colorado Natural Heritage Program
  • VIMS Wetland Condition Assessment Tool

 


Systematic field confirmation

 

Montana Natural Heritage Program (MTNHP):1 MTNHP has validated the Landscape Integrity Model (MTLIM) approach in a recent project to identify wetlands in Minimally Disturbed Condition in the Rocky Mountain West. When followed by visual inspection of aerial photos to identify stressors that a GIS might have missed (e.g., recent development, wildfire, timber harvested, new roads), the model has proved to be very accurate for these purposes. When used across all condition classes, the model is less reliable. For example, in the original model development, MTNHP validated the MTLIM by comparing MTLIM-predicted categorical ratings with actual rapid assessment categorical ratings to three different datasets with mediocre results.

  1. An independent sample of 180 wetlands drawn from a dataset of 1800 mapped palustrine and lacustrine wetlands throughout Montana, with 60 randomly selected and 120 selected specifically to incorporate a variety of environmental and disturbance gradients and unmapped areas found that 51% of ranks were correctly predicted.
  2. The entire dataset of 1800 palustrine and lacustrine wetlands.
  3. 108 assessments completed using the Ecological Integrity Assessments (EIA), a rapid field-based assessment developed by MTNHP found that only 55.5% of ranks were correctly predicted.

Michigan Tech Research Institute (MTRI):2,3,4 Michigan Department of Transportation (MDOT) staff evaluated the accuracy of the Wetland Mitigation Site Suitability Tool ( WMSST) by comparing its results for 20 sites against those obtained for the same sites by MDOT using field monitoring data. This process was completed in two steps:

  1. For ten sites, MTRI analysts followed workflow procedures that MDOT developed for the WMSST to report site suitability rankings (high, medium, low). MTRI then provided these rankings to MDOT staff, which compared them to rankings it obtained for each site based on evaluations of previously-collected field monitoring data.
  2. For the other ten sites, MTRI and MDOT staff followed MDOT's workflow procedures collaboratively to obtain site suitability rankings, which were then compared to rankings MDOT obtained for each site through an evaluation of field monitoring data.

The WMSST correctly assessed wetland suitability for 19 of the 20 sites. For the one site that had been classified incorrectly, MTRI and MDOT determined the cause of the error by applying the step in MDOT's workflow procedures that directs users to explore WMSST input and visualization data. Following this step, MTRI and MDOT determined the USGS SSURGO hydric soil classification for the site had been incorrect. They determined that had the classification been correct, the site's suitability would have been modeled correctly.

Overall, the validation study demonstrated that the WMSST could produce substantial savings for MDOT, reducing the costs required for evaluating potential mitigation sites by 73%.

USACE Sunrise River Watershed-Based Mitigation Pilot:5 The Corps used rapid field surveys (more rapid than traditional rapid assessment methods) to validate two sites within each of ten subwatersheds of the Sunrise River watershed. These sites were all identified as high priorities by the Spatial Decision Support System ( SDSS) and represented a wide variety of size classes. The sites evaluated were generally representative of wetlands within the watershed, though they may not have represented a statistically ideal sample. Results from the validation exercise showed that priorities identified by the model generally matched field observations.

USGS Forest Breeding Bird Decision Support Model:6 Over the last five years, agency personnel from the Fish and Wildlife Service (FWS) and U.S. Geological Service (USGS), and state conservation agencies have conducted time- and distance-based avian point count field assessments to validate the Mississippi Alluvial Valley (MAV) model results. These field assessments identify presence/absence and bird densities for particular bird species within different forest blocks.

Wisconsin Department of Natural Resources (WDNR):7,8 For the Milwaukee river basin assessment, WDNR used random stratified sampling to test for errors of commission and omission in the potentially restorable wetland (PRW) layer for three watersheds. WDNR staff visited randomly selected points within PRWs and within non-PRWs to assess the accuracy of the PRW layer at each point. WDNR found the accuracy of the tool to be "very acceptable," exceeding 80% in the three watersheds. In addition, WDNR validated this PRW layer by conducting an extensive survey of PRW polygons in five high priority subwatersheds, testing for errors of omission. WDNR recruited county field staff from local conservation agencies with expertise in wetland restoration to adjust boundaries of PRW polygons as necessary and provide an evaluation of the technical feasibility of restoration within each PRW area. These county staff evaluated all possible sites, accessing private property when possible with the permission of landowners.

Results showed that error varied widely among subwatersheds, but the most frequent error was PRW polygons that were actually still wetlands. Reason for error included land use change, reversion of drained hydric soil back to wetlands, and errors of omission in the version of the Wisconsin Wetland Inventory available to the project. The Wisconsin Wetland Inventory has since been updated based on 2005 aerial photography for the counties in the basin and is now being updated based on 2010 aerial photography.

The Habitat Quality Index tool's output for 3 of the umbrella species in the original Milwaukee river basin assessment was tested against known occurrences of these species. Results were considered acceptable for 2 of the 3 species.

 

Back to top


Correlation analysis

 

Colorado Natural Heritage Program (CNHP):9,10 For two river basins, Landscape Integrity Model (LIM) results were correlated with the results of field-based rapid assessment methods that included the Human Disturbance Index (HDI), Ecological Integrity Assessment (EIA), and Mean C assessment. Because each correlation was strong, CNHP concluded that the LIM model was an effective tool for the assessment of landscape integrity, though weights on input stressors may need adjustments. CNHP did not find a strong correlation between LIM results and intensive data obtained using the Vegetation Index of Biotic Integrity (VIBI), but the VIBI method is also under development and was only applied to a handful of sites. An example of the correlations used by CNHP to validate its LIM model is shown below, which relates LIM scores with EIA results. Overall, CNHP found that field-based methods demonstrated a similar pattern of stressors as obtained using the LIM, with wetlands at lower elevations (e.g., marshes and saline wetlands) having lower landscape integrity scores and wetlands at higher elevations (e.g., fens and riparian shrublands) having higher scores.

 

Virginia Institute of Marine Science (VIMS) Wetland Condition Assessment Tool:11 VIMS obtained acoustic signatures of wetland wildlife (captured by sound recording devices) to directly sample habitat provision at 27 sites throughout the coastal plain. These measurements served as the basis of an 'analysis of similarity' that was used to validate the ability of landscape prioritization land cover scores and rapid assessment stressor counts to predict habitat quality. In addition, VIMS used Pearson correlation to demonstrate the relationship between land use metrics (e.g., percent pasture, percent rowcrops) and water quality measures (total dissolved nitrogen, total suspended sediment, incision ratio).

 

Back to top

Wetland Prioritization Study Main Page


1Vance LK. 2009. Assessing Wetland Condition with GIS: A Landscape Integrity Model for Montana. A Report to The Montana Department of Environmental Quality and The Environmental Protection Agency. Montana Natural Heritage Program, Helena, MT. 23 pp. plus appendices.
2 Michigan Tech Research Institute. 2009. Validation Report: Wetland Mitigation Site Suitability Tool. Accessed from:http://quickplace.mtri.org/QuickPlace/tarut/PageLibrary8525724B004F2A59.nsf/h_Toc/BA5057037CC37873852575B60045FF3C/?OpenDocument
3 
Brooks C, Powell R, Shuchman R, Leonard G. Developing and applying a geospatial decision support tool for efficient identification of wetlands mitigation sites.
4 Michigan Tech Research Institute. 2009. Validation Report: Wetland Mitigation Site Suitability Tool. Accessed from:http://quickplace.mtri.org/QuickPlace/tarut/PageLibrary8525724B004F2A59.nsf/h_Toc/BA5057037CC37873852575B60045FF3C/?OpenDocument
5 
Interview in 12/2011 with Tim Smith, Enforcement and Compliance Coordinator, U.S. Army Corps of Engineers, St. Paul District.
6 Interview on 8/1/2011 with Daniel Twedt, Wildlife Biologist, United States Geological Survey.
7 Kline J, Bernthal T, Burzynski M, Barrett K. 2006. Milwaukee River Basin Wetland Assessment Project: Developing Decision Support Tools for Effective Planning.
8 Bernthal T, Kline J, Burzynski M, Barrett K. 2007. Milwaukee River Basin Wetland Assessment Project: Phase Two: Groundtruthing the Potentially Restorable Wetlands Layer.
9 Colorado Natural Heritage Program. 2011. Statewide Strategies to Improve Effectiveness in Protecting and Restoring Colorado's Wetland Resource.
10 Feedback provided on 5/16/2012 by Joanna Lemly, Wetland Ecologist, Colorado Natural Heritage Program.
11 Center for Coastal Resources Management, Virginia Institute of Marine Science, College of William and Mary. 2007. Development of a nontidal inventory and monitoring strategy for Virginia - completion of phase II (coastal plain and piedmont physiographic provinces): Final report to the Environmental Protection Agency Region III.