HOME | ABOUT US | CONTACT | REGIONAL MANAGERS
 
  Research & Development  
 
  Experimental Products / Prototypes  
     •   BookletChart™ (Experimental)  
     •   Water Level and Current Forecast Guidance  
 
  Hydrographic Support  
     •   Autonomous Underwater Vehicles (AUV)  
     •   Remotely Operated Vehicles (ROV)  
     •   Phase Differencing Bathymetric Sonar  
     •   Sound Velocity Profiler  
     •   Tidal Corrector Application (TCARI)  
 
  Products and Database Development  
     •   Coast Pilot Relational Database  
     •   VDatum Enhancements  
     •   Coastal Digital Elevation Models  
     •   nowCOAST  
 
  Coastal & Estuarine Ocean Modeling  
     •   Forecast System Modeling  
     •   Inundation Modeling  
     •   Ecological Modeling Applications  
     •   Community Modeling & Standards  
     •   Model Evaluation Environment  
 
  Learn About  
     •   Hydrographic Survey Equipment  
     •   Datums and Transformations  
     •   How Hydrodynamic Models Are Used  
 
  Resources  
     •   Technical Reports & Publications  
     •   Standards and Requirements  
 
  Data Portals  
     •   nowCOAST: Real-Time Coastal Data Map Portal  
     •   Tides and Currents (General)  
     •   Physical Oceanographic Real-Time System (PORTS)  
     •   Operational Forecast Systems  
     •   Sea Nettle Forecast Guidance  
     •   North American Horizontal Datum Conversion Utility  
     •   University of New Hampshire, Joint Hydrographic Center  
 

Model Skill Assessment
Standards for Hydrodynamic Model Skill Assessment
The National Ocean Service (NOS) is developing and implementing oceanographic nowcast and forecast modeling systems to support navigational and environmental applications in U.S. coastal waters. These prediction systems provide NOAA users with nowcasts (i.e., analysis) and forecast guidance of water levels, currents, water temperature, and salinity for the next 24 to 36 hours.  NOS requires these modeling systems, whether developed within or outside NOS, be assessed for skill in adherence to NOS standards. Skill assessment is an objective measurement of how well the model nowcast or forecast guidance does when compared to observations. The Coast Survey Development Laboratory (CSDL) leads the effort to develop coastal ocean and estuarine nowcast/forecast modeling systems. Once these systems demonstrate sufficient skill they are transitioned to run operationally by NOS’ Center for Operational Oceanographic Products and Services (CO-OPS). Therefore, CSDL and CO-OPS have partnered in the development of standards to evaluate operational hydrodynamic modeling systems. Skill assessment statistics that quantify model performance are calculated from significant model variables (e.g., water levels and currents) according to prescribed routines. In order for a nowcast/forecast system to be accepted operationally the skill assessment statistics must meet prescribed targets as described in the standards documentation.

Hydrodynamic Model Skill Assessment Software
A skill assessment software package has been designed to perform skill assessment for water levels, currents, temperature, and salinity for different model systems in both tidally dominated and non-tidal regions. The skill assessment statistics (see Table 1) are computed automatically using data files containing observed, nowcast, and forecast variables. The observations, such as verified water levels, currents at NOS Physical Oceanographic Real Time System (PORTS) stations, and tidal constituents, can be directly acquired via the Internet from a CO-OPS database. Different types of data are processed and the skill assessment results are listed in tables valid at the selected verification stations. The package’s processing routines include tidal prediction, harmonic analysis, gap filling, filtering (or singular value decomposition), and other methods. The routines also include ways of concatenating nowcast and forecast guidance, and in extracting extrema.  This package can be run in Unix or Linux environments.  All Fortran programs can be compiled using Fortran compilers, version 77, or above. Documentation of the skill assessment software is available which describes the analysis and programs in further detail and is available for download.

Table 1.  Skill Assessment Statistics

Variable

Explanation

Error  The error is defined as the predicted value, p, minus the reference (observed or astronomical tide value, r : ei = pi - ri.


SM Series Mean

The mean value of a series y. 

Calculated as

Table 1.  Skill Assessment Statistics


RMSE

Root Mean Square Error
      
Calculated as

SD

Standard Deviation
 
Calculated as


CF(X)   Central Frequency. Fraction (percentage) of errors that lie within the limits +X.
POF(X)  Positive Outlier Frequency. Fraction (percentage) of errors that are greater than X.
NOF(X) Negative Outlier Frequency. Fraction (percentage) of errors that are less than -X.
MDPO(X) Maximum Duration of Positive Outliers. A positive outlier event is two or more consecutive occurrences of an error greater than X. MDPO is the length of time (based on the number of consecutive occurrences) of the longest event.
MDNO(X) Maximum Duration of Negative Outliers. A negative outlier event is two or more consecutive occurrences of an error less than -X. MDNO is the length of time (based on the number of consecutive occurrences) of the longest event.
WOF(X) Worst Case Outlier Frequency. Fraction (percentage) of errors that, given an error of magnitude exceeding X, either (1) the simulated value of water level is greater than the astronomical tide and the observed value is less than the astronomical tide, or (2) the simulated value of water level is less than the astronomical tide and the observed value is greater than the astronomical tide.






























References

Hess, K. W., T. F. Gross, R. A. Schmalz, J. G. W. Kelley, F. Aikman, III, E. Wei, and M. S. Vincent, 2003: NOS standards for evaluating operational nowcast and forecast hydrodynamic model systems, NOAA Technical Report NOS CS 17, 47 pp.

Zhang, A-J., K.W. Hess, E. Wei, and E. Myers, 2006: Implementation of model skill assessment software for water level and current in tidal regions, NOAA Technical Report NOS CS 24, 61 pp.

User Survey  | Privacy Policy  |  Disclaimer  |  NOAA's National Ocean Service  |  NOAA  |  U.S. Department of Commerce 
Web site owner: NOAA Office of Coast Survey