National Oceanic and
Atmospheric Administration
United States Department of Commerce

Science Data Integration

image of wave glider and data stream

The PMEL Science Data Integration Group (SDIG) is devoted to data management, data integration management and its related software development. This group has developed core technologies that are used by various groups in NOAA as well as entities outside NOAA to manage, access and visualize scientific data assets. Other tools and frameworks, such as the Environmental Research Division's Data Access Program (ERDDAP), supplement the in-house developed software. This group supports data integration for PMEL’s scientific efforts and NOAA-wide with software development, and guidance on data management.

Some of the core capabilities developed by the SDIG are:

  • Ferret is an interactive computer visualization and analysis environment designed to meet the needs of oceanographers and meteorologists analyzing large and complex gridded data sets. It runs on most Unix systems, and on Windows XP/NT/9x using X windows for display. It can transparently access extensive remote Internet data sources using OPeNDAP (formerly known as DODS). Ferret is widely used in the oceanographic community to analyze data and create publication quality graphics.  Recently, ferret has been extended to support python by creating PyFerret.  PyFerret is a python module that is wrapped around traditional Ferret.   PyFerret allows for enhanced graphics and access to computational libraries, while maintaining Ferret’s advanced capabilities to retrieve, analyze, manipulate and save data.  
  • The Live Access Server (LAS) is a specialized Web Application framework for displaying and analyzing Earth Science data, but even so it is built using conventional Web application technologies. The LAS is a highly configurable web server designed to provide flexible access to geo-referenced scientific data. It can present distributed data sets as a unified virtual database through the use of DODS networking. Ferret is the default visualization application used by LAS, though other applications (Matlab, IDL, GrADS, ...) can also be used.
  • The myDashboard data submission tool introduces an intelligent, streamlined data submission process to national data archives such as NOAA’s National Ocean Data Center (NODC) and the Department of Energy’s (DOE) Carbon Dioxide Information and Analysis Center (CDIAC). The web-based application also performs quality control on the data before submission and allows the data owner to place the data in a quarantined state until the data are ready for submission to a national archive for public access.

Other than software development, this group’s members are involved as key members in the  ESGF Compute working Team, the Observing System Monitoring Center (OSMC) and the Unified Access Framework (UAF) efforts.