You are here

Distributed Data Access

NOAA National Operational Model Archive and Distribution System (NOMADS) utilizes the concept of distributed data access.

Functional Overview

The NOMADS framework is a distributed data system that promotes the combining of datasets between distant participants using open and common server software and methodologies. Users effectively access model and observational data and products in a flexible and efficient manner from archives or in real-time through existing Internet infrastructure.

The framework of the server and software used to manipulate the archive database and distribute the real-time data is the Open source Project for a Network Data Access Protocol (OPeNDAP) formally known as the Distributed Ocean Data System (DODS). OPeNDAP is a binary-level protocol designed for the transport of scientific data subsets over the Internet. Through OPeNDAP, applications can open and read subsets from remote datasets.

The Grid Analysis and Display System (GrADS) is an open source desktop tool that integrates the analysis and display of a wide variety geophysical data. OPeNDAP is an open source software project that provides a way to share data over the Internet. NOMADS combines these technologies to greatly expand the utility of the client and server concept. OPeNDAP-enabled clients such as GrADS, Ferret, IDL, Matlab, and end-user programs linked with various OPeNDAP client-libraries, can access datasets and subsets of datasets over the network from OPeNDAP servers. The server can perform calculations over large amounts of data and only transmit (fairly small) results back to the client. This avoids transmitting large amounts of data to the client.

"Downloading data from the OPeNDAP server to your local system is generally a misuse of the OPeNDAP server. The OPeNDAP data server is like a library in the purist sense of the term. When you check out a book from the library, do you photocopy the entire book, return it, and then read your copy? It is difficult to overcome the habit of working with data that you don't actually have on your own disk. The capability to access and use gigabytes of binary data over the Internet has not existed before. It does not make sense for terabytes of disk across many institutes to be redundantly occupied by the same data that are freely available over the net."

Getting Started with OpeNDAP

An OPeNDAP Quick Start Guide

Using a Web Browser with OPeNDAP

Using a Spreadsheet Application with OPeNDAP

Desktop Services ("Clients")

For data users, the currently available OPeNDAP servers include data formats for many desktop applications, and the user base for OPeNDAP enabled clients is expanding. NOMADS users have multiple options to access data from their desktops. The OPeNDAP website maintains a list of OPeNDAP-enabled analysis applications and libraries.

With a certain level of understanding of the OPeNDAP protocol, a user can also use a web browser or even a spreadsheet to access OPeNDAP data.

OPeNDAP Servers

A data provider can range from a single scientist and a laptop computer, to a major data center. To make data available under the NOMADS framework, a provider must install an OPeNDAP Server as described on the OPeNDAP website.