Partnering with Amazon Web Services on Big Data

NEXRAD visualization of Hurrican Katrina's landfall in 2005

This visualization, created with NEXRAD Level II data, depicts Hurricane Katrina as it made landfall in 2005.

On October 27, 2015, Amazon Web Services announced the public release of NOAA’s Next-Generation Weather Radar, or NEXRAD, data for anyone to use. Through cooperative activities as a part of NOAA’s Big Data Project, the real-time feed and full historical archive of original resolution (Level II) NEXRAD data, from June 1991 to present, will be freely available through the company’s cloud infrastructure.

NCEI, in partnership with the National Weather Service, preserves and provides access to the over 270 terabytes and 180 million files of historical data archived in the magnetic tape archive at its headquarters in Asheville, North Carolina. Providing a copy of these data to the cloud enhances their availability, promotes new business opportunities, and makes them more valuable for the entire Nation.

This Big Data Project combines NOAA’s tremendous volume of high-quality environmental data and advanced data products, private industry’s vast infrastructure and technical capacity, and the American economy’s innovation and energy to expand access to our data resources for both private industry and the public. NOAA designed this collaboration to create a level playing field in which everyone has equal opportunity to access and use the data.

Ranked as the Nation’s second most valuable geospatial dataset by the National Science and Technology Council, providing fast, efficient, and easy access to NEXRAD data is of great importance. However, the unwieldy size and limitations associated with retrieving the data from tapes made the full dataset very difficult to access and transmit from NCEI. Over several months, NCEI, with our partners the Cooperative Institute for Climate and Satellites–North Carolina, transmitted the entire dataset to Amazon Web Services.

With a copy of NEXRAD data available via Amazon Web Services, users can now access the entire dataset in one place without the need for special arrangements. Users no longer need to move the data across networks or maintain their own computer power to store it. And, users can access processing, analysis, and visualization resources on Amazon Web Services along with the data storage infrastructure. Overall, the move gives more users equal access to the data and allows them to easily develop comprehensive statistical and probabilistic products while still maintaining NCEI’s role as the official archive of the NEXRAD data.

For more information, see: