National Environmental Satellite, Data, and Information Service Home Page. National Oceanic and Atmospheric Administration Home Page. National Climatic Data Center Home Page. Department of Commerce Home Page. NOAA, National Environmental Satellite, Data, and Information ServiceNational Climatic Data Center, U.S. Department of Commerce
NOAA's Weather and Climate Toolkit

Weather and Climate Toolkit - Batch Processing

The Weather and Climate Toolkit (WCT) may be used for batch processing by following the instructions below. All options are controlled with an XML configuration file.

Data Exporter Instructions

XML Configuration File

Batch Processing Tips

Batch/Command-Line Map Creation

WCT Exporter Instructions:

Top of page

1) Download the WCT distribution:

Current Stable release: wct-2.4.2 .zip

2) Unzip the WCT distribution: 
         wct-2.4.2
.zip
          |
          |-- wct -- dist      (Main distribution libraries)
               |---- lib       (Supporting libraries)
               |---- wct       (Unix/Mac script)
               |---- wct.bat   (Windows script)
               |---- wctBatchConfig.xml  (Example config file)
3) Set Environment Variables: (Optional, but recommended)
Set the WCT_HOME environment variable to the unzipped 'wct' folder location.
For example: C:\Utilities\wct (on Windows) or /home/steve/bin/wct (Unix)
For more information on setting environment variables, please refer to documentation on your specific operating system or Unix shell.

If the WCT_HOME environment variable is not set, you must run the 'wct' command from the wct installation directory.

4) Write or modify the XML configuration file:
This file is described in detail below.

5) Run 'wct' from command line (DOS, Terminal or Unix Shell):
From the command-line or script, run 'wrt.bat' or 'wrt.sh' with the following arguments:

wct <in> <out> <format> <config>
where:
  • <in> = Input URL, filename or directory. If directory, then all NEXRAD files in directory are processed. A URL must be for a specific file, not a directory.
  • <out> = Output file or directory for processed files. If a directory, then the output files use the input filename. The directory must already exist or it will be treated as a file.
  • <format> = Output format (use one of the following abbreviations). shp=Shapefile, wkt=Well-Known Text, asc=Arc/Info ASCII Grid, flt=Arc/Info Binary Grid, grd=GrADS Binary Grid, nc=NetCDF, tif=GeoTIFF, csv=Comma-Separated Text File (Alphanumeric-Only), rnc=Raw NetCDF
  • <config> = XML configuration file for export options. This may be a file or URL.

6) List file options...
The Exporter may also be run with a list of input files and destination directories:

wct <listfile> <format>
where:
  • <listfile> = Listfile with format of infile/dir/url,outfile/dir,config-xml-file/url.
    Example Listfile
  • <format> = Output format (use one of the following abbreviations). shp=Shapefile, wkt=Well-Known Text, asc=Arc/Info ASCII Grid, flt=Arc/Info Binary Grid, grd=GrADS Binary Grid, nc=NetCDF, tif=GeoTIFF, csv=Comma-Separated Text File (Alphanumeric-Only), rnc=Raw NetCDF

7) Example command line arguments:

a) Download latest N1R product from the National Weather Service (NWS) and save as a Shapefile using the specified config file.

<in> = http://weather.noaa.gov/pub/SL.us008001/DF.of/DC.radar/DS.p19r1/SI.kgsp/sn.last
<out> = D:\Nexrad_Viewer_Test\Batch\output
<format> = shp
<config> = D:\Nexrad_Viewer_Test\Batch\wctBatchConfig.xml


b) Process all files in C:\ViewerData\HAS999900001\ to NetCDF using the specified config file.

<in> = C:\ViewerData\HAS999900001\
<out> = D:\Nexrad_Viewer_Test\Batch\output
<format> = nc
<config> = D:\Nexrad_Viewer_Test\Batch\wctBatchConfig.xml


c) Process all files in listfile to GeoTIFF using the specified config files.

<listfile> = D:\Nexrad_Viewer_Test\Batch\listfile.txt
<format> = tif

Top of page

Weather and Climate Toolkit XML Configuration File


Download example configuration file: wctBatchConfig.xml
There is documentation inside this file explaining the various options and configurations.

Top of page

Batch Processing Tips


1. Output Redirection


'>' will redirect all output (standard out and standard error)
'1>' will redirect standard out
'2>' will redirect standard error

To redirect to nowhere (disable) the output, use 'NUL' on Windows and '/dev/null' on Linux and Mac.

Examples:
  1. Ignore all output:
    Windows: [ command arguments ] >NUL
    Linux: [ command arguments ] >/dev/null

  2. Ignore standard output and send error messages to screen:
    Windows: [ command arguments ] 1>NUL
    Linux: [ command arguments ] 1>/dev/null

  3. Ignore standard output and send error messages to file:
    Windows: [ command arguments ] 1>NUL 2>errors.log
    Linux: [ command arguments ] 1>/dev/null 2>errors.log



Top of page

Batch/Command-Line Map Creation


1. Choose a command-line GIS package to generate maps

Example Open-Source packages include FWTools/Mapserver/GDAL, NCL and R
Example Proprietary packages include Matlab and IDL.


2. Example with FWTools/Mapserver/GDAL

Using the Mapserver utiltiy 'shp2img', you can create complex map images with a single command-line call and a Mapserver configuration file (map file).

Check out the example Unix script and Mapserver Map File to see how the image below was created.

The states and county shapefiles were downloaded from the NOAA/NWS AWIPS Shapefile Database. The map background is the NASA Blue Marble image downloaded from a Web Map Service (WMS).

Image of example map

Top of page

Please contact Steve Ansari ( Steve.Ansari@noaa.gov ) if you have specific questions or comments about this software.