Warning Decision Support System -- Integrated Information

Automated algorithms, tools and displays for analysis, diagnosis and forecasts of severe weather phenomena       www.wdssii.org


Real-time Data

Discussion Forum

For developers
Obtaining WDSS-II


Getting started using WDSS-II for research

Note that this page deals only with research use. The free version of WDSS-II has restricted use. If you wish to use WDSS-II in real-time, you should license a version from the University of Oklahoma.
Currently, we have versions of the software for Linux 2.4 (such as Fedora Core 1 and Red Hat Enterprise 3) and Linux 2.6 (such as Fedora Core 2 and Red Hat Enterprise 4). The WDSS-II tar file you downloaded contains tools and algorithms for interacting with, and building weather data sets. For more information on WDSS-II, please see this webpage.
  1. Software Installation
  2. Obtaining radar data
  3. A note on data formats
  4. A note on using scripts
  5. Processing WSR-88D data
  6. A note on rearranging index files.
  7. Creating cartesian lat-lon grids from data from one or more radars
  8. Lightning data
  9. Visualization


To install the software, follow these steps:
  1. Download the software to your local machine.
  2. Create a directory and untar the file there:
    • mkdir /home/username/WDSS2
    • cd /home/username/WDSS2
    • tar xvfz <name-of-downloaded-file>
  3. You should have three directories:
    • bin: this contains all the algorithms and tools that you can run
    • lib: this contains all the libraries to which the binaries are linked.
    • w2config: this contains configuration information for the software.
  4. Add the binary directory to your PATH. You will add a line like:
    • export PATH=/home/username/WDSS2/bin:$PATH
  5. Add the library directory to your LD_LIBRARY_PATH. You will add a line like:
    • export LD_LIBRARY_PATH=/home/username/WDSS2/lib:$LD_LIBRARY_PATH
  6. Set an environment variable called W2_CONFIG_LOCATION to specify a path for WDSS-II to search. You will add a line like:
    • export W2_CONFIG_LOCATION=/home/username/w2config:/home/username/WDSS2/w2config
  7. Make sure that all the libraries referenced by the display program exist. See display installation instructions for more details.

Obtaining Radar Data

You can download radar data from the web at NCDC.  Download, and uncompress the data. You will be able to do this if you use the command tar xvf filename.tar and then use the command uncompress *.Z . These data are in Level-II format. To save space, you may want to compress the files with gzip * or bzip2 * -- WDSS-II tools can read data compressed with gzip or bzip2 directly.

The Level-II radar files can be converted to netcdf using the tool 'ldm2netcdf' that is part of the WDSS-II distribution. See the section on "Creating Netcdf Products from Level-II files" for details.

Note on Data Formats

The gridded data produced by the WDSS-II tools are all in NetCDF format (See data format documentation ). You can read these files into Matlab, C, Java, Fortran, Perl, Python, etc. You can find a list of these interfaces at Unidata's NetCDF page . Remember that the files are probably gzipped. Unzip them using "gunzip name-of-file" before attempting to use these NetCDF interfaces. Also, WDSS-II by default stores data in a "sparse-grid format" where the data are run-length encoded. General-purpose netcdf tools may not be able to handle this. You can tell WDSS-II to not use sparse-grids by setting a flag in the file w2config/misc/dataformat. The WDSS-II display will automatically uncompress on the fly, so you don't have to run gunzip to view the data using wg.

You can view the structure of a NetCDF file using the tool "ncdump.pl": The single-radar data are mostly in polar format. The multi-radar data are mostly in lat-lon-grid format.

Scripts for research projects

It is very likely that in the course of a project, you would need to download and prepare a number of data cases by running several algorithms on it. In such cases, it is better to have a simple script that will prepare the data case for you. Here is an example of a script for running algorithms. Follow the instructions in the script itself on how to run it, and how to modify things.
The script in this case was written for a student to study whether cleaned up reflectivity data (cleaned up using a neural network) improved the performance of two algorithms -- w2segmotion and netssap. To do that, we need to generate both the raw data and the edited data, then run the two algorithms on both data streams. That is what the script does.
  1. converts the original radar data to netcdf and dealiases the velocity
  2. converts the LB (real-time queue) to an archive case (xml file). This is required so that the algorithms that follow will stop and exit. If they read data from a LB, they will never exit since they will be in real-time streaming mode.
  3. runs the neural net to clean up the reflectivity data and produce ReflectivityQC and ReflectivityQComposite
  4. runs w2vil to generate the ReflectivityMaximum product
  5. runs segmotion and netssap on the unedited stream and puts the output in a subdirectory called "unedited"
  6. runs segmotion and netssap on the quality-controlled stream and puts the output in a subdirectory called "qc"
Edit this script to perform the tasks you need to do for your project.

Creating Netcdf Products from Level-II files (processing WSR-88D radar data)

Level-II to unedited Netcdf files

Run the program ldm2netcdf

Getting dealiased velocity data

To get edited velocity data (required if you need shear/rotation products, etc.), you will also need to add a (-d) option when you run ldm2netcdf.

Getting cleaned up reflectivity data

To get cleaned up reflectivity data, you will need to run the algorithm w2qcnn (qc = quality control).

Creating Azimuthal Shear and Rotation Products

To create azimuthal shear and rotation products, you need to run off the dealiased (edited) velocity products using a program called w2circ.

Running the Single-Radar Severe Storms Analysis Package (SSAP)

The Severe Storms Analysis Package (SSAP) includes the following algorithms that will run on the radar data:
  1. Storm-cell identification and tracking (SCIT, also called the Cell Table)
  2. Hail Detection Algorithm (HDA)
  3. Mesocyclone Detection Algorithm (MDA)
  4. Tornado Detection Algorithm (TDA)
To run the SSAP on a single-radar case, you need to have some configuration files:
  1. cd WDSS2
  2. mkdir netssapdat
  3. cd netssapdat
  4. Download ssapdat.tgz into that directory.
  5. Untar the file (tar xvfz ssapdat.tgz)
  6. You should have a number of .dat files
  7. Whenever you run "netssap" you need to be in this directory.
Whenever you wish to run netssap, move (cd) to the WDSS2/netssapdat directory, and type the command "netssap" to get a list of options.

Converting code_index.fam into a XML file (So algorithms can exit)

Database description

In the XML database, the "selections" tag provides a unique name for each piece of data. The "params" tag provides the location of the netcdf file that the data can be found in. Since the XML file is sorted, you can use it as the input to your program if you need to get a list of data files.

Creating a Lat-Lon-grid of radar data

You can combine data from one or more radars and grid the data into a 3D lat-lon-height grid. To do that, you will use the tool w2merger.  If you are combining data from more than one radar, you will have to feed the data into the merger using a synchronizing mechanism called w2simulator (you don't need to do this if you are creating a grid of single-radar data).

You will also need to pre-create a cache for the combination of the radars and grid dimensions using createCache. This way, the actual combination will be at real-time speeds. You don't have to do this if you are processing an archive case from a single radar -- the cache will be created on demand. Just be aware that the very first time (depending on the size of your grid), the merger will take much longer than usual. The next time you run things on the same grid, it should be faster.

To summarize, if you are doing a multi-radar case, you need to use w2simulator and createCache . You also need to run w2simulator, w2merger and w2segmotionll simultaneously.

createCache -- pre-create computations for radar/grid combinations

w2simulator -- synchronization

To run archived data through a multi-radar process, the files on disk have to fed to the process in "real-time".

Merging the data

Motion Estimates (to feed into the merger, optional)

Lightning Data

WDSS-II has several tools for operating with lightning data:

Visualizing Weather Data

To visualize weather data, use the display program "wg". wg will work only if you have a graphics card on your workstation and have OpenGL enabled.
To use wg, you need to have the glib2 RPMs for GTK2 installed. To see if they already exist on your machine, cd to the WDSS2/bin directory and type "ldd wg". Any library in that list that is not found, you need to find and install. For example, you may find that you need whichever RPM provides these libraries for your distro:
  1. libgtk-x11-2.0.so.0
  2. libgdk-x11-2.0.so.0
  3. libgdk_pixbuf-2.0.so.0
Please find out the exact libraries and RPMs by doing "ldd wg".
Follow the display documentation link to get help on using the display.
Remember, that for research, you can visualize netcdf files using other software as well.

Questions. Email Kiel Ortega