VAPOR-WRF-NCL
Using NCL with VAPOR to
Visualize WRF-ARW data
January 2012
Web links updated April 2013
Version 2.1
Introduction
This document is a tutorial on the combined use of two NCAR-developed visualization tools, VAPOR (Visualization and Analysis Platform for atmospheric, Oceanic, and solar Research) and NCL (NCAR Command Language). NCL is used to construct 2D data plots from WRF data, and then these 2D plots can be inserted in an appropriate location in a VAPOR 3D scene of the WRF data.
A basic familiarity with NCL and VAPOR may be helpful to the reader, but not required for this tutorial.
This tutorial makes use of a WRF-ARW simulation of typhoon Jangmi, a typhoon with sustained winds of 160mph, passing over the Philippines, Taiwan, and China, during September and October 2008. To limit the size of the data to download for this exercise, we focus on the first 10 hours of the day of September 28, when the typhoon passed over Taiwan.
Several related VAPOR documents may be useful with this tutorial:
?The VAPOR/WRF Data and Image Preparation Guide. Sections 1-3 of this guide provide background information on the procedures of this tutorial.
?VAPOR User's Guide for WRF Typhoon Research shows how to use VAPOR to visualize the same dataset (typhoon Jangmi) that is used in this tutorial.
?The document Visualization of WRF Data using VAPOR: A Georgia Weather Case Study is a tutorial that shows how to use VAPOR to visualize WRF data, including
images such as those generated here.
The reader should follow the following 5 steps of image preparation and usage. Each step requires completion of the preceding steps.
1.Obtaining the needed data and software for this tutorial.
2.Converting the WRF output data to a VAPOR dataset.
3.Creating horizontal and vertical plots of the WRF output data using NCL. These images
are initially produced as postscript files; then, using the NCL script “wrf2geotiff.ncl”, the output images are converted to a geo-referenced tiff (geo-tiff) file.
4.Obtaining a geo-referenced terrain image for a region containing the WRF simulation
output. This uses the VAPOR shell script “getWMSImage.sh”.
5.Inserting the NCL plots and the terrain image into a 3D scene, and visualize the images
together with the WRF 3D weather data in VAPOR.
At the end of this document is provided, for reference, an additional section, namely the Appendix, containing the NCL scripts used in section 3.
1 Prepare software and data
To work through these examples you will need the following:
1.VAPOR 1.5 or later
Vapor installers can be obtained from the VAPOR download site,
https://www.wendangku.net/doc/0713918712.html,/page/vapor-download#Binary. In most cases the binary
installers work well; it is not necessary to compile VAPOR. Installation instructions are at https://www.wendangku.net/doc/0713918712.html,/docs/vapor-installation/vapor-installation . It is best to
install VAPOR on a computer that has a good graphics card; in particular, most nVidia
and ATI cards work well with VAPOR.
2.NCL version 5.1 or later: If NCL is not already installed on your system, follow the
download and installation instructions provided at: https://www.wendangku.net/doc/0713918712.html,/Download/
NOTE: You must have the file “.hluresfile” install ed in your home directory, for these
examples to work properly. You can download an example of this file from
https://www.wendangku.net/doc/0713918712.html,/Document/Graphics/hlures.shtml.
3.WRF output data files of typhoon Jangmi on September 28, 2008. This data was
provided by Dr. Bill Kuo and Dr. Wei Wang of NCAR, and Dr. Minsu Joh of KISTI.
These WRF output files are for 10 hours of the day that typhoon Jangmi passed over
Taiwan. The domain is a moving D02 domain nest that tracks the typhoon. These WRF output files are provided as either a zip file or as a gzip/tar file. The WRF output files
have been renamed without colons (:) in the name (the trailing :00:00 has been removed from the name) because Windows does not allow colons in file names.
Create a new directory to hold these WRF output files. Download and uncompress one of the following files. (Note that you will need about 850MB of storage to hold the
unzipped data files; the files to download are about 450MB. Shorter versions
(containing only two time steps, around 80MB) are also available for users lacking
sufficient storage space.):
?Ten time-step files: jangmiWrfout.tar.gz and jangmiWrfout.zip
?Shorter two time-step files: jangmiWrfout_small.tar.gz and
jangmiWrfout_small.zip
4.Two additional (UNIX) utilities are needed; check to be sure they are available on your
system. If you are using Windows, these must be available in the Cygwin-X
environment:
?convert (from ImageMagick , version 6.2 or later is best)
?psplit (this comes with NCL)
5.The three NCL scripts we use are included in this document, and versions of these are
also available at the NCL/WRF website. Download each of the following scripts to the same directory where you have put the WRF output files:
?wrf_Height.ncl (this plots humidity, temperature, pressure and wind at a fixed
elevation)
?wrf_Precip.ncl (this plots precipitation tendency with pressure isobars)
?wrf_crossSection2.ncl (this is a vertical plot of relative humidity) Note that numerous other NCL scripts that generate plots of WRF data are available at
the NCL/WRF website; we have chosen the above three to illustrate various options for using NCL with VAPOR. The other NCL scripts on the WRF/NCL examples page can similarly be converted to produce plots that display in VAPOR. In the VAPOR
examples/NCL directory, five such converted example NCL scripts are provided.
2 (Optional) Convert the WRF output data to a VAPOR
dataset
Starting with VAPOR 2.0, it is no longer necessary to convert WRF output files to VAPOR data format: the VAPOR user interface (vaporgui) can read WRF output files directly. If your dataset is very large then it is still useful to perform the data conversion, because a Vapor Data Collection (VDC) will perform better on such large data; however, for the
purposes of this tutorial you may skip the conversion step, and, rather than load the VAPOR data, you can import the WRF output files directly.
We do provide the data conversion here, as an optional exercise as follows: This conversion is performed in two steps. The conversion requires that you have already installed VAPOR.
Before the commands are run on UNIX platforms, you should ensure VAPOR has been set up in your current shell, by sourcing the command vapor-setup.csh (for c-shell) or vapor-setup.sh (for the Bourne shell)
1.Run wrfvdfcreate. This application scans the WRF output files, and constructs a
VAPOR metadata file that describes the dataset. It also provides some useful
information about the coordinate extents of the WRF domain. For this exercise, the
default options suffice. Other options are described in the wrfvdfcreate man pages.
CD to the directory in which you have downloaded and unzipped the WRF output
files. Then issue the following command:
wrfvdfcreate wrfout_d02_2008-09-28* jangmi-09-28.vdf On completi on of this command you will see that the file “jangmi-09-28.vdf” has
been created. The console output of this command will look something like the
following:
Console output resulting from running wrfvdfcreate on typhoon data
Note the latitude and longitude extents of the data; these will be useful later when obtaining a terrain image for this domain.
2.Run wrf2vdf: This application reads all the 2D and 3D variables of the WRF output
files, and converts them the format that VAPOR uses, namely a VDC (Vapor Data
Collection).
Issue the command:
wrf2vdf jangmi-09-28.vdf wrfout_d02_2008-09-28*
The command output will list all the variables and time steps that are converted.
This command will take a minute or so to complete. On completion, the VDC
data is in a subdirectory jangmi-09-28_data of the current directory. The
jangmi-09-28_data directory will contain a subdirectory for each variable,
representing each WRF variable in a multi-resolution (wavelet) form.
The first few lines of the console output of this command are shown below.
The first few lines of the console output resulting from running wrf2vdf
3 Make NCL scripts to plot data as georeferenced tiff images
We start with three different NCL scripts as provided on the WRF/NCL Web page, and
illustrate the changes that are necessary to convert these scripts, with sufficient detail that the user should be able to apply the same steps to other NCL scripts to obtain geo-referenced output images. The three scripts are: wrf_Height.ncl, wrf_Precip.ncl, and
wrf_CrossSection2.ncl. These three scripts were chosen because the resulting images are
appropriate for three different kinds of display in VAPOR:
?The wrf_Height.ncl script produces an image that presents the data on a particular horizontal plan. The resulting images can appropriately be positioned in the VAPOR scene at the height of that plane.
?The wrf_Precip.ncl script produces a plot that presents precipitation tendencies and sea-level pressure. This plot can appropriately be positioned on any horizontal plane
in the VAPOR scene, or can be applied to the terrain surface.
?The wrf_CrossSection2.ncl script produces a vertical plot of temperature and relative humidity along a particular plane, parallel to the X-Z axis. The resulting images
should be positioned to coincide with the specified vertical plane in the VAPOR scene. Each of these three scripts reads in a WRF dataset and produces a number of NCL plots to a console window (as X11 output). We shall convert these to produce Postscript (.ps) output files, and the Postscript output will then be converted to georeferenced TIFF files. In the Appendix we provide three versions of each of these scripts: The initial version (obtained from the WRF/NCL page), a first modification (customized to produce a series of plots, exactly one per timestep, from the sample WRF data set), and a final version that produces a georeferenced TIFF file.
The conversion process involves the following steps:
1.Modify the script so that it generates only one image at each time step. (If you want to
use multiple images at the same time steps then you need to run a separate script for each of the different images)
2.Modify this script so that it will loop over all time steps in the WRF files. This may
involve producing an outer loop that loops over several files, and an inner loop that
handles the time steps in each file.
3.Modify the script so that the output is a postscript (.ps) file
4.Insert the following new lines into your NCL script:
o Near the top, insert a line to load wrf2geotiff.ncl (This script and other NCL examples are by default installed in the subdirectory
share/vapor-x.x.x/examples/NCL, of the $(VAPOR_HOME)
directory where vapor is installed, where “x.x.x” should be replaced by “1.5.2” or
whatever version of VAPOR is being used.
(N.B. for Windows users: On Windows, the file wrf2geotiff.ncl is installed in
$(VAPOR_HOME)/share/examples/NCL/. However, NCL does not
allow blanks in the file path for the NCL load instruction. To ensure there are no
blanks in the path, copy the wrf2geotiff.ncl file to your current directory, and then
change the load instruction to:
load “wrf2geotiff.ncl”
o After the NCL workstation “wks” is created, call the function
wrf2geotiff_create(wks). In these examples we name the returned
opaque object “wrf2gtiff”.
o If this is a vertical plot, disable georeferencing by calling:
wrf2geotiff_disableGeoTags(wrf2gtiff)
o Set the plot resource pltres@gsnFrame=False. This enables manual control of the frame advance.
o Each time a plot is created, insert two lines:
Call wrf2geotiff_write(), to cause the plot to be written to file. The
final argument to wrf2geotiff_write() determines whether or not the image will be
cropped.
Call frame(wks) to advance the frame.
o At the end of the script, insert a call to wrf2geotiff_close(), which will cause all the images to be combined into one geo-tiff file with appropriate geo-
referencing.
5.When you run this new script, make sure you execute it from a shell where you have
already sourced vapor_setup.csh or vapor_setup.sh , to set up the correct execution
environment. The result will be one georeferenced tiff file with all the images (one for
each time step), ready to be loaded into a VAPOR 3D scene.
3.1 Conversion of wrf_Height.ncl
We follow the above steps in some detail using the first example script, wrf_Height.ncl.
For the other two scripts, we just illustrate the changes that are made for this conversion.
First, make sure that your environment is set up to run the wrf_Height.ncl script. To see that the wrf_Height.ncl script operates correctly, find the “addfile” command at line 14 in the script, and edit it by providing the path to the first WRF file, as follows:
a = addfile(“wrfout_d02_2008-09-28_00.nc”,”r”)
Note that we assume that the script is in the same directory as the wrfout file. If that is not the case, you need to include the path with the name of the file in the above line. Issue the command:
ncl wrf_Height.ncl
You should see an image of humidity, temperature, pressure, and wind, at elevation 0.25 km, followed by an image at 2.0 km.
Next, we modify this script so that it produces one image at each time step, and at a higher elevation: i.e. we make an image of humidity, temperature, pressure and wind at 5.0km.
Edit the loop over levels, by replacing
do level = 0, nlevels-1 ; LOOP OVER LEVELS
with:
level = 1
change the line (above the TIME LOOP):
height_levels = (/250., 2000./)
to
height_levels = (/250., 5000./)
Remove the line:
end do ; END OF LEVEL LOOP
Again run the script. You should see only one image, a plot at height 5km.
To make a plot at all ten times, we must modify the script to loop over the various WRF output files. Replace the line that we previously edited:
a = addfile(“wrfout_d02_2008-09-28_00.nc”,”r”)
with the following lines:
wrffiles = systemfunc(“ls wrfout_d02_2008-09-28*”)
numFiles = dimsizes(wrffiles)
do i = 0, numFiles-1
wrffiles(i) = wrffiles(i) +”.nc”
end do
inpFiles = addfiles(wrffiles,”r”)
These lines will create an array of filenames (“inpFiles”), one for each wrf output file. To loop over these filenames, insert the following two lines:
do ifile = 0, numFiles-1
a = inpFiles[ifile]
Immediately above the line
times = wrf_user_list_times(a) ;get times in the file
Also, to terminate this loop over files, insert the line:
end do ;END OF FILE LOOP
right after the line:
end do ;END OF TIME LOOP
Note that the original script was producing images every other time step. To get an image for each time step, change the line:
do it = 0, ntimes-1, 2 ; TIME LOOP
do it = 0, ntimes-1 ; TIME LOOP
(This won’t affect our current plot s, because there is only one time step per file in the Jangmi dataset; however this change would be needed for datasets with multiple times in a file). Again execute the command:
ncl wrf_Height.ncl
and you will see ten images, one for each of the ten time steps in the WRF data.
This file is saved as “wrf_Height_FirstMod.ncl” in the Appendix for your reference. The third of the resulting images is the following:
Currently the wrf_Height.ncl script produces X11 graphics to your screen. We next modify it so that it produces a geo-referenced tiff file as output. This requires the following changes
to the script:
At or near the top, insert as one line:
load “$VAPOR_HOME/share/examples/NCL/wrf2geotiff.ncl”
(Windows users should change the load instruction to:
load “wrf2geotiff.ncl”
and copy the file wrf2geotiff.ncl from “$VAPOR_HOME/share/examples/NCL/ into the current directory, as discussed above.)
The above line loads the library of NCL commands that will be used later in the file.
wrf2geotiff.ncl is an auxiliary library that is shipped in the NCL examples directory of the VAPOR distribution.
Next, comment out the following line (precede with a semicolon “;”):
Type = “x11”
And un-comment (remove the preceding “;”):
; type = “ps”
This makes the output a postscript file instead of x11. Postscript files are needed for the conversion to tiff.
After the line
wks = gsn_open_wks(type,”plt_HeightLevel”)
Insert the line:
wrf2gtiff = wrf2geotiff_open(wks)
This creates an opaque pointer that is used to track the geotiff capture process
Note: For vertical plots (such as the wrf_CrossSection2_Final.ncl example) it is necessary to insert a statement here to disable the georeferencing capability. The wrf_Height.ncl plot is a horizontal plot, not a vertical plot; but, if it were vertical, at this point we would insert the statement “wrf2geotiff_disableGeoTags(wrf2gtiff)”.
After the line:
pltres = True
Insert the line:
pltres@gsnFrame = False
This gsnFrame resource needs to be False for us to control the timing when a new page
is output.
The plot is created by the call to wrf_map_overlays
plot = wrf_map_overlays(a,wks,...,pltres,mpres)
after that line, insert the two lines:
wrf2geotiff_write(wrf2gtiff, a, times(it),wks,plot,False)
frame(wks)
The above two lines cause information about the image to be saved (time, georeferencing) , so that it will later be output. The final argument to wrf2geotiff_write() determines whether or not the output image will be cropped to the WRF domain extents. By setting it to “False” we will obtain an image that extends beyond the plot area, with annotation,
colorbar, legends, etc. displayed outside the plot area. If you would like to not display the annotation, set this final argument to True. The annotation can also be cropped when it is displayed in VAPOR. The “frame(wks)” command is required (along with the
“pltres@gsnFrame=False” resource definition i nserted previously) in order to cause a new frame to be started after the georeferencing is saved.
Finally, immediately before the end at the end of the script, insert the line:
wrf2geotiff_close(wrf2gtiff,wks)
This last statement triggers the necessary file conversions needed to convert the postscript images to an output geo-tiff file.
Run the script. You will note several messages to the console. When the script completes, you should find a file “plt_HeightLevel.tiff” in the current directory. This is a geo-tiff file that includes the ten plots at height 2km together with geo-referencing information.
This script is saved as wrf_Height_Final.ncl in the Appendix.
3.2 Conversion of wrf_Precip.ncl
This script provides three plots. Here we only use the second of the three, which plots total precipitation tendency with sea level pressure isobars. We perform similar changes as were made in the previous example, to produce one image at each time step. This example, unlike the first example, is not associated with a particular height in the scene. Note that there will be several warning messages at the first time step, since total precipitation tendency requires two times to be calculated. The modified script, wrf_Precip_FirstMod.ncl, has been modified to produce one plot at each timestep. The following is the image at the third time step:
To produce a geo-referenced tiff output, follow the same steps that were followed in section 3.1, resulting in the ncl script wrf_Precip_Final.ncl. When you run this script, the output will be “plt_Precip.tiff”.
3.3 Conversion of wrf_CrossSection2.ncl
This script provides a vertical slice through the domain, plotting relative humidity and temperature. The original script produces 3 vertical plots, the first one at a constant x coordinate, the second at a constant y coordinate, and the third a diagonal slice with both x and y changing.
The conversion of this script is very much like the two preceding scripts, but the following should be noted:
?Since VAPOR only supports axis-aligned image planes, we use only the first plot (ip =
1). To adapt this to the Jangmi data (which is on a 200x200 horizontal grid) we set the
starting and ending (x,y) coordinates to be (0,84) and (200,84). We could easily modify this to be aligned with the YZ plane instead of the XZ plane.
?Because this is a vertical plot, we do not use georeferencing. The output tiff file will contain the WRF date/time stamps, but not the georeferencing from the WRF output. To turn off georeferencing, call:
wrf2geotiff_disableGeoTags(wrf2gtiff)
immediately after the call to wrf2geotiff_open(wks).
All other changes are similar to the changes discussed in the above two sections. Below is the figure that results from the second time step of this series:
Refer to the Appendix, which includes: the original script:
wrf_CrossSection2.ncl
?The version that produces one plot per timestep:
wrf_CrossSection2_FirstMod.ncl
?and the final version that produces a geotiff:
wrf_CrossSection2_Final.ncl
4 Obtain a terrain image for the domain of the simulation When we display the NCL/WRF plots in VAPOR, it will be useful to have a geo-referenced terrain image, showing us the location of the typhoon relative to the local geography (e.g. the island of Taiwan). We explain here how to easily obtain an image using the VAPOR utility “getWMSImage.sh”.
First, we must establish the latitude and longitude range for this terrain image. Fortunately, this information was already provided when we executed wrfvdfcreate in Section 2. The output of that command indicated that the longitude range was 117.053 to 126.302 degrees, and the latitude was from 18.7224 to 27.4392 degrees. We choose a slightly larger range to provide a geographic context.
The VAPOR shell command getWMSImage.sh can easily retrieve a geo-referenced image for this space. (In fact, the default behavior of this command is to obtain a satellite image for the specified rectangle.) Additional options are described in the man page. For example, you can also use this command to obtain maps of political boundaries or rivers.
To obtain an image for the specified coordinates, you must be connected to the Internet, and the current directory must be writable. On Linux or Mac, you must have sourced vapor-setup.csh. Issue the command:
getWMSImage.sh –o jangmiTerrain.tiff 115 15 130 30
On completion, a terra in image “jangmiTerrain.tiff” will be written to the current directory. The latitude goes from 15 to 30 degrees North and the longitude goes from 115 to 130 degrees East.
5 Display these images in the VAPOR scene
We can now use VAPOR to perform a 3D visualization of typhoon Jangmi, using the images we have created. If you are not familiar with VAPOR, we recommend that you first read the first section of the tutorial “Visualization of WRF Data Using VAPOR: A Georgia Weather Case Study”. This section will explain in detail the operations that we are about to perform.
https://www.wendangku.net/doc/0713918712.html,unch vaporgui
2.From the Data menu, Import the wrf output files into the current session. Alternately, if
you converted the data to a VDC (as described in Section 2), then load the file “jangmi-
09-28.vdf” into default session. You should see a white rectangular shape in the
visualizer window.
3.From the Edit menu, select “Edit Visualizer Features”. Next to “Scene Stretch Factors
X, Y, Z” set the values to 1, 1, 30. This will result in the 3D scene being stretched a
factor of 30 in the vertical (Z) direction, after you click the “OK” button. You will see that the white rectangular shape is now box-shaped.
4.Click on the “Image” tab. We first display the terrain image that we created in Section 4.
Click the button “Select Image File” and select the image “jangmiTerrain.tiff” that we generated. Check the “Instance 1:” checkb ox in the Image tab to display the terrain
image. Also check the box “Apply Image to Terrain” so that this image will appear on the earth’s surface in the scene. The VAPOR scene should look as follows:
Terrain image displayed in VAPOR Image panel
5.Next, we insert the image sequence we calculated using the wrf_Height.ncl script. We
place these horizontal images at elevation 5000m (this was the elevation that was used for calculation of the image by the NCL script). This is done as follows:
At the top of the Image panel, under Renderer Control, click the “New” button to create a new image renderer. Click on “Instance: 2” to select the second instance, then click the “Select Image File” button. Select the file “plt_HeightLevel.tiff” and then check the “Instance: 2” checkbox to view this image. To place it at elevation 5000m, set the Z coordinate under “Plane/Image Center” to 5000, by typing 5000 in the corresponding text box and then pressing the Enter key. Reposition the viewpoint by dragging with the left mouse button in the scene (you may also want to translate or zoom, using the middle and right buttons) so that you can see both the terrain and the data plot, as shown below:
Plot of humidity, temperature, etc. at elevation 5000, with annotation
Note that the annotation in the plot is shown in the VAPOR scene. To crop this
annotation, check the box “Crop to bounds” in the Image panel.
6.Next we insert the vertical image calculated using wrf_CrossSection2.ncl. Press the
“New” button under Renderer Control in the Image tab, and then click “Instance: 3” to select the new Image instance. Click the button “Select Image File” and select the file “plt_CrossSection2.tiff”. In the “Orientation” selector, above the file name, choose “X-Z”, since this is a vertical slice parallel to the X-Z plane. Then check the “Instance: 3”
box to display the vertical slice.
By default the vertical slice is positioned in the middle of the scene, at y coordinate 400000, which corresponds to a grid coordinate of 100. The correct y-
coordinate should be 84 (this y-coordinate was set in the NCL script). To move the plane to the correct value, click repeatedly on the left side of the Y slider for Plane/Image
center, until the y-coordinate (displayed under Grid extents (X,Y,Z) ) is 84. Alternatively just type in the value 336000 (800000*84/200) for the Y coordinate of Plane/Image
center, and press the Enter key. You will see the following image (note that the
wrf_Height plot has been cropped):
Vertical plot of humidity and temperature inserted at y coordinate 84
7.Next we insert the precipitation images calculated using wrf_Precip.ncl. Again click on
the “New” button under Renderer Control, then select “Instance: 4”, and then choose the image file “plt_Precip.tiff”. When you check the “Instance:4” checkbox you will see the first image of the sequence displayed at the vertical middle height of 12715m. The first image is not very interesting, so click the “play” button (?) in the animation toolbar at the top of the VAPOR window, and play the animation forward. VAPOR display will be
similar to the following at time step 5:
Precipitation plot inserted at timestep 5
8.We can use other visualization capabilities of VAPOR in conjunction with the images we
have created in NCL. The 3D extent of typhoon Jangmi can be illustrated by a volume rendering of the QCLOUD variable. From the DVR panel, select the QCLOUD variable and check the “Instance:1” checkbox to enable a volume rendering. Then set the four color control points in the DVR transfer function editor to white. The resulting
visualization shows the location of clouds relative to the variables plotted in the NCL
images: