An example workflow for creating SuperDARN map files.

RST version: 5.1.1

These set of processing steps are provided as a quick example for how to create SuperDARN convection maps. Some of the parameters chosen for the mapping process (steps 4 through 8), may not be best suited for all analysis. The Data Analaysis Working Group encourages further investigation into other possible options described in the RST readthedocs site and the API documentation to better suit a specific usage of SuperDARN data. The group also encourages contacting one of the SuperDARN institutions for any questions related to these processing steps and the appropriate usage to suit the analysis being conducted.

  1. Fit-level data generation (given rawacf files have been acquired from one of the SuperDARN data distribution sources)

    Often rawacf files are distributed as bzip2 compressed files, but need to be uncompressed before processing (i.e. bunzip2 [inputfile].rawacf.bz2). To process rawacf data to fitacf data, use the make_fit binary in one of the following ways:

    fitacf2:

    make_fit -fitacf2 ~/raw_dir/YYYYMMDD.HHMM.ss.SID.rawacf > ~/fit_dir/YYYYMMDD.HHMM.ss.SID.fitacf

    fitacf3:
    Fitacf3 processing is similar but adds a step of despecking to remove isolated pixels form range-time fitacf data.

    make_fit -fitacf3 ~/raw_dir/YYYYMMDD.HHMM.ss.SID.rawacf > ~/fit_dir/YYYYMMDD.HHMM.ss.SID.temp.fitacf3 fit_speck_removal -quiet ~/fit_dir/YYYYMMDD.HHMM.ss.SID.temp.fitacf3 > ~/despeck_dir/YYYYMMDD.HHMM.ss.SID.fitacf3

    More info on fit-level data generation can be found here. Also, more info on the need for despecking and how this process works can be found here.

  2. Gridding fitacf files for a given radar into a daily grid file

    Sampling interval: 120 seconds
    Minimum range: 800 km to remove E-region scatter
    Maximum range: 2000 km to remove most of the 1.5 and higher hop scatter
    Maximum velocity error: 100 m/s to remove data with large velocity errors (the cutoff is based on fitacf3 velocity error statistics)
    Maximum velocity: 3000 m/s to remove velocity spikes caused by low-quality data Channel: for multi-channel radars that have multiple channels in one file, non-experimental data suitable for this processing is found onchannel A
    Scatter filtering: ionospheric scatter only (based on flag set in previous step)

    make_grid -xtd -tl 120 -i 120 -minsrng 800 -maxsrng 2000 -vemax 100 -vmax 3000 -cn A -ion ~/despeck_dir/YYYMMDD*.SID.fitacf3 > ~/single_grid_dir/YYYYMMDD.SID.grd

    The addition setting in this line is:

    • -xtd Creates an extended output that includes both the power and spectral width measurements and their errors

    More info on the make_grid binary can be found here

  3. Combining daily grid files form individual radars into a single daily file per hemisphere

    combine_grid ~/single_grid_dir/YYYYMMDD*.grd > ~/all_grid_dir/YYYMMDD_HEM.grd

    Where the HEM value differentiates between the northern and southern hemisphere. Only data from the same hemisphere should be used as an input to the combine_grid process. Noted the use of the wildcard asterisk (*) in the example line above as the goal is to include files from multiple radars as an input to combine_grid.

  4. Initializing a map file for a given day and hemisphere

    map_grd ~/all_grid_dir/YYYMMDD_HEM.grd > ~/map_dir/YYYYMMDD_HEM.empty.map

    Note: By default, this code creates files for the northern hemisphere. For the southern hemisphere, use the -sh switch after the map_grd portion of the line.

  5. Adding the Heppner-Maynard Boundary (HMB)

    Minimum spectral width: 100 m/s to remove the low-width sub-auroral echoes while searching for the HMB

    map_addhmb -wdt 100 ~/map_dir/YYYYMMDD_HEM.empty.map > ~/map_dir/YYYYMMDD_HEM.hmb.map

  6. Adding the Interplanetary Magnetic Field (IMF)

    There’s two ways to add IMF data to the map file.

    Fixed Values

    map_addimf -vb -bx 1.21 -by -7.8 -bz 0.56 -vx 367.5 -kp 3 ~/map_dir/YYYYMMDD_HEM.hmb.map > ~/map_dir/YYYYMMDD_HEM.imf.map

    IMF/Kp from a File

    map_addimf -d 00:10 -if ~/imf_data_dir/rst_imf_kp.YYYY.txt ~/map_dir/YYYYMMDD_HEM.imf.map
    • -d 00:10 signifies a delay time of 10 min from the magnetopause of omni data
    • rst_imf_kp_YYYY.txt is an ASCII file with IMF and Kp (Kp is optional) data for the year YYYY with one-minute step with a format of:
      • Year Month Day Hour Minute Second Bx By Bz Vsw_x Kp

      • For example (no header is included in the ASCII file):

        2024 01 01 00 03 0.0 4.10 -3.09 -1.39 -303.60 0.67
        2024 01 01 00 04 0.0 4.01 -3.46 -0.88 -304.30 0.67
        2024 01 01 00 05 0.0 4.02 -3.43 -1.05 -304.90 0.67

  7. Adding Statistical Model

    Model order: 6 (to avoid small-scale “ringing” from sharp velocity gradients)
    Doping level: l for low

    map_addmodel -o 6 -d l ~/map_dir/YYYYMMDD_HEM.imf.map > ~/map_dir/YYYYMMDD_HEM.model.map

    Note without specifying a model, the Thomas Shepherd 2018 without Kp (TS18) model is used.

  8. Adding Spherical Harmonic Fit

    map_fit ~/map_dir/YYYYMMDD_HEM.model.map > ~/map_dir/YYYYMMDD_HEM.map

The resulting YYYYMMDD_HEM.map file is a usual map file that contains information related to convection mapping. More information on the data fields contained in the map files can be found in the rst readthedocs documentation