Evaluation of FitACF 3.0 and despecking routine
FitACF 3.0 was developed by Pasha Ponomarenko and Keith Kotyk in 2015-2016. It was first included in the RST in January 2018 as a non-default command line option in make_fit.
To complement the new fitting algorithm, a new routine called fit_speck_removal was introduced into RST in 2021 to mitigate the impacts of non-Gaussian noise and interference on the fitted data. This routine was developed by Emma Bland and Pasha Ponomarenko.
At the SuperDARN Workshop in May 2021, the DAWG proposed that FitACF 3.0 become the standard fitting algorithm for processing raw SuperDARN data. After the workshop, the PI Executive Council requested some additional information and evaluation of fitACF 3.0 and the despecking routine. This information will be hosted on this webpage.
To contribute plots/analysis to this page, please contact Emma.
Meeting summaries & feedback
15/07/2021: Notes from PI discussion of FitACF 3.0
17/08/2021: Summary of the follow-up discussion (04/08/2021)
24/11/2021: PIEC feedback to the DAWG
Summary of previous work (2015-2020)
Summary of previous SuperDARN Workshop presentations about FitACF 3.0
The full set of presentations and other documents about FitACF 3.0 are available here.
New results
Click each heading to expand
We have produced two-dimensional histogram plots comparing the values of the following parameters: lag zero power, fitted power (p_l), velocity, spectral width (w_l) and elevation. Sample plots are provided for the Clyde River, Hankasalmi, Hokkaido East and McMurdo radars. PIs are invited to use the test IDL code to produce similar plots for their own radars.
To demonstrate that the new despecking routine is effective at removing isolated points in range-time space, we performed the following analysis:
- Selected 5 days of common mode data. All 38 radars are represented in at least one of the selected days.
- Processed the raw data with both FitACF2.5 and FitACF 3.0 (RST version 4.6)
- Applied the despecking routine to both the FitACF2.5 and FitACF 3.0 data
- Determined the total number of valid fits (
qflg=1), the number of isolated (‘bad’) fits and the number of clustered (‘good’) fits for FitACF2.5 and FitACF3.0, before and after despecking.
The top panel of each plot shows the total number of valid fits (isolated and clustered) for each fitting algorithm, before and after despecking. The bottom panel shows the number of clustered points expressed as a percentage of the total number of valid fits.
Note: This analysis is different to the analysis requested by the PIEC. We thought that it was more objective to examine the number of clustered and isolated points for both versions of FitACF.
Conclusion
The proposed combination of (FitACF 3.0 + despecking) results in a higher number of good fits and a lower percentage of noise ‘leakage’ compared to FitACF 2.5.
Fitting XCF leads to a larger statistical uncertainty in determining lag 0 phase as compared to the direct measurements from XCF.
The fitted lag 0 phase (FitACF 2.5) tends to spread some elevation angle values over the whole range. This difference with the measured values (FitACF 3.0) is due to increased phase variance caused by:
- cross-range interference in non-zero lags, especially for ACFs affected by Tx overlap;
- imperfect phase ‘unwrapping’.
Note: XCF phase fitting is still performed in FITACF3.0 so that the “fitted” elevation angle values are also available. They show the same “spread” with respect to the “measured” values as in FITACF2.5
Responses to queries about specific datasets
This interval has been analysed in detail by Emma and Pasha and the main findings were (link):
- there was an internal interference/noise that was acknowledged by the PI on 14 January 2019 (link)
- the number of averages is extremely low (<=5) so that the data do not meet the basic criteria for a statistically valid ACF.
The DAWG recommended blocklisting these data.
This BOREALIS-specific issue has already been analysed and resolved (link), and the necessary corrections have been implemented in RST4.6 on 21 June 2021.
- Plots for the 29 May 2021 example are available here
- Kevin Sterne has confirmed that the data on davit.ece.vt.edu have been processed with the older software (http://davit.ece.vt.edu/images/tenplot/20210529.tenplot.pola.jpg). The same data processed with the newer version of the code show no such issue.
They have never been used in published research
XCF power/velocity fits represent inferior replicas of the respective ACF fits due to the wider interferometer beam that lets in more noise/interference. The original reason for their implementation is unclear, most probably “just nice to have…”
Substantial (~50%) reduction in processing speed and storage space
- Processing time: FitACF 3.0 is ~4.5 times slower than FitACF 2.5 because extra steps are required for the correct implementation of least squares fitting. Fitting XCF power will approximately double the processing time.
- Size of
fitacffiles: currently the file size is similar for FitACF2.5 and FitACF3.0. Including the XCF power/velocity fits will effectively double the file size.
Alternative options
One can always use FitACF 2.5 to obtain the XCF power/velocity fits. FitACF 2.5 will continue to be included in the RST indefinitely.
Note: Re-introducing XCF power fitting into FitACF 3.0 will require:
- additional research in order to implement it correctly (the FitACF 2.5 implementation is far from being perfect)
- significant full-time-equivalent contribution from highly-qualified programmers for development and testing of the code
- non-zero resource investment into ongoing software maintenance/upgrades
After some investigation, we have concluded that adding a despecking command line option to make_fit will result in less-than-optimal performance of the make_fit routine and a higher risk of corrupted files in comparison to executing two already-existing routines consecutively. It would also require considerable effort to implement.
The workflows of make_fit and fit_speck_removal are very different. make_fit reads, processes and writes each time record individually, and each record is independent of the others. In contrast, fit_speck_removal performs the filtering in range-time space for each beam and channel separately, and then writes a new fitacf file in the original time order. This is incompatible with the record-by-record workflow of make_fit.