Evaluation of FitACF 3.0 and despecking routine


FitACF 3.0 was developed by Pasha Ponomarenko and Keith Kotyk in 2015-2016. It was first included in the RST in January 2018 as a non-default command line option in make_fit.

To complement the new fitting algorithm, a new routine called fit_speck_removal was introduced into RST in 2021 to mitigate the impacts of non-Gaussian noise and interference on the fitted data. This routine was developed by Emma Bland and Pasha Ponomarenko.

At the SuperDARN Workshop in May 2021, the DAWG proposed that FitACF 3.0 become the standard fitting algorithm for processing raw SuperDARN data. After the workshop, the PI Executive Council requested some additional information and evaluation of fitACF 3.0 and the despecking routine. This information will be hosted on this webpage.

To contribute plots/analysis to this page, please contact Emma.

Initial discussions


15/07/2021: Notes from PI discussion of FitACF 3.0

17/08/2021: Summary of the follow-up discussion (04/08/2021)

Summary of previous work (2015-2020)


Summary of previous SuperDARN Workshop presentations about FitACF 3.0

The full set of presentations and other documents about FitACF 3.0 are available here.

New results


Click each heading to expand

We have produced two-dimensional histogram plots comparing the values of the following parameters: lag zero power, fitted power (p_l), velocity, spectral width (w_l) and elevation. Sample plots are provided for the Clyde River, Hankasalmi, Hokkaido East and McMurdo radars. PIs are invited to use the test IDL code to produce similar plots for their own radars.

To demonstrate that the new despecking routine is effective at removing isolated points in range-time space, we performed the following analysis:

The top panel of each plot shows the total number of valid fits (isolated and clustered) for each fitting algorithm, before and after despecking. The bottom panel shows the number of clustered points expressed as a percentage of the total number of valid fits.

Note: This analysis is different to the analysis requested by the PIEC. We thought that it was more objective to examine the number of clustered and isolated points for both versions of FitACF.

Conclusion

The proposed combination of (FitACF 3.0 + despecking) results in a higher number of good fits and a lower percentage of noise ‘leakage’ compared to FitACF 2.5.

Fitting XCF leads to a larger statistical uncertainty in determining lag 0 phase as compared to the direct measurements from XCF.

The fitted lag 0 phase (FitACF 2.5) tends to spread some elevation angle values over the whole range​. This difference with the measured values (FitACF 3.0) is due to increased phase variance caused by:​

Note: XCF phase fitting is still performed in FITACF3.0 so that the “fitted” elevation angle values are also available. They show the same “spread” with respect to the “measured” values as in FITACF2.5​

Summary document on external noise effects in lag 0 power and phase

Responses to queries about specific datasets


This interval has been analysed in detail by Emma and Pasha and the main findings were (link):

The DAWG recommended blocklisting these data.

This BOREALIS-specific issue has already been analysed and resolved (link), and the necessary corrections have been implemented in RST4.6 on 21 June 2021.

Software implementation


They have never been used in published research

XCF power/velocity fits represent inferior replicas of the respective ACF fits due to the wider interferometer beam that lets in more noise/interference. The original reason for their implementation is unclear, most probably “just nice to have…”

Substantial (~50%) reduction in processing speed and storage space

Alternative options

One can always use FitACF 2.5 to obtain the XCF power/velocity fits. FitACF 2.5 will continue to be included in the RST indefinitely.

Note: Re-introducing XCF power fitting into FitACF 3.0 will require:

After some investigation, we have concluded that adding a despecking command line option to make_fit will result in less-than-optimal performance of the make_fit routine and a higher risk of corrupted files in comparison to executing two already-existing routines consecutively. It would also require considerable effort to implement.

The workflows of make_fit and fit_speck_removal are very different. make_fit reads, processes and writes each time record individually, and each record is independent of the others. In contrast, fit_speck_removal performs the filtering in range-time space for each beam and channel separately, and then writes a new fitacf file in the original time order. This is incompatible with the record-by-record workflow of make_fit.