Bug #441
open
F3200 filter
0%
Description
This is a feature request, not a bug.
I have seen a large number of F3200 "suspecting replicated data" warnings, most of them spurious. HDH eliminated about 90% of the unnecessary F3200 warnings by providing an option to issue no F3200's for 0-D variables. Even so, in my experience the majority of F3200's have been spurious.
The F3200 exception is based on a comparing few indicators such as mean and maximum. For my own use, I wrote (in Python) a filter to read the suspect data a second time and compare the entire data arrays, item by item. For the comparison I used the method favored by numerical analysts and implemented by the function numpy.ma.allclose(): scalars a and b are close if |a-b|<atol+rtol*max(a,b). My default tolerances are atol=1.e-9 and rtol=1.e-6, but any small numbers would work. So far I have sent hundreds of data sets through this filter, and now I have zero spurious warnings.
I think that a better place for this second-pass check would be the QC tool.