Verification of Statistical Results

Verification of statistical results is part of routine internal Quality Control of every statistical programming unit.

We have performed several instances of independent result verification, in addition to the internal routine quality control. To our great surprise, we have quite regularly identified major problems, not only in secondary but also in primary analyses, although internal quality control and even independent programming had certainly taken place. This did also affect the overall statistical interpretation in some instances.

The explanation for this unexpected finding is that the internal quality control is often not truly independent. For example, issues may have originated at an earlier stage: when pulling together analysis sets, when pooling centres, or when producing derivations.

Another situation we have observed during verification activities was that the internal quality control was though effective, but that the statistical method-ology was not appropriate or according to best praxis. Violation of distributional assumptions for example, which should have been addressed according to Statistical Analysis Plans, had not been identified by the internal reviewers but only during independent verification.

Truly independent verification of statistical results will thus

a.      increase the credibility of results, and

b.      ensure reproducibility of results, especially if analyses have been performed by service providers.

Recommendation:

Independent result verification should be routinely incorporated into statistical processes, with its scope adapted to the importance of the study.