An open letter to Forensic Science International Genetics on data disclosure

Tweet

Dear Sirs

There are many reasons, and increasing demands, for the full disclosure of the underlying data used to support opinions in science [1,2,3], and forensic science in particular. This is even more important when there is increasing scepticism of the claims made by the proponents of the techniques used by forensic scientists [4], and profiling of ever-smaller amounts of DNA [5,6]. The debate on the reliability and validity of Low Template DNA techniques has featured calls for the disclosure of the full data and experimental regimen underlying published data [7,8,9,10] that would enable a thorough and meaningful assessment of the claims being made.

With the publication of the paper by Wetton et al[11] we (the undersigned) are now placed in an unfortunate, and perhaps unique, position caused by our having had sight of much of the data used in the paper and having formed an alternative view of the permissible inferences from the same data. Disclosure of that data was forced by a UK court ruling in a criminal case, but we cannot now provide the scientific community with that alternative interpretation because of the imposition of non-disclosure agreement insisted upon by the UK Forensic Science Service Ltd (a government-owned company). Our opinion requires open access to the full dataset, which is not included in the current paper.

We have, for the same reasons and with the same imposed conditions, assessed the data produced in dilution experiments that may or may not form the basis for the work referred by Wetton et al as ‘in preparation’ [12]. We also take a different view on whether these data support the proposition that, “ the correlation between input amounts and peak height that is still maintained at 34 cycles”, which is important underpinning for the interpretation of mixtures. In the event of publication, we are again prohibited from expounding our argument from the full data. In the meantime, it seems inappropriate for the authors to have given apparent authority to a study that has not even been through the review process and that has background significance for the value of the instant paper.

We believe that this situation is intolerable in a scientific environment, but especially in a forensic environment that already has an inbuilt inequality of arms regarding access to research [13]. This matter exposes the fundamental difficulty, which arises from the inequality in resources of the players, for those involved in challenging these assertions.

Proponents of these techniques claim commercial sensitivity as a reason for non-disclosure of the underlying data. Yet they simultaneously deny any novelty in the techniques in justifying refusals to disclosure requests in the UK, New Zealand, and USA. Given that the techniques are not patent, nor secret, there can be no valid reason for such secrecy, especially when those same techniques are being used in circumstances where they may determine the liberty of an individual.

The UK Forensic Science Regulator’s draft Codes of Practice and Conduct [14] clearly intend such data to be made available;

“The provider shall provide a validation report in sufficient detail to allow independent assessment of the adequacy of the work carried out in demonstrating that the method, product or service conforms to the specified requirements and is fit for purpose. It need not contain all the experimental data, but a summary of this data shall be provided and the raw data shall be available for inspection if required.”

There is clearly little point in making such data, “available for inspection”, if the conclusions from that inspection are effectively prohibited from being discussed within the scientific community because of restrictions on disclosing the data.

Perhaps in the adversarial environment, the principles of good science are being subsumed to special interests. The ‘ Guidelines for Responsible Data Management in Scientific Research’, endorsing previous NIH policy, state;

“After a project's research has been published or patented, any information related to the project should be considered open data. Other researchers may request raw data or miscellaneous information related to the project in order to verify the published data or to further their own research project.” [15]

The International Council for Science states;

“Scientific progress relies on full and open access to data and on the open disclosure of research results in the scientific literature.” [16]

The question must arise as to how many of the papers that this journal and others have published are founded on detailed data from which the wider scientific community would form different and contrary conclusions. We note and support the comments of Murphy, and others, that,

“Many labs do not engage in validation studies or routine exercises of blind proficiency testing, and those that do such testing tend to shroud the result in secrecy rather than publish them publicly as in other scientific disciplines.” [13]

On the basis of our case experience, and the substantial reasons presented here and elsewhere, we repeat previous proposals[7,9], and widespread scientific opinion[1,2,3,15,16], that availability to the wider scientific community of all of the experimental details and data to enable a thorough and meaningful assessment of the claims made by authors is essential. This should be a requirement of publication in reputable scientific journals. In those circumstances where commercial sensitivity, or any other reason, is used for failure to disclose, then the editorial board must consider and publish simultaneously with the paper the reasons for such non-disclosure.

Professor Allan Jamieson

Dr Scott Bader

[This letter was submitted to, and rejected, for publication by Forensic Science International Genetics]

Posted July 2011

END

back to top>>