Statistical models for mixture interpretation

DNA profiling is based on the ability to copy or ‘amplify’ small numbers of DNA molecules to produce a much larger number which can then be detected.  The initial DNA is called the ‘template’.  When the amount of DNA is very small, as measured by the increase in some sampling effects, the sample is termed a ‘low template’ (LT) or ‘low copy number’ (LCN) sample.

In LCN or LTDNA the problem is that some alleles may not be observed at all, or are uncertain because they have only been observed once^[1]. 

Dropout

Very low amounts mean that some alleles are not detected in every run, and possibly not at all.  When an allele should appear but does not, then the phenomenon is called dropout.  That problem of course is only apparent if an interpreter knows what DNA is present in the sample, or if more than one run is performed and the allele is absent in one or more of the runs, or in the extreme if an entire locus has no alleles present in the profile. 

Dropout is the extreme form of peak height variability.  In standard DNA profiling with the recommended amount of DNA (0.5ng – 1.5ng; 500pg – 1500pg) then the peak heights are consistent between runs.  When very low amounts of DNA are used the peak heights can vary considerably between runs.

Dropin

An additional complication of analysing very low amounts of DNA is that spurious alleles, supposedly from the environment, are detected.  These are again identified by the fact that they do not belong to a known or expected DNA profile.  They were detected in the original validation studies from pre-2000 performed by the FSS Ltd.

Consensus profiles

In order to avoid the identification of dropin alleles as ‘true’ components within a mixture many LCN processes count only those alleles that occur in two or more runs; single occurrences are regarded as dropin. 

Probabilistic genotyping 

Some authors created models that attempt to provide the probability of the evidence by estimating the probability that these alleles were actually there even though not seen (drop-out) or are ‘uncertain’ in the analysis.  These calculations had also to take account of the probability that they were indeed spurious alleles and were ‘drop-in’ (i.e. had been introduced into the original sample – a form of contamination).

Drop-out of an allele in one analysis while it is present in another is the extreme form of this peak height variation.  Some models therefore ignore peak height and consider only whether an allele is present regardless of any information that peak height may contribute to the assessment.  Models which ignore peak height are termed ‘semi-continuous’ (used in this case).  Ignoring peak height may be acceptable given the wide variation in peak heights when only low amounts of DNA are present and so the peak height itself is variable. 

Some models include consideration of peak height within the DNA profiles (which are graphs of the amount of each DNA component within a sample).  These models are called ‘continuous’. 

A further difference among models is how the probability of drop-out or drop-in is calculated.  Some use empirical data derived from laboratory experiments while others use estimates or simply insert values that maximise the parameter under each hypothesis.

Software programmes

The complex calculations involved in some of these approaches has led to the development of a number of software programmes which claim to calculate the LR for mixtures.  Each of these uses a different method to perform the calculation.

The President’s Council of Advisors on Science and Technology (PCAST) is an advisory group of leading scientists and engineers, appointed by the President of the United States to provide scientific advice.   Crucially, these advisers were primarily drawn from outside the forensic science community (whatever that is) and so represent a much wider, and in some respects less biased, perspective on the methods being employed in the forensic field. 

In September 2016, PCAST released a critique of several methods used in ‘forensic science’, including the interpretation of mixed DNA profiles^[2].  The PCAST report stated:

“The fundamental difference between DNA analysis of complex-mixture samples and DNA analysis of single-source and simple mixtures lies not in the laboratory processing, but in the interpretation of the resulting DNA profile.  ...

 probabilistic genotyping software programs clearly represent a major improvement over purely subjective interpretation. However, they still require careful scrutiny to determine

(1) whether the methods are scientifically valid, including defining the limitations on their reliability (that is, the circumstances in which they may yield unreliable results) and

 (2) whether the software correctly implements the methods. This is particularly important because the programs employ different mathematical algorithms and can yield different results for the same mixture profile.

Appropriate evaluation of the proposed methods should consist of studies by multiple groups, not associated with the software developers, that investigate the performance and define the limitations of programs by testing them on a wide range of mixtures with different properties. …

A number of papers have been published that analyze known mixtures in order to address some of these issues [with interpreting DNA mixtures]. Two points should be noted about these studies.

First, most of the studies evaluating software packages have been undertaken by the software developers themselves. While it is completely appropriate for method developers to evaluate their own methods, establishing scientific validity also requires scientific evaluation by other scientific groups that did not develop the method.

Second, there have been few comparative studies across the methods to evaluate the differences among them—and, to our knowledge, no comparative studies conducted by independent groups”

Summary on statistical models

There are different statistics that can be used to evaluate the significance of mixtures of DNA.  One of these is the Likelihood Ratio (LR).

The LR as routinely presented to a court, as well as being misunderstood, is difficult to comprehend and explain, and does not provide any indication of:

• how many other people may be considered potential contributors to the profile,
• the LR for any or all of those potential contributors, or
• the values of the numerator and denominator.

Low template mixtures of DNA create specific additional problems in the statistical evaluation.

Different models have been proposed to evaluate low template mixtures (continuous and semi-continuous).

Some of these models require computer software to calculate.  Different programmes have been developed.  These use different models and formulae to each other.  There is concern that these programmes have not been fully tested (validated).

Although proponents of these models and computer programmes which perform the calculations claim to take account of dropin and dropout, others claim that although the models may be sound, the data to support the calculations is not available.  Without reliable data, the calculations are themselves unreliable.  Computer programmes require extensive validation as there is no ‘true’ answer to which the results can be compared.