Yet another trial by press release for personal genomics

A press release launched yesterday by the European Society of Human Genetics announces two upcoming presentations at the Society's annual meeting next week, both critical of personal genomics.

How valid are the criticisms? It's hard to say; this is a press release about two promised conference presentations, and in neither case (as far as I can tell) has the actual research been published. That hasn't stopped the release being trumpeted by media as a blow against the personal genomics industry: the Daily Mail is predictably hyperbolic, leading with "Call to ban mail-order DNA disease testing kits that claims to predict life-threatening illnesses"; the Guardian runs with the only marginally better "Genetics tests flawed and inaccurate, say Dutch scientists". In the absence of the details of the analysis (I've emailed one of the quoted scientists to see if I can get hold of this) there's not much we can do except sift through the press release - a depressing way to discuss science.

So, what's the actual story here? Firstly, a presentation by Erasmus University's Cecille Janssens will discuss alleged problems with the risk prediction algorithms used by personal genomics companies 23andMe and deCODEme. The researchers "simulated genotype data for 100,000 individuals based on established genotype frequencies and then used the formulas and risk data provided by the companies to obtain predicted risks for eight common multi-factorial diseases". Here are the key claims:

"So individuals in the increased risk group may have a four-fold increased risk of disease, but they are still far more likely not to develop the disease at all. For T2D [type 2 diabetes], where the companies calculated the average risk at around 25%, 32% of those assigned to the increased risk group would actually develop T2D compared to 22% in rest of the study population. This difference in disease risk is too small to be of relevance", said [Erasmus University] Professor [Cecille] Janssens.

Right - so current personal genomic profiles offer a pretty small but non-zero benefit in terms of risk prediction for common, complex diseases, with the worst outcome being for weakly heritable diseases like type 2 diabetes. No surprises there; the companies typically express their predictions in ways that communicate the magnitude of increased risk fairly effectively. It's unclear from the press release how the companies should be conveying risk differently.

"deCODEme predicted risks higher than 100% for five out of the eight diseases", Ms Kalf will say. "This in itself should be enough to raise considerable concern about the accuracy of these predictions – a risk can never be higher than 100%. In the case of AMD one in every 200 individuals in the group would have received a predicted risk of higher than 100%, suggesting that they would definitely develop the disease."

That is indeed strange. In the Guardian piece, deCODE Genetics CEO Kari Stafansson appears to argue that the team has misused the company's risk prediction algorithm:

"We never report a lifetime risk over 90%. This is not how we use these models."

Either the team got the models wrong, or deCODEme has failed to adequately communicate exactly which algorithm they use, or there's something fundamentally wrong with deCODEme's risk prediction. Without more details from the authors it's hard to know which of these is true, although given deCODE's impressive academic pedigree in risk prediction I'm putting a pretty low prior on that final possibility.

"They only take genetic factors into account when predicting risks for consumers, whereas in most multi-factorial diseases other modifiable risk factors, such as diet, environment, exercise and smoking have a much stronger impact on disease risk", said Professor Janssens.

This is a non-issue. Companies can't include environmental factors in their risk prediction models because the interactions between genetic and environmental factors is unknown. In general, the companies do an appropriate job of informing consumers of the fact that variables other than genetics affect risk (23andMe, for instance, indicates the fraction of risk explained by genetics vs environment directly underneath each of its detailed disease risk predictions). Again, it's unclear how the companies should do a better job here.

The second presentation described by the press release is substantially more annoying. This time it's about attitudes towards personal genomics among clinicians, based on "a survey of a representative sample of clinical geneticists from 28 countries across Europe on their experience of and attitudes to DTC genetic testing". In other words, the researchers asked a group of people who see themselves as the gatekeepers to genetic information whether someone else should be allowed to provide that information. The results are predictable:

"Clinical geneticists' concerns with DTC genetic tests are mostly related to the fact that these tests usually lack clinical validity and utility. Moreover, these tests are usually carried out without the provision of genetic counselling. According to the experiences of clinical geneticists, patients often do not know how to interpret the results they receive and are often confused by them. However, almost all clinical geneticists feel that they have a duty to provide counselling if patients contact them after having purchased a DTC genetic test," says Dr. Howard.

So clinical geneticists feel that poor ignorant consumers can't properly interpret the results of whole genome scans. Whether that actually translates into genuine difficulty in interpretation, of course, is quite a different matter. On this topic I'll see your press release and raise you this one from late last year, which reported on a survey of over 1,000 personal genomics customers presented at the American Society of Human Genetics 2010 meeting (and not yet published); those researchers asked respondents to examine actual risk reports from personal genomics companies, and found that around 95% of them interpreted the results correctly.

The release continues:

"A person who undergoes a genetic test has to be accompanied – explanations, physical aid, the right to choose whether to know or not – and this is not true in the case of direct access to such a test", said one survey respondent.

"Genome-wide scans by companies are totally unacceptable, as they can derive sensitive information about medically relevant conditions and will also provide lots of information which is difficult to interpret, even for medical professionals", said another respondent. Presenting the results of such tests directly to individuals is unacceptable, the majority of those surveyed said.

You read that correctly - this information is difficult to interpret "even for medical professionals", yet clearly the answer here is to force everyone to access their own genetic information through those same confused professionals. This argument is just as ludicrous here as it was when the American Medical Association made it earlier this year.

Bear in mind that the clinicians interviewed here - clinical geneticists - would not be the doctors that most consumers of personal genomics products would be forced to consult with before they can look at their own DNA. Instead, we would be legislatively required to sift through our data in the presence of general practitioners with (typically) zero formal training in modern genomics or genetic disease risk prediction. This is an absurd outcome; yet the press release argues explicitly for this approach:

Currently only a few European countries, for example France and Switzerland, have legislation that states that genetic tests can only be accessed via individual medical supervision. "Although this model is sometimes criticised for being too paternalistic", says Professor Borry, "in the absence of a good working pre-market control of genetic tests, it could be a useful way of responding to some of the concerns over DTC testing."

So, take-home message here: clinicians believe that people should be forced to pay clinicians in order to access their own genetic information. What a surprise!

While it is important that personal genomics companies consult with medical professionals in devising their risk prediction algorithms and interfaces, regulation of genetic testing should not be based on the views of the traditional gatekeepers of genetic information. It should also not be based on unsupported ideas about the ability of consumers to interpret risk predictions, or on paternalistic views regarding the potential dangers of learning more about our own genomes without a clinician to hold our hands. And please, forcing consumers to access their results in the presence of someone who knows less than they do about modern genetics is not a viable working solution to the impending challenges of genomic medicine.

In the complete absence of any evidence for harm to consumers caused by the personal genomics industry, the appropriate working solution is simple: enforce existing regulations to control technical accuracy and punish false claims, continue working to push companies to provide their customers with access to all of the information required to make a sound decision, and continue to work to generate evidence about the best approaches for generating and communicating risk information to the public.

See also: outraged comments by Razib and Trey. Here's an excellent quote from Razib:

I think one of the key issues is that these genetic professionals view DTC genomics in the same category as a pharmaceutical. I view DTC genomics as part of the same family of consumer and social goods as information technology. When viewed in the context of our current medical-industrial infrastructure it seems that on the margin the future opportunities to reduce morbidity through better lifestyle choices and more information via DTC genomics is a no brainer. If there was social science evidence that people who receive false positive results are regularly committing suicide then obviously my preference for loose regulation would need re-examination. That would be like someone jumping off the tenth floor of an office building after a “blue screen of death.”

But from what I’ve seen at places likes Genomes Unzipped and FuturePundit these dystopian visions of mass hysteria don’t end up to panning out. Until that point it seems that the best avenue toward improvement of this technology is to allow the trial and error process of innovation to continue.