23andMe contributes to longevity GWAS controversy

As an addendum to my previous post on the controversial "longevity genes" study, you should go and check this out. It's a post on the blog of personal genomics company 23andMe, and it's a pretty impressive piece of scientific dissection of the longevity GWAS paper - in addition to detailing a variety of methodological problems with the study, the authors actually used the 23andMe database to look at the predictive value of the longevity GWAS algorithm on their own customers:

We took a preliminary look in our customer data to see if the proposed SNP-based model described in Sebastiani et al. is predictive of exceptional longevity. A commonly used measure of test discrimination is to calculate how often, for a randomly selected case and control, a test correctly assigns a higher score to the case. This is known as the "c statistic" or "area under the curve". The authors of the new study say their model scored a 0.93 for this statistic. But when we compared 134 23andMe customers with age ≥ 95 to more than 50,000 controls, we obtained a test statistic of 0.532, with a 95% confidence interval from 0.485 to 0.579. Using 27 customers with age ≥ 100, we get a value of 0.540, with a 95% confidence interval from 0.434 to 0.645. A random predictor of longevity would give a 0.5 on this scale, so based on our data, performance of this model is not significantly better than random. Even with our small sample size, we can also clearly exclude values as high as the published result of 0.93.

Small numbers, but certainly not good news for Sebastiani et al.

Ironically, a few days ago the longevity GWAS authors were expressing concern that direct-to-consumer genetic testing companies might use their longevity prediction algorithm without providing sufficient caveats:> However, people who have already had their genomes analyzed, through services such as 23andMe, will soon be able to predict their risk score through a free website that Perls' collaborator is developing. But Perls hopes to head off commercial efforts to market this kind of test. "We are concerned that the marketing [for such a test] will not mention the shortcomings of the test," says Perls.

No need to worry - 23andMe has now done a massively better job of conveying "the shortcomings of the test" than the study's authors have, and I think we can now safely say that the authors won't need to head off a stampede to commercialise their algorithm.