A bug in the software used by researchers to interpret fMRI data could invalidate fifteen years worth of neuroscientific research, a paper claims.
Three of the most popular pieces of software for fMRI – SPM, FSL and AFNI – were all found to have false positive rates of up to 70 per cent. These findings could invalidate "up to 40,000 papers", researchers claim.
fMRI measures blood flow inside the brain and, by proxy, brain activity. It assumes cerebral blood flow is coupled or correlated with neural activity, and has been used to explore how the human brain responds to robots, how memory and imagination interact, how the brain looks when someone has an idea and more.
"Though fMRI is 25 years old, surprisingly its most common statistical methods have not been validated using real data," said Anders Eklund.
The team used resting-state fMRI data from 500 people, split into 25 groups, and measured this data against each other to generate 3 million comparisons.
This resting-state fMRI data from 499 healthy controls was downloaded from the 1,000 Functional Connectomes Project. The authors said resting-state data "should not contain systematic changes in brain activity", but their previous work showed this assumption can have a large impact on the degree of false positives.
And this, the team say, "may have a large impact on the interpretation of neuroimaging results".
"It is not feasible to redo 40,000 fMRI studies, and lamentable archiving and data-sharing practices mean most could not be reanalysed either," the team said. "Considering it is now possible to evaluate common statistical methods using real fMRI data, the fMRI community should, in our opinion, focus on validation of existing methods.
"It is important to stress we have focused on inferences corrected for multiple comparisons in each analysis, yet some 40 per cent of a sample of 241 recent fMRI papers did not report correcting [these] comparisons, meaning many group results in the fMRI literature suffer even worse false-positive rates than found here."
The study has been published in PNAS.
This article was originally published by WIRED UK
