Scientists rely on brain-wide association studies to measure brain structure and function—using MRI brain scans — and link them to complex characteristics such as personality, behavior, cognition, neurological conditions and mental illness.
Magnetic Resonance Imaging equipment in the hospital. Image credit: Max Pixel, CC0 Public Domain
New research published in Nature from the University of Minnesota and Washington University School of Medicine in St. Louis is highlighting a path forward for brain imaging use in diagnostics, prognostics and treatment response in psychiatric, psychological and neurological conditions. The research shows that most published brain-wide association studies are performed with too few participants to yield reliable findings.
The study used publicly available data sets—involving a total of nearly 50,000 participants—to analyze a range of sample sizes and found:
- Brain-wide association studies need thousands of individuals to achieve higher reproducibility. Typical brain-wide association studies enroll just a few dozen people.
- So-called “underpowered” studies are susceptible to uncovering strong but misleading associations by chance while missing real but weaker associations.
- Routinely underpowered brain-wide association studies result in a surplus of strong yet irreproducible findings.
- New parameters for brain-wide association studies provide direction for better utilizing findings to improve clinical practice and mental health care.
“For decades we’ve been highlighting the potential for MRI to assist in the clinical care—including the diagnosis, risk, response to treatment, etc.—for mental health disorders and neurologic conditions. However, the full potential has not been realized,” said Damien Fair, PA-C, PhD, senior author and Redleaf Endowed Director for the Masonic Institute for the Developing Brain (MIDB) at the University of Minnesota. “We now know our missteps and are redefining the required parameters, the so-called ‘special sauce,’ to move forward effectively.”
To identify problems with brain-wide association studies, the research team began by accessing the three largest neuroimaging data sets: the Adolescent Brain Cognitive Development Study (11,874 participants), the Human Connectome Project (1,200 participants) and the UK Biobank (35,375 participants). Then, they analyzed the data sets for correlations between brain features and a range of demographic, cognitive, mental health and behavioral measures, using subsets of various sizes. Using separate subsets, they attempted to replicate any identified correlations. In total, they ran billions of analyses, supported by the MIDB Informatics Group and the powerful computing resources of the Minnesota Supercomputing Institute.
The researchers found that brain-behavior correlations identified using a sample size of 25—the median sample size in published papers—usually failed to replicate in a separate sample. As the sample size grew into the thousands, correlations became more likely to be reproduced. Robust reproducibility is critical for today’s clinical research.
Senior author Nico Dosenbach, MD, PhD, an associate professor of neurology at Washington University, says the findings reflect a systemic, structural problem with studies that are designed to find correlations between two complex things, such as the brain and behavior.
“It’s not a problem with any individual researcher or study. It’s not even unique to neuroimaging,” said Dosenbach. “The field of genomics discovered a similar problem about a decade ago with genomic data and they took steps to address it. The NIH began funding larger data-collection efforts and mandating that data must be shared publicly, which reduces bias and as a result, genome science has gotten much better. Sometimes you just have to change the research paradigm. Genomics has shown us the way.”
Neuroimaging studies are expensive and time-consuming—just an hour on an MRI machine can cost $1,000. But if all of the data from multiple small studies were pooled and analyzed together, including statistically insignificant results and minuscule effect sizes, the result would likely approximate the correct answer, Dosenbach said.
“The future of the field is now bright and rests in open science, data sharing and resource sharing across institutions in order to make large data sets available to any scientist who wants to use them. This very paper is an amazing example of that,” said Fair. “Here at MIDB, we’re trying to provide the scientific community from all walks of life the resources necessary to do the work optimally.”
Source: University of Minnesota