Frank Harrell Jr., Ph.D., delivers last week's Discovery Lecture. (photo by Dana Johnson)
Discovery Lecture probes impact of ‘information allergy’ epidemic
A serious illness plagues biomedical and epidemiologic research as well as clinical practice.
The condition, dubbed “information allergy,” causes well-meaning investigators to come to erroneous conclusions — sometimes at the cost of thousands of lives.
“Information allergy is a disease that is epidemic, but you may not have heard about it,” said Frank Harrell Jr., Ph.D., professor of Biostatistics at Vanderbilt University Medical Center and founding chair of the department, at last week's Discovery Lecture.
In his talk, Harrell cautioned investigators and clinicians about the dangers of this illness and illustrated the various ways that important information is sometimes ignored.
“Information is the result of processing data and organizing it in a way that adds to the knowledge of the receiver of the information,” he said. “Optimum decision making must use the most complete — and current — set of information that the decision maker is capable of handling.”
Too often, though, much of the information available through biomedical and clinical research is ignored or overlooked.
One field, Harrell says, that is prone to “information allergy,” is nutritional epidemiology. “Findings are coming out all the time,” he said. “The public doesn't know what to believe — is a food good for you or bad for you? One reason is that the type of research that's done traditionally in nutritional epidemiology leaves a lot to be desired.”
In particular, he noted that nutritional epidemiology researchers tend to ignore confounding variables — factors that might provide an alternative conclusion. For example, in a large study of food intake and breast cancer rates, the authors determined that 11 foods were associated with breast cancer incidence. However, when controlling for confounding factors, no foods were associated with breast cancer incidence. This illustrated how ignoring relevant information can lead to wildly different conclusions than if the proper analysis had been done.
Another error Harrell sees frequently is the tendency of researchers, and particularly clinicians, to categorize patients or research subjects using “cutpoints,” or thresholds that break a group into two distinct populations, such as high vs. low, or diseased vs. healthy.
“Categorizing continuous variables is one of the great evils of analyzing data. For some reason, physicians are afraid of continuous variables. They attempt to find cutpoints so they can think of things as 'high' or 'low.' But mathematically, such a cutpoint cannot even exist unless the relationship of the marker and the outcome is discontinuous. And we don't see many discontinuous relationships in medicine,” he said.
“Cutpoints are really disasters…it's surprising that so many researchers are still using them.”
Harrell ended his talk by illustrating the fatal consequences of information allergy. A clinical trial of anti-arrhythmia drugs — believed to prevent cardiac deaths by preventing arrhythmias — actually caused the deaths of 24,000 to 69,000 of patients from arrhythmias. The flawed hypothesis leading to the trial was based on a complex type of “information allergy” that included categorizing continuous variables, overlooking confounding factors, and ignoring preclinical work that indicated harm.
“This case study of the consequences of ignoring information,” Harrell concluded, “shows that ignoring information can kill.”
For a complete schedule of the Discovery Lecture Series and archived video of previous lectures, go to www.mc.vanderbilt.edu/discoveryseries.