October 26-27 2020 Boston, MA
Senior Vice President, Head of Digital Sciences
Though the p-value “determines everything we do in drug development or medical research,” says Dr. Ulo Palm, it may be one of the most misunderstood and misused quantities in experimental science—drug discovery included. At its core, the p-value shows the probability that an observed effect was due to random chance. In other words, if a drug seems to outperform a placebo with an associated p-value of 0.05, there’s only a 5 percent chance that the study was wrong and that the drug is, in fact, no better than the placebo. A p-value of 0.05 is the accepted threshold for validity in most scientific research, even though it’s an arbitrary standard set nearly a century ago by statistician Sir Ronald Fisher. “People don’t often realize that this p-value of 5 percent was pulled out of thin air,” Dr. Palm says. “If Sir Ronald Fisher had had six fingers, we would all be using a p-value of 6 percent.”
The issue, Palm says, is that an arbitrary dividing line of 0.05 leads journal publishers (and paper authors themselves) to reject or ignore real effects that don’t happen to meet the threshold. If a drug trial yields a p-value of more than 0.05, “You should never ever say it is not working,” he tells Harry. “You can only say we were not able to make a determination. That’s it.” By examining the spread of a data set, confidence intervals, data from individuals, and other measures, Palm says, today’s researchers can get a more realistic picture of the promise of new compounds as medicines.