In this month’s message, I’d like to emphasize the importance of transparent, reproducible data. Regardless of whether your interests involve clinical, translational or basic science research, we are all dependent upon, and driven by, data.
Data that we generate must be reproducible by other groups if research is to make an impact on the human condition. The lack of reproducibility has been a stumbling block for pharmaceutical firms that rely on published papers of preclinical research to form initiatives for new therapeutics (see http://www.nature.com/nature/journal/v483/n7391/full/483531a.html).
Recently, the NIH has expressed concern about the lack of reproducibility of published results, and four areas related to reproducibility will now need to be specifically addressed in NIH proposals (see: http://grants.nih.gov/reproducibility/index.htm). This discussion should make us all consider how we analyze the raw data from our experiments.
In our laboratory, we have a weekly 2 to 2 ½ hour marathon session that is about 20 percent teaching and 80 percent data analysis and experimental design. In particular, we like to see the results from single experiments, including the raw data and the variance from the mean in the replicates. I don’t like seeing a composite figure all “prettied up,” until the data is examined.
In my opinion, one of the best ways to minimize the chance of publishing erroneous data is for the principal investigator to examine the raw data. In addition, clear instructions on the criteria for deleting a data point, or an entire experiment, as an “outlier” must be established for each experiment.
The lab’s team members come from different backgrounds and training, and it is incumbent on the PI to establish how data is collected, analyzed, interpreted, and presented. The tendency to show the PI only the data that “looks good” (and fits the hypothesis) is highly entrenched in some cultures, and it is our responsibility to show that we do science for discovery, regardless of what the results demonstrate.
There are countless stories about researchers entering a project with a preconceived notion of the outcomes, and ultimately being surprised with unexpected and novel findings. That is the thrill of science, and it can only be had by meticulous attention to the data.
Sincerely,
Stephen Liggett, MD
Vice Dean for Research
Professor of Medicine, Molecular Pharmacology and Physiology
USF Health Morsani College of Medicine
