The incorrect use of statistics in scientific articles seems to be a never-ending discussion topic. A current controversy involves a decision by Basic and Applied Social Psychology in 2015 to ban the use of P-values (i.e., null hypothesis testing) in articles appearing in their journal. This prompted the American Statistical Association to publish, in 2016, a policy statement on the use of P-values in research publications. Reinhart (2015) in his book, Statistics Done Wrong: The Woefully Complete Guide, gives a good overview of the sorts of statistical mistakes made in science, with many biological examples.
There are also attempts to gauge how severe the misuse of statistics is in various biological disciplines. The article on the website hosted by influentialpoints.com (Dransfield and Brightwell, 2012) provides an overall guide to statistics misuse in biology, with a bias toward medicine. The authors of this site categorized errors found in an examination of “several thousand papers” and the article posted is abstracted from their book (Brightwell and Dransfield, 2013).
A recent evaluation of incorrect analyses of interaction effects in the neurosciences found that about half the published articles had statistical issues when analyzing factorial treatment designs, with some apparently severe enough to call the study’s conclusions into question (Nieuwenhuis et al., 2011). A recent Nature article by Allison et al. (2016) discussed how easy it was to find mistakes in data handling in publications, but how hard it was to get them fixed. Although there are many reasons why a statistical analysis may or may not be appropriate, only those most applicable to horticulture will be discussed below.
We examined issues of the JASHS published between Jan. 2014 and Jan. 2015 inclusive, for statistical problems. This was prompted by an interest in revising the currently antiquated instructions to authors about the use of statistics in the society’s journals. To do this, we needed to identify the kinds of statistical methodologies required by current authors to support their findings, the kinds of data being collected, and what authors were actually doing when analyzing the data. The revised version of statistics instructions will be appearing separately. Here, we describe the kinds of statistical errors most commonly made by authors in this journal and characterize the patterns of errors and omissions we found. These are not necessarily fatal flaws, but reveal weaknesses that may affect conclusions. We then ascribe probable causes and suggest some possible remedies. We hope this review will be helpful both to authors and reviewers.
BenjaminiY.HochbergY.1995Controlling the false discovery rate: A practical and powerful approach to multiple testingJ. R. Stat. Soc. B.57289300
BrightwellR.DransfieldR.D.2013Avoiding and detecting statistical malpractice: Design and analysis for biologists with R. 25 May 2016. <http://influentialpoints.com/aboutus.htm>
DransfieldR.D.BrightwellR.2012Statistical mistakes in research: Use and misuse of statistics in biology. 16 Nov. 2015. <http://influentialpoints.com/Training/statistical_mistakes_in_research_use_and_misuse_of_statistics_in_biology.htm>
FatimaT.SobolevA.P.TeasdaleJ.R.KramerM.BunceJ.HandaA.K.MattooA.K.2016Fruit metabolite networks in engineered and non-engineered tomato genotypes real fluidity in a hormone and agroecosystem specific mannerMetabolomics12103
HochbergY.TamhaneA.C.1987Multiple comparison procedures. Wiley New York NY
JohnsonR.A.WichernD.W.2007Applied multivariate statistical analysis. 3rd ed. Pearson New York NY
LittellR.C.MillikenG.A.StroupW.W.WolfingerR.D.SchabenbergerO.2006SAS for mixed models. 2nd ed. SAS Institute Cary NC
MillikenG.A.JohnsonD.E.2009Analysis of messy data: Designed experiments. Vol. 1 2nd ed. Chapman and Hall/CRC New York NY
NieuwenhuisS.ForstmannB.U.WagenmakersE.J.2011Erroneous analyses of interactions in neuroscience: A problem of significanceNat. Neurosci.1411051107
R Core Team2013R: A language and environment for statistical computing. 25 May 2016. <http://www.R-project.org/>
ReinhartA.2015Statistics done wrong: A woefully complete guide. No Starch Press San Francisco CA
SchlotterY.M.VeenhofE.Z.BrinkhofB.RuttenV.P.SpeeB.WillemseT.PenningL.C.2009A GeNorm algorithm-based selection of reference genes for quantitative real-time PCR in skin biopsies of healthy dogs and dogs with atopic dermatitisVet. Immunol. Immunopathol.129115118
UC3 Data Pub Blog2012Archiving data best practices data sharing. 20 Apr. 2016. <https://datapub.cdlib.org/2012/11/20/thanks-in-advance-for-sharing-your-data/>
WestfallP.H.YoungS.S.1993Resampling-based multiple testing: Examples and methods for P-value adjustment. Wiley New York NY