Statistical errors are among the most common reasons reviewers raise concerns about an otherwise well-designed medical study. Many of these mistakes aren’t the result of poor science — they’re the result of common misunderstandings about how statistical methods work, and how to interpret their results honestly. This guide covers the errors reviewers flag most often, and how to avoid them.
1. Choosing the Wrong Statistical Test
One of the most fundamental errors is applying a statistical test that doesn’t match the type of data or study design being used — for example, applying a test that assumes normally distributed data to a small, skewed sample, or using an independent-samples test on paired/repeated measurements. Before analysis, confirm your test’s underlying assumptions actually match your data.
2. Misinterpreting the P-Value
A p-value indicates the probability of observing results at least as extreme as those found, assuming the null hypothesis is true — it is not the probability that the null hypothesis itself is true, and it says nothing about the size or clinical significance of an effect. A statistically significant result (p < 0.05) with a tiny, clinically meaningless effect size is a common and avoidable overstatement in medical manuscripts. Reporting effect sizes and confidence intervals alongside p-values gives a far more complete picture.
3. Inadequate Sample Size and Power
Studies that are underpowered — meaning too small to reliably detect a true effect if one exists — risk both false negatives and unstable effect estimates. A power analysis conducted during study design (not after data collection) should justify the target sample size based on an expected effect size, and this justification should be reported in the methods section.
4. Multiple Comparisons Without Correction
Running many statistical tests on the same dataset without adjusting the significance threshold inflates the risk of finding “significant” results purely by chance. When multiple comparisons are unavoidable, corrections such as the Bonferroni method, or a pre-specified primary outcome with clearly labeled secondary/exploratory analyses, help readers correctly interpret which findings are confirmatory versus hypothesis-generating.
5. Confusing Correlation with Causation
Observational study designs — cross-sectional, cohort, or case-control — can establish association but cannot, on their own, establish causation. Manuscripts that use causal language (“X causes Y,” “X leads to Y”) based on observational data are a frequent and easily avoidable reviewer critique. Precise language (“X was associated with Y”) keeps conclusions honest.
6. Selective Reporting of Outcomes
Reporting only the statistically significant outcomes from a study while omitting non-significant ones distorts the overall picture of the research and is considered a form of publication bias. Pre-registering study outcomes, and reporting all pre-specified outcomes regardless of significance, protects against this.
7. Missing Data Handled Incorrectly
Simply excluding participants with missing data (complete-case analysis) without examining why data is missing can introduce bias, particularly if the missingness is related to the outcome being studied. Describing how missing data was handled — and considering appropriate methods like multiple imputation where relevant — strengthens the credibility of the analysis.
8. Overfitting in Predictive Models
Building a statistical or machine learning model with too many variables relative to the sample size risks fitting noise rather than genuine patterns, producing a model that performs well on the original dataset but fails to generalize. Reporting internal and, ideally, external validation results helps reviewers assess whether a model’s performance is real or an artifact of overfitting.
How to Protect Your Manuscript from Statistical Critique
- Involve a statistician early in study design, not just at the analysis stage
- Pre-register your study protocol and primary outcomes where possible
- Report effect sizes and confidence intervals, not just p-values
- Be explicit and honest about study limitations, including power and missing data
- Use precise, non-causal language for observational findings
Why This Matters for Peer Review
Reviewers at IJMS and other rigorous peer-reviewed journals are specifically trained to evaluate whether statistical methods are correctly applied and whether conclusions are proportionate to the evidence. Addressing these common pitfalls before submission — rather than during a revision cycle — meaningfully improves the odds of a smooth review process.
Final Thoughts
Rigorous statistics are not a formality; they are what makes a study’s conclusions trustworthy. Before your next submission, double-check your analysis against these common pitfalls, then review the IJMS Scope and submit through the Paper Submission page.
For detailed statistical reporting guidelines widely used across biomedical journals, see the EQUATOR Network, which maintains reporting standards such as CONSORT, STROBE, and PRISMA.