Chapter 15 Checklists

by Jade Benjamin-Chung

15.1 Pre-analysis plan checklist

  • Brief background on the study (a condensed version of the introduction section of the paper)
  • Hypotheses / objectives
  • Study design
  • Description of data
  • Definition of outcomes
  • Definition of interventions / exposures
  • Definition of covariates
  • Statistical power calculation
  • Statistical model description
  • Covariate selection / screening
  • Standard error estimation method
  • Missing data analysis
  • Assessment of effect modification / subgroup analyses
  • Sensitivity analyses
  • Negative control analyses

15.2 Code checklist

  • Does the script run without errors?
  • Is code self-contained within repo and/or associated Box folder?
  • Is all commented out code / remarks removed?
  • Does the header accurately describe the process completed in the script?
  • Is the script pushed to its github repository?
  • Does the code adhere to the coding style guide?
  • Are all warnings ignorable? Should any warnings be intentionally suppressed or addressed?

15.3 Manuscript checklist

This is adapted in part from this article.

  • Have you completed the relevant reporting checklist, if applicable? (Collection of checklists)
  • Are the study results within the manuscript replicable (i.e., if you rerun the code in the study’s repository, the tables and figures will be exactly replicated?)
  • Is a target journal selected?
  • Is the title declarative, in other words, does it state the object/findings rather than suggest them?
  • Is the word count of the manuscript close to the target journal’s allowance?
  • Does the manuscript adhere to the formatting guide of the target journal?
  • Does the manuscript use a consistent voice (passive or active – usually active is preferred … pun intended)?
  • Is each figure and table (including supplementary material) referenced in the main text?
  • Is there a caption for each figure and table (including supplementary material)?
  • Are tables/figures and supplementary material numbered in accordance with their appearance in the main text?
  • Does the text use past tense if it is reporting research findings or future tense if it is a study protocol?
  • Does the text avoid subjective wording (e.g., “interesting”, “dramatic”)?
  • Does the text use minimal abbreviations, and are all abbreviations defined at first use?
  • Does the text avoid directionless words? (e.g., instead of writing, ‘Precipitation influences disease risk’, write, ‘Precipitation was associated with increased disease risk’).
  • Does the text avoid making causal claims that are not supported by the study design? Be careful about the words “effect”, “increase”, and “decrease”, which are often interpreted as causal.
  • Does the text avoid describing results with the word “significant”, which can easily be confused with statistical significance? (see references on this topic here)
  • Have you drafted author contributions? Do they follow the CRediT Taxonomy for author contributions?

15.4 Figure checklist

  • Are the x-axis and y-axis labeled?
  • If the figure includes panels, is each panel labeled?
  • Are there sufficient numerical / text labels and breaks on the x-axis and y-axis?
  • Is the font size appropriate (i.e., large enough to read, not so large that it distracts from the data presented in the figure?)
  • Are the colors used colorblind friendly? See a colorblind-friendly palette here, a neat palette generator with colorblind options here, and an article on why this matters here
  • Are colors/shapes/line types defined in a legend?
  • Are the legends and other labels easy to understand with minimal abbreviations?
  • If there is overplotting, is transparency used to show overlapping data?
  • Are 95% confidence intervals or other measures of precision shown, if applicable?