Chapter 18 Checklists

by Jade Benjamin-Chung

18.1 Pre-analysis plan checklist

Before registering a pre-analysis plan, verify that it includes all elements described in the Write and register pre-analysis plans section. See the following pre-analysis plans for examples from our lab:

18.2 Code checklist

  • Does the script run without errors?
  • Is code self-contained within repo and/or associated Box folder?
  • Is all commented out code / remarks removed?
  • Does the header accurately describe the process completed in the script?
  • Are file paths relative (using here()) rather than absolute?
  • Are assertions present after merges, filters, and key variable creation steps?
  • Do unit tests pass (testthat::test_dir('9-tests'))?
  • Are random seeds set where relevant?
  • Is the script pushed to its github repository?
  • Does the code adhere to the coding style guide?
  • Are all warnings ignorable? Should any warnings be intentionally suppressed or addressed?

18.3 Manuscript checklist

This is adapted in part from this article.

  • Have you completed the relevant reporting checklist, if applicable? (Collection of checklists)
  • Are the study results within the manuscript replicable (i.e., if you rerun the code in the study’s repository, the tables and figures will be exactly replicated?)
  • Is a target journal selected?
  • Is the title declarative, in other words, does it state the object/findings rather than suggest them?
  • Is the word count of the manuscript close to the target journal’s allowance?
  • Does the manuscript adhere to the formatting guide of the target journal?
  • Does the manuscript use a consistent voice (passive or active – usually active is preferred … pun intended)?
  • Is each figure and table (including supplementary material) referenced in the main text?
  • Is there a caption for each figure and table (including supplementary material)?
  • Are all numbers in the text (sample sizes, point estimates, CIs, p-values) generated programmatically from the analysis output rather than manually typed?
  • Are tables/figures and supplementary material numbered in accordance with their appearance in the main text?
  • Does the text use past tense if it is reporting research findings or future tense if it is a study protocol?
  • Does the text avoid subjective wording (e.g., “interesting”, “dramatic”)?
  • Does the text use minimal abbreviations, and are all abbreviations defined at first use?
  • Does the text avoid directionless words? (e.g., instead of writing, ‘Precipitation influences disease risk’, write, ‘Precipitation was associated with increased disease risk’).
  • Does the text avoid making causal claims that are not supported by the study design? Be careful about the words “effect”, “increase”, and “decrease”, which are often interpreted as causal.
  • Does the text avoid describing results with the word “significant”, which can easily be confused with statistical significance? (see references on this topic here)
  • Have you drafted author contributions? Do they follow the CRediT Taxonomy for author contributions?

18.4 Figure checklist

  • Are the x-axis and y-axis labeled?
  • Are the axis limits appropriate (e.g., not truncating data or exaggerating differences)?
  • If the figure includes panels, is each panel labeled?
  • Are there sufficient numerical / text labels and breaks on the x-axis and y-axis?
  • Is the font size appropriate (i.e., large enough to read, not so large that it distracts from the data presented in the figure?)
  • Are the colors used colorblind friendly? See a colorblind-friendly palette here, a neat palette generator with colorblind options here, and an article on why this matters here
  • Are colors/shapes/line types defined in a legend?
  • Are the legends and other labels easy to understand with minimal abbreviations?
  • If there is overplotting, is transparency used to show overlapping data?
  • Are 95% confidence intervals or other measures of precision shown, if applicable?
  • Does the figure have a consistent theme/style with other figures in the manuscript?