Revision of the ARRIVE guidelines: rationale and scope
Scientists, funders and the public are increasingly concerned about the reproducibility of preclinical research, including studies that use animals.1 While the reasons for failing to reproduce the methods and findings of a study are complex and wide ranging, a lack of transparency stemming from poor reporting clearly contributes to the problem.2 The NC3Rs coordinated the development of the Animal Research: Reporting of In Vivo Experiments (ARRIVE) reporting guidelines in 2010.3 The guidelines consist of a 20-item checklist that covers the key information that should be described in a scientific publication. The goal is to ensure that the reader can assess the methodological rigour of the experiment, and other scientists can evaluate and reproduce the methods.
To date, over a thousand journals, funders and research institutes support and endorse ARRIVE.4 The guidelines have contributed to the understanding of the issues that compromise reproducibility, and prompted actions to improve the situation. For example, major UK funders now explicitly mandate a comprehensive description of the study design, including plans to minimise experimental bias. Grant applicants must also explain how the number of animals to be used was decided and provide detailed statistical analysis plans to ensure that peer reviewers and panel members can fully assess the rigour and validity of the proposed research.5
Have the guidelines improved reporting? In the 7 years since the ARRIVE guidelines were published, researchers have sought to measure the impact of the guidelines on the quality of reporting,6 7 with mixed results. A recent randomised controlled trial in PLOS ONE 8 showed that mandating the completion of an ARRIVE checklist with manuscript submission, with no additional emphasis on reporting during the editorial process, did not improve adherence to the guidelines in published papers. While these results are disappointing to those seeking immediate change, this study provides an evidence base to improve the guidelines, and ultimately the rigour and reproducibility of animal studies.
In the light of methodological advances in science and experience with the guidelines since their introduction, the NC3Rs convened an international working group to revise them. The authors here are members of the working group, and include funders, journal editors, statisticians, methodologists and animal researchers from academia and industry. The aim of this report is to provide readers and stakeholders with information about areas that we are currently working on to improve the ARRIVE guidelines during 2018. This work includes:
Prioritising the items of the ARRIVE guidelines
Each of the 20 items of the ARRIVE guidelines are important for various reasons. For example, a description of study design (item 6), how the animals were allocated to groups (item 11) and how the sample size was chosen (item 10) are crucial to understand how reliable and robust the findings are. Similarly, items such as the experimental procedures (item 7) or animal characteristics (item 8) are important to ensure that papers contain enough information for others to replicate and build upon the study. Other ARRIVE items such as the scientific background (item 3) and relevance to other species (item 19) provide information about the context of the study.3 In their current form, the guidelines do not lend themselves easily to retrospective evaluation; assessing whether a manuscript includes all 20 ARRIVE items necessitates operationalising the checklist into over a hundred separate elements.9 To enable a more manageable approach for assessing the quality of reporting in manuscripts, we plan to organise the items in the ARRIVE guidelines into tiers reflecting different levels of priority; tier 1 items will include the most important items on which initial efforts from authors, reviewers and journals should focus. We will carry out a Delphi exercise,10 to structure communications within the working group and with external stakeholders, and reach a consensus on the criteria defining the tiers, and on the most appropriate tier for each item. Importantly, the tiers will also enable a stepwise approach for journals and others to improve reporting standards, the objective being that ultimately all manuscripts will include all elements of the guidelines.
Prioritising subsets of items in this way will provide straightforward measures for journals, institutions and researchers. We anticipate that journals will continue to recommend that authors follow the ARRIVE guidelines in their entirety, to encourage comprehensive reporting. At the same time, focusing editorial efforts on a smaller number of key pieces of information that can be particularly scrutinised by editors and reviewers will enable a more rapid assessment of both individual manuscript quality, and the overall impact of their improvement strategies. This is an approach already used by some journals.11 12 With the use of text mining and machine learning technologies, automating many of these checks is possible,13 and coordinating work on top tier items will accelerate the development of tools to facilitate this.
Developing an Explanation and Elaboration document
Understanding the rationale for a set of guidelines is essential for securing support from the scientific community. The Consolidated Standards of Reporting Trials (CONSORT) statement, for example, has been accompanied by an Explanation and Elaboration document since its second iteration in 2001.14 This summarises the evidence behind each item of the guidelines and explains why each item is important to include in a manuscript.
A recent survey of in vivo researchers carried out by the NC3Rs15 highlighted that the main reason for authors not including an ARRIVE item in a manuscript was because they did not think it was necessary to disclose that information. Incomplete reporting is also exacerbated by the fact that some of the concepts included in the guidelines, such as measures to reduce bias, are not well understood by researchers or not considered relevant for their own research.16 To address this, we are now developing an Explanation and Elaboration document for the ARRIVE guidelines. This document will provide explanations and definitions for technical terms, empirical evidence in support of each ARRIVE item, as well as examples from the published literature on how authors might report items. Following publication of the document, the information will be made readily accessible via a dedicated ARRIVE website.
Revising the guidelines
We are reviewing specific ARRIVE items to ensure that the guidance provided is in line with the current best evidence. Where evidence is lacking we will seek to develop it. The revision is an opportunity to improve the clarity of individual items, ensure their relevance across the breadth of in vivo research and enhance the logical flow of information within the guidelines. In recent years, scientific organisations such as publishers and funders have produced additional guidance to improve the reporting of preclinical research (eg, NINDS’s call for greater transparency in preclinical research,17 NIH’s Principles and Guidelines for Reporting Preclinical Research,18 Nature’s Reporting Life Sciences Research checklist,19 Cell guidelines,20 British Journal of Pharmacology’s guidance for reporting experimental design and analysis21). Such guidelines will be taken into consideration in the revision. The ARRIVE guidelines are not intended to supersede journal or model-specific guidelines but the level of support from funders and journals puts ARRIVE in a unique position to serve as the basis for more specialised guidelines. The publishing landscape has also evolved over the last decade and the revised guidelines will reflect these changes by providing advice on emerging best practice. Additionally, external stakeholders with expertise in preclinical research reporting will be able to suggest further revisions via the above-mentioned Delphi exercise.
The scope of the ARRIVE guidelines is broad; they are designed to be flexible and accommodate the reporting of comparative studies in a wide range of research areas. As such, the existing guidelines were formulated to provide general advice for heterogeneous study types. However, recent calls have been made to encourage researchers to explicitly distinguish between exploratory and hypothesis-testing studies.22 23 For hypothesis-testing studies that are using inferential statistics, the manuscript would be expected to describe the primary and secondary outcome measures, the parameters used in the sample size calculation, whether the study protocol was preregistered, and if so, where it can be found. Exploratory studies, on the other hand, are designed to generate hypotheses; they might confer the same importance to all outcomes measured, might justify the sample size based on feasibility or experience and would report only descriptive statistics. Thus, the reporting requirements for exploratory and hypothesis-testing studies can differ, and the revision will ensure that the updated guidelines provide adequate advice for both.
Final remarks
Revising the guidelines is just the first step; their primary goal is to improve transparency and the standards of reporting, but transparent reporting can be used to address common weaknesses in the design and conduct of animal research and encourage researchers to adopt more rigorous scientific practices. Ultimately, the ARRIVE guidelines will form the basis for a powerful suite of tools and resources that provide optimal support for researchers to improve the design, conduct and reporting of in vivo research; this will also benefit research users and stakeholders tasked with assessing the quality and translational value of preclinical research.
Improving reporting should be a community-wide effort, and the working group recognises the importance of engaging others in the evolution of the ARRIVE guidelines. It is essential that this endeavour includes scientists from a range of research fields and countries, and we encourage the community to share their experience and views.
Acknowledgments
We would like to acknowledge the late Doug Altman’s instrumental contribution to this project. Doug collaborated with the NC3Rs on the work leading up to the development of the ARRIVE guidelines, he was an author on the original guidelines and an active member of the present working group to revise them. It was a pleasure to work with him over the years and he will be sorely missed.