Web-based Tool to Improve the Assessment of Reporting (COBPeer)

September 10, 2018 updated by: Isabelle BOUTRON, Assistance Publique - Hôpitaux de Paris

Evaluation of an Online Tool (COBPeer) Dedicated to Junior Peer Reviewers for Assessing the Reporting of Randomized Controlled Trial Reports

The peer review process is a cornerstone of biomedical research publication. Despite being essential, the assessment of the completeness of the reporting and the identification of switched outcomes are not appropriately performed. Furthermore, these tasks do not suppose a high level of expertise and could be performed by junior researchers.

To assess the completeness of reporting and identified switched outcome(s), junior peer reviewers could use a simple online tool (COBPeer) based on the CONSORT (Consolidated Standards of Reporting Trials) 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCTs. The tool would feature bullet points eliciting the meaning of each checklist item. The aim of this study that will compare the accuracy of junior peer reviewers using the tool to that of usual peer reviewers when evaluating the completeness of reporting and switched of primary outcome(s) in reports of RCTs

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

BACKGROUND: Context Inadequate reporting is a frequent cause of waste of research. To overcome this issue, the CONSORT statement, an evidence-based, minimum set of recommendations for reporting RCTs was developed in 1996. These guidelines have since been updated in 2001 and more recently in 2010. Many journals endorse the CONSORT statement. Some journals provide recommendations to authors to follow the CONSORT guidelines and some editors enforce the use of the CONSORT guidelines by requesting authors to submit a checklist in either the submission or acceptance stage. Nevertheless, inadequate reporting remains.

HYPOTHESE: To assess the completeness of reporting, junior peer reviewers could use a simple online tool based on the CONSORT 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCT

OBJECTIVE: 1) Develop an online tool and training module dedicated to junior peer reviewers for a) assessing the completeness of reporting of key items and b) identifying switched primary outcome(s) in reports of RCTs. 2) Compare the performance of junior peer reviewers using this tool with the usual peer-review process in identifying inadequate reporting and switched primary outcome(s) in reports of RCTs.

STUDY DESIGN: randomized ratio 1:1 controlled trial with two parallel arms. Participants will be invited to participate by invitation.

ELIGIBILITY: eligible participants are juniors peer reviewers. Junior peer reviewers are defined as early stage researchers: master students, PhD students, residents involved in clinical research during their study, and clinicians who have never reviewed a manuscript

INTERVENTION: the peer-review tool will be based on 10 items of the CONSORT Statement. This tool reminds the peer reviewer of the CONSORT item which must be reported by explicating it with a series of questions. The training module will explain each item to be evaluated with examples followed by a series of exercises from published randomized trial extracts.

MANUSCRIPT SELECTION: sample of randomized controlled trials published by Annals of Emergency Medicine, British Medical Journal, British Medical Journal Open and BioMed Central series medical.

EVALUATION BY THE JUNIOR PEER REVIEWERS: the junior peer reviewer who has accepted to participate and who has passed the training with success, will have to evaluate 1 article of the selected randomized sample.

EVALUATION OF THE USUAL PEER REVIEWERS: the evaluation by the usual peer-reviewers will be obtained by an analysis of the comments of the peer-reviewers which are available on line or obtained from the publisher.

GOLD STANDARD: peers of researchers who are experts in conducting systematic reviews will evaluate each article independently from other evaluations. Disagreements will be resolved by consensus.

PRIMARY OUTCOME: the mean number of items accurately classified per manuscript

SECONDARY OUTCOMES: the mean number of items accurately classified per manuscript for the 10 CONSORT items; the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).

SAMPLE SIZE: 120 manuscripts and 120 peer reviewers juniors.

Study Type

Interventional

Enrollment (Actual)

120

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • France
      • Paris, France, 75004
        • Hotel Dieu, 1, place du parvis de notre dame

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

18 years and older (ADULT, OLDER_ADULT)

Accepts Healthy Volunteers

Yes

Genders Eligible for Study

All

Description

Inclusion Criteria:

  • master students,
  • PhD students,
  • residents involved in clinical research during their study,
  • clinicians who have never reviewed a manuscript

Exclusion Criteria:

  • none

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: OTHER
  • Allocation: NON_RANDOMIZED
  • Interventional Model: PARALLEL
  • Masking: SINGLE

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
NO_INTERVENTION: USUAL PEER REVIEWER
Two researchers will read all the peer-review reports and editors' comments for the first round. The researchers will determine whether the peer-reviewers and/or editors raised some concern on the completeness of reporting of the 10 CONSORT items considered and identified a switch primary outcome(s) between the manuscript and the register. The assessment of all peer-review reports and editors' comments for each manuscript will be combined (i.e., the item will be rated as incompletely reported if at least one peer reviewer rated it as such). The 2 researchers will be blinded to the gold standard assessment.
EXPERIMENTAL: COBPeer
Junior peer reviewers will be invited to participate in an online training course on peer review (COBPeer).
Other: COBPEER
At the end of the online training, they can participate in the study and so peer review 1 manuscript randomly selected from our sample with our tool (COBPeer).
NO_INTERVENTION: GOLD STANDARD
Pairs of systematic reviewers will independently extract data from eligible reports. Reviewers involved in the data extraction will have expertise in the conduct of systematic reviews and will assess the completeness of reporting from the systematic reviewer perspective. They will not have access to the tool to avoid being influenced by the tool. The systematic reviewers will also systematically compare the primary outcome(s) reported in the manuscript and the primary outcome(s) reported in the registry and will document any discrepancies.

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
the mean number of items accurately classified per manuscript
Time Frame: through study completion, an average of 6 month
Each item will be classified as"adequately reported" (yes/no)
through study completion, an average of 6 month

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
the mean number of items accurately classified per manuscript for the 10 CONSORT items
Time Frame: through study completion, an average of 6 month
Each item will be classified as"adequately reported" (yes/no)
through study completion, an average of 6 month
the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).
Time Frame: through study completion, an average of 6 month
test for pair proportion
through study completion, an average of 6 month

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Assistance Publique - Hôpitaux de Paris

Collaborators

University of California, Los Angeles
University of Oxford
University of Ottawa

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (ACTUAL)

December 10, 2017

Primary Completion (ACTUAL)

January 20, 2018

Study Completion (ACTUAL)

January 20, 2018

Study Registration Dates

First Submitted

April 3, 2017

First Submitted That Met QC Criteria

April 13, 2017

First Posted (ACTUAL)

April 18, 2017

Study Record Updates

Last Update Posted (ACTUAL)

September 11, 2018

Last Update Submitted That Met QC Criteria

September 10, 2018

Last Verified

September 1, 2018

More Information

Terms related to this study

Keywords

Other Study ID Numbers

  • AC001

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Peer Review

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in Equatorial Guinea

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

3
Subscribe