Pre- and Post-Course Assignments

Notes

The pre- and post-course assignments have been designed to simulate a "real" peer-review process. The emails that are sent to the participants about the pre- and post-course assignments include an introduction to the assignment and are then followed by a peer review request that has been adapted from requests used by the journal partner. The assignments can use real manuscripts that have passed peer review but are not yet published, pre-prints, or papers that have already been published.

Pre-course Assignment

Rationale

Gauge the baseline peer review ability of participants

Assignment description

We request four manuscripts from the partner journal that have passed the peer review stage, but have not yet been published from our journal partner. If you are not working without a journal partner, any pre-print or published paper would also be sufficient.

The participants are randomly assigned to one of the four manuscripts. They are then asked to write a review for one of the four manuscripts as if they were getting the request directly from the journal. They are given little guidance on how to write a peer-review report but are encouraged to use any resources available to them.

Participants have two weeks to complete the review, and they are able to request an extension if needed.

Assessment of the pre-course assignment

[Insert evaluation strategy here]

The baseline peer-review knowledge and ability can be used as a guide to help with assigning the workshop groups.

Related email templates

Email: Pre-course assignment

Post-course Assignment

Rationale

Assess participants' growth from the course

Assignment description

We use the same four manuscripts that were distributed as the pre-course assignment. For the post-course assignment, we recommend that each participant be assigned one of the four pre-course assignments that they have not previously reviewed.

Ideally, based on the timeframe of the course, the manuscripts that the participants have reviewed for the pre- and post- course assignments will be published shortly after the completion of the course. The participants will then be able to compare their own reviews to the published reviews for the papers (if partnering with an open review journal).

Assessment of the post-course assignment

[Insert evaluation strategy here]

	Peerspectives Review Report Quality Score (Criteria specific for The BMJ)	Points possible
Specific Content	Comments on study objectives, rationale	2
	Comments on methods-related aspects (e.g. study design, statistical analyses, sampling, suitability of reported effect sizes/confidence intervals/p-values, reporting quality - relevant for replicability/reproducibility)	2
	Comments on tables, figures, visual aspects of the manuscript	2
	Comments on the interpretation of results and whether the conclusions are supported by the data	2
	Comments on specific manuscript strengths (study, theory, methods, argument) and limitations (including biases)	2
	Comments on manuscript structure, flow, writing style and/or language	2
General items	Structure: includes a short summary as well as some specific comments	2
	Specific reflection on importance, relevance and suitability, also given the journal's specific scope/readership	2
	Focus on “global concerns” (larger structural, logic/reasoning issues) rather than detailed “local concerns” (spelling, grammar, formatting)	2
	Thorough, respectful, constructive critique, including positive and negative comments, also offering suggestions rather than simply labeling problems	2
	Appropriate comment density, overall review length demonstrates the reviewer’s investment in peer review while not overwhelming the reader	2
	Filled out reviewer form completely, including declaring any competing interests or no competing interests	2
	Review is written in understandable English (language, meaning of statements clear, few, if any, distracting typos/grammatical mistakes) - 1 point only	1

The teaching team provides feedback on the post-course assignment following submission.

Related email templates

Email: Post-course assignment