Pre- and Post-Course Assignments - Revision history

Meghan.Forrest at 06:15, 12 September 2025

2025-09-12T06:15:48Z

← Older revision		Revision as of 06:15, 12 September 2025
Line 137:		Line 137:
	'''Related email templates'''		'''Related email templates'''

	'''''Email:''''' ''Post-course assignment feedback''</blockquote>		'''''Email:''''' ''[[Pre- and Post-Course Assignments: Post-course assignment feedback (Email)\|Post-course assignment feedback]]''</blockquote>
	[[Category:Pre- and Post-Course Assignments]]		[[Category:Pre- and Post-Course Assignments]]

Meghan.Forrest at 06:10, 12 September 2025

2025-09-12T06:10:34Z

← Older revision		Revision as of 06:10, 12 September 2025
Line 137:		Line 137:
	'''Related email templates'''		'''Related email templates'''

	'''''Email:''''' ''~~add template for sending out~~ feedback~~, including individual scores and what happened to the reviewed manuscripts (if already known)~~''</blockquote>		'''''Email:''''' ''Post-course assignment feedback''</blockquote>
	[[Category:Pre- and Post-Course Assignments]]		[[Category:Pre- and Post-Course Assignments]]

Jessica.Rohmann: /* Rationale */

2025-08-22T15:19:57Z

Rationale

← Older revision		Revision as of 15:19, 22 August 2025
Line 35:		Line 35:

	=== Rationale ===		=== Rationale ===
	~~Assess~~ participants~~' growth from~~ the course		To provide participants with specific feedback on their own review reports after they have completed the course.

	=== Assignment description ===		=== Assignment description ===

Jessica.Rohmann: /* Assessment of the pre-course assignment */

2025-08-22T15:19:04Z

Assessment of the pre-course assignment

← Older revision		Revision as of 15:19, 22 August 2025
Line 26:		Line 26:

	=== Assessment of the pre-course assignment ===		=== Assessment of the pre-course assignment ===
	~~[Insert~~ evaluation ~~strategy here]~~		Other than for the purposes of our program evaluation study, we have not performed a formal assessment of the pre-course assignment. Since the purpose of the assignment is to give participants a first experience with receiving a peer review assingment "cold" (before receiving any training), we do not believe that giving them specific feedback on the review itself is valuable at this stage. We recommend that the instructors read through the submitted assignments prior to starting the course to get an idea of the partcipants' levels and degree of familiarity with the reviewing task. We have also incorporated a detailed self-reflection exercise as one of the homework assignments. Here, participants review their own pre-course assignment after having attended the lectures, reflecting on the report's strengths and weaknesses, which is then discussed as a group.

	~~The baseline peer~~-~~review knowledge and ability can be used as a guide~~ to help with ~~assigning~~ the ~~workshop~~ groups.<blockquote>		We have also used the pre-course assignment to help us assign the workshop groups. If you are running a course with multiple workshop groups, you could try to distribute participants who generate stronger baseline review reports evenly between the groups.<blockquote>
	==== Related email templates ====		==== Related email templates ====
	'''''Email:''' [https://peerspectives.premier.charite.de/index.php/Pre-_and_post-course_assignments:_Pre-course_assignment_(Email) Pre-course assignment]''</blockquote>		'''''Email:''' [https://peerspectives.premier.charite.de/index.php/Pre-_and_post-course_assignments:_Pre-course_assignment_(Email) Pre-course assignment]''</blockquote>

Jessica.Rohmann: /* Pre-course Assignment */

2025-08-22T15:11:19Z

Pre-course Assignment

← Older revision		Revision as of 15:11, 22 August 2025
Line 23:		Line 23:
	We assign participants to review one of the four manuscripts at random. They receive an invitation from us via email to review their assigned manuscript as if they were getting the request directly from the journal, with the primary difference being that their review should be submitted to us via return email rather than uploaded via the journal partner's submission system. Beyond what is provided in the journal partner's usual reviewer invitation email, no guidance on how to write a peer-review report is provided. However, participants are told they may use any resources available to them (as do actual invited peer reviewers).		We assign participants to review one of the four manuscripts at random. They receive an invitation from us via email to review their assigned manuscript as if they were getting the request directly from the journal, with the primary difference being that their review should be submitted to us via return email rather than uploaded via the journal partner's submission system. Beyond what is provided in the journal partner's usual reviewer invitation email, no guidance on how to write a peer-review report is provided. However, participants are told they may use any resources available to them (as do actual invited peer reviewers).

	Participants are given a a two-week deadline to complete the review assignment, and they are able to request a one-week extension if needed. Beginning at the two-week deadline, chase emails ~~are sent~~ until the review is received.		Participants are given a a two-week deadline to complete the review assignment, and they are able to request a one-week extension if needed. Beginning at the two-week deadline, we send chase emails every 3 days until the review is received.

	=== Assessment of the pre-course assignment ===		=== Assessment of the pre-course assignment ===

Jessica.Rohmann: /* Assignment description */

2025-08-22T15:09:13Z

Assignment description

← Older revision		Revision as of 15:09, 22 August 2025
Line 19:		Line 19:

	=== Assignment description ===		=== Assignment description ===
	We request four manuscripts from the partner journal that have passed the peer review stage, but have not yet been published from our journal partner. If you are not working ~~without~~ a journal partner, any pre-print or published paper would also be sufficient. If any peer reviews have been published together with the paper, instructors should be aware that students may access/use this content while preparing their assignments.		We request four manuscripts from the partner journal that have passed editorial triage to the peer review stage but have not yet been published from our journal partner. If you are not working with a journal partner, any pre-print or published paper would also be sufficient. If any peer reviews have been published together with the paper, instructors should be aware that students may access/use this content while preparing their assignments.

	~~The~~ participants ~~are randomly assigned~~ to one of the four manuscripts. They ~~are then asked~~ to ~~write a~~ review ~~for one of the four manuscripts~~ as if they were getting the request directly from the journal. ~~They are given little~~ guidance on how to write a peer-review report ~~but~~ are ~~encouraged to~~ use any resources available to them.		We assign participants to review one of the four manuscripts at random. They receive an invitation from us via email to review their assigned manuscript as if they were getting the request directly from the journal, with the primary difference being that their review should be submitted to us via return email rather than uploaded via the journal partner's submission system. Beyond what is provided in the journal partner's usual reviewer invitation email, no guidance on how to write a peer-review report is provided. However, participants are told they may use any resources available to them (as do actual invited peer reviewers).

	Participants ~~have~~ two ~~weeks~~ to complete the review, and they are able to request an extension if needed.		Participants are given a a two-week deadline to complete the review assignment, and they are able to request a one-week extension if needed. Beginning at the two-week deadline, chase emails are sent until the review is received.

	=== Assessment of the pre-course assignment ===		=== Assessment of the pre-course assignment ===

Jessica.Rohmann: /* Assignment description */

2025-08-22T15:04:37Z

Assignment description

← Older revision		Revision as of 15:04, 22 August 2025
Line 19:		Line 19:

	=== Assignment description ===		=== Assignment description ===
	We request four manuscripts from the partner journal that have passed the peer review stage, but have not yet been published from our journal partner. If you are not working without a journal partner, any pre-print or published paper would also be sufficient.		We request four manuscripts from the partner journal that have passed the peer review stage, but have not yet been published from our journal partner. If you are not working without a journal partner, any pre-print or published paper would also be sufficient. If any peer reviews have been published together with the paper, instructors should be aware that students may access/use this content while preparing their assignments.

	The participants are randomly assigned to one of the four manuscripts. They are then asked to write a review for one of the four manuscripts as if they were getting the request directly from the journal. They are given little guidance on how to write a peer-review report but are encouraged to use any resources available to them.		The participants are randomly assigned to one of the four manuscripts. They are then asked to write a review for one of the four manuscripts as if they were getting the request directly from the journal. They are given little guidance on how to write a peer-review report but are encouraged to use any resources available to them.

Jessica.Rohmann: /* Rationale */

2025-08-22T15:02:53Z

Rationale

← Older revision		Revision as of 15:02, 22 August 2025
Line 16:		Line 16:

	=== Rationale ===		=== Rationale ===
	To ~~gauge~~ the ~~baseline~~ peer review ability of incoming course participants and to provide participants with a simulated real-world experience of being invited to review a paper at the partnering journal.		To provide the instructor with an idea of the peer review ability of incoming course participants prior to starting the course and to provide participants with a simulated real-world experience of being invited to review a paper at the partnering journal.

	=== Assignment description ===		=== Assignment description ===

Jessica.Rohmann: /* Rationale */

2025-08-22T15:01:44Z

Rationale

← Older revision		Revision as of 15:01, 22 August 2025
Line 16:		Line 16:

	=== Rationale ===		=== Rationale ===
	~~Gauge~~ the baseline peer review ability of participants		To gauge the baseline peer review ability of incoming course participants and to provide participants with a simulated real-world experience of being invited to review a paper at the partnering journal.

	=== Assignment description ===		=== Assignment description ===

Toivo.Glatz at 14:59, 22 August 2025

2025-08-22T14:59:07Z

← Older revision		Revision as of 14:59, 22 August 2025
Line 40:		Line 40:
	We use the same four manuscripts that were distributed as the pre-course assignment. For the post-course assignment, we recommend that each participant be assigned one of the four pre-course assignments that they have not previously reviewed.		We use the same four manuscripts that were distributed as the pre-course assignment. For the post-course assignment, we recommend that each participant be assigned one of the four pre-course assignments that they have not previously reviewed.

	Ideally, based on the timeframe of the course, the manuscripts that the participants have reviewed for the pre- and post- course assignments will be published shortly after the completion of the course. The participants will then be able to compare their own reviews to the published reviews for the papers (if partnering with an open review journal).		Ideally, based on the timeframe of the course, the manuscripts that the participants have reviewed for the pre- and post- course assignments will be published shortly after the completion of the course. The participants will then be able to compare their own reviews to the published reviews for the papers (if partnering with an open review journal).<blockquote>
			==== Related email templates ====
			'''''Email:''' [https://peerspectives.premier.charite.de/index.php/Pre-_and_post-course_assignments:_Post-course_assignment_(Email) Post-course assignment]''</blockquote>

	=== Assessment of the post-course assignment ===		=== Assessment of the post-course assignment ===
	[~~Insert evaluation strategy here~~]		The teaching team provides feedback on the post-course assignment following submission. We developed a Review Report Quality Score, with items adapted from previous studies <ref>https://kb.wisc.edu/instructional-resources/page.php?id=114199</ref><ref>https://peerj.com/articles/17514/#supp-1</ref> and tailored towards the specific criteria of The BMJ ['''maybe also cite?]'''. The following table outlines the 13 items, divided into general and specific criteria, each worth 2 points with the exception of the last language item which is only worth 1 point.
	{\| class="wikitable"		{\| class="wikitable"
	\|+		\|+
Line 95:		Line 97:
	\|}		\|}

	~~Items adapted from:~~		==== Scoring ====
			Each item is scored based on the following criteria, awarding 0, 1 or 2 points depending on the completeness of reporting. No partial points are awarded.
	* <nowiki>https://kb.wisc.edu/instructional-resources/page.php?id=114199</nowiki>
	* <nowiki>https://peerj.com/articles/17514/#supp-1</nowiki>

	==== Scoring: ====
	{\| class="wikitable"		{\| class="wikitable"
	\|+		\|+
Line 116:		Line 114:
	\|}		\|}

	==== Feedback: ====		==== Feedback ====
			To provide an interpretation of the overall review report quality, the points of all 13 items are summed up to a total score, which falls into one of the four categories outlined in the table below. This should give students an immediate impression on how they performed and if their work was up to the standard that BMJ editors expect to see in high quality reviews. The course instructors can add additional comments after these standard evaluation if there are notable positive or negative aspects, e.g., praising thorough checks of supplement, study protocol and trial registration or advising against the use of inappropriate language, etc.
	{\| class="wikitable"		{\| class="wikitable"
	\|+		\|+
Line 135:		Line 134:
	\|}		\|}

			Students receive an email with the individual scoring of all items, as well as the overall feedback and any additional comments (if applicable). In case the reviewed papers are still currently under consideration at the journal partner, reports that scored >20 points can be forwarded to the journal as an actual review report. In cases where manuscripts have been published in the meantime, a link to the paper will be shared with the student.<blockquote>
			'''Related email templates'''

			'''''Email:''''' ''add template for sending out feedback, including individual scores and what happened to the reviewed manuscripts (if already known)''</blockquote>
	~~The teaching team provides feedback on the post-course assignment following submission.<blockquote>~~
	~~==== Related email templates ====~~
	'''''Email:''' ~~[https://peerspectives.premier.charite.de/index.php/Pre-_and_post-course_assignments:_Post-course_assignment_~~(~~Email~~) ~~Post-course assignment]~~''</blockquote>
	[[Category:Pre- and Post-Course Assignments]]		[[Category:Pre- and Post-Course Assignments]]