8.9 Marking Written Exams Online

In risr/assess it is possible to set up users of the system to mark Written Exams online with a role of Marker, Senior Marker or Reviewer. These roles are applicable to any system user that has the permission: Exams - Mark Exams assigned to them. The roles are given to users when setting the marking type per Exam, they do not exist in the system as that role permanently.

Markers for Written Exams are not Examiners in risr/assess. They do not have to be entered in the Manage Examiners page, this is only for Examiners in OSCE Exams.

Marker Role

Description

Marker Role

Description

SAQ/MWQ Marker

Enters marks for individual candidates for the assigned Item types.

VSAQ Marker

Marks Item responses rather than individual candidates for the assigned Item types.

Reviewer

Reviews marks confirmed by a marker but can not enter or modify marks.

Senior Marker

Enters a final mark for double marked items when the marks given are outside the the allowable marking discrepancy. Marks within the allowable discrepancy are automatically averaged by the system.

For MWQs, the discrepancy is calculated per part. A senior marker may be required to enter a mark for only one part of an item.

Markers can be assigned to specific Item types which limits their access to the assigned Items only. The assignment of these markers can be done before or after an exam has been delivered. Importantly, allocating markers to an exam before excluding candidates that “Did not attend” will also included excluded candidates in the markers allocated pool. Candidates need to be excluded first to not count towards the total number of scripts to be marked by a marker.

The amount of marking to be completed (marking load) can be shared by multiple markers across the same items with random and anonymous allocation of candidates responses to markers. This can be done by using cohorts to split specific numbers of candidates or items out between specific numbers of markers, or by creating Marking Pools where markers can log in and mark as many Items or Candidates as they wish.

To aid the markers, model answers and marking criteria can be displayed while marking is being completed and the percentage discrepancy can be defined in the Exam setup. Double marking can be performed with a senior marker assigned to resolve discrepancies between markers.

Online marking is available for the following Item types:

Item Type

Available Written Marking Types

Item Type

Available Written Marking Types

  • SAQ (Short Answer Question)

  • MWQ (Multipart Written Question)

  • Single

  • Single with reviewer

  • Double

  • Double with reviewer

  • Double with senior marker

  • VSAQ (Very Short Answer Question)

  • PVSA

  • Single

  • Single with reviewer

Configuration Options for Double Marking Discrepancy

A change was introduced in server versions 8.7 onwards. This has meant there is a change to the default discrepancy (as mentioned above) and this has allowed for customisation of an Instance configuration based on a customer’s individual assessment rules. Below is a table outlining what the original configuration option was (in Version 8.6) and what the possibilities are now, noting the new default. Please raise a ticket with the service desk if you would like to change the configuration of the discrepancy threshold from subpart score to total item score.

Server Version

Senior Marking Required When

Senior Marking is required to mark

What's auto calculated

Where is marking increased

Default 

 

8.6

Total Score breached

All sub parts of MWQ

When marking is within tolerance of the threshold, an auto score is generated for all sub parts and total of item

Item score triggers Senior Marking for all sub-parts

Default in all subsequent versions

 

8.7.x

Configurable - a) Sub-Part Score

When total breached - only part in breach requires marking

When sub-parts of an item are within threshold, the calculated score cannot be changed

Subpart(s) of item outside of threshold trigger Senior marking for each individual subpart

 

 

8.7.x

Configurable - b) Total Item Score

Only part in breach of variance set

All sub-parts of item within threshold have an auto calculated score that is editable

Item score triggers Senior Marking for sub-parts outside of threshold

Configuration of Candidate identifier visible to marker

There is a configuration setting regarding how the candidate is identified to the marker. This setting can be changed by risr/assess, please get in touch through the Service Desk and we will apply the appropriate setting for your organisation.

Setting

Description

Setting

Description

Set to id

Marker sees a marking ID (assignment object). This is a unique candidate ID per exam.

Set to none

Marker sees no identifier.

Set to gmc

Marker sees the GMC number (Candidate ID).

Set to name

Marker sees the candidate name.