8.9 Marking Written Exams Online
- Miles Skinner (Unlicensed)
- El Pascoe
In risr/assess it is possible to set up users of the system to mark Written Exams online with a role of Marker, Senior Marker or Reviewer. These roles are applicable to any system user that has the permission: Exams - Mark Exams assigned to them. The roles are given to users when setting the marking type per Exam, they do not exist in the system as that role permanently.
Markers for Written Exams are not Examiners in risr/assess. They do not have to be entered in the Manage Examiners page, this is only for Examiners in OSCE Exams.
Marker Role | Description |
|---|---|
SAQ/MWQ Marker | Enters marks for individual candidates for the assigned Item types. |
VSAQ Marker | Marks Item responses rather than individual candidates for the assigned Item types. |
Reviewer | Reviews marks confirmed by a marker but can not enter or modify marks. |
Senior Marker | Enters a final mark for double marked items when the marks given are outside the the allowable marking discrepancy. Marks within the allowable discrepancy are automatically averaged by the system. For MWQs, the discrepancy is calculated per part. A senior marker may be required to enter a mark for only one part of an item. |
Markers can be assigned to specific Item types which limits their access to the assigned Items only. The assignment of these markers can be done before or after an exam has been delivered. Importantly, allocating markers to an exam before excluding candidates that “Did not attend” will also included excluded candidates in the markers allocated pool. Candidates need to be excluded first to not count towards the total number of scripts to be marked by a marker.
The amount of marking to be completed (marking load) can be shared by multiple markers across the same items with random and anonymous allocation of candidates responses to markers. This can be done by using cohorts to split specific numbers of candidates or items out between specific numbers of markers, or by creating Marking Pools where markers can log in and mark as many Items or Candidates as they wish.
To aid the markers, model answers and marking criteria can be displayed while marking is being completed and the percentage discrepancy can be defined in the Exam setup. Double marking can be performed with a senior marker assigned to resolve discrepancies between markers.
Online marking is available for the following Item types:
Item Type | Available Written Marking Types |
|---|---|
|
|
|
|
Configuration Options for Double Marking Discrepancy
A change was introduced in server versions 8.7 onwards. This has meant there is a change to the default discrepancy (as mentioned above) and this has allowed for customisation of an Instance configuration based on a customer’s individual assessment rules. Below is a table outlining what the original configuration option was (in Version 8.6) and what the possibilities are now, noting the new default. Please raise a ticket with the service desk if you would like to change the configuration of the discrepancy threshold from subpart score to total item score.
Server Version | Senior Marking Required When | Senior Marking is required to mark | What's auto calculated | Where is marking increased |
Default
8.6 | Total Score breached | All sub parts of MWQ | When marking is within tolerance of the threshold, an auto score is generated for all sub parts and total of item | Item score triggers Senior Marking for all sub-parts |
Default in all subsequent versions
8.7.x | Configurable - a) Sub-Part Score | When total breached - only part in breach requires marking | When sub-parts of an item are within threshold, the calculated score cannot be changed | Subpart(s) of item outside of threshold trigger Senior marking for each individual subpart |
8.7.x | Configurable - b) Total Item Score | Only part in breach of variance set | All sub-parts of item within threshold have an auto calculated score that is editable | Item score triggers Senior Marking for sub-parts outside of threshold |
Configuration of Candidate identifier visible to marker
There is a configuration setting regarding how the candidate is identified to the marker. This setting can be changed by risr/assess, please get in touch through the Service Desk and we will apply the appropriate setting for your organisation.
Setting | Description |
|---|---|
Set to id | Marker sees a marking ID (assignment object). This is a unique candidate ID per exam. |
Set to none | Marker sees no identifier. |
Set to gmc | Marker sees the GMC number (Candidate ID). |
Set to name | Marker sees the candidate name. |