8.9 How to mark Written Exams online
- Miles Skinner (Unlicensed)
risr/assess enables online marking. Online marking is available for individual SAQs (Single Answer Questions) and SAQs that form part of MWQs (Multiple Written Questions). This guide outlines the key features and includes step by step instructions to help you use them effectively.Introduction
Markers can be created and assigned in 2 ways:
Created manually in settings then assigned to items/questions and group/s (if required)
Marker details are imported - this creates and assigns them automatically
Markers are created in risr/assess and then assigned to the item/s they are required to mark. There is also the option to create a Senior Marker role.
Markers can also be grouped, groups are used to divide marking between different markers; groups also facilitate double marking.
1. How to create and assign markers manually.
If you have a small number of markers or need to add an additional marker for example, this method may work best for you.
Go to settings
Select Manage Examiners
Enter examiner credentials and select Role as Marker or Senior Marker
Manual assignment of markers to questions/items
Markers can be assigned to individual questions before or after an exam has been completed. Select a marker from the list and select the relevant question/s for this marker.
Eg. Select Marker101 to mark SAQ#4
Use of Groups
1. Multiple markers sharing marking of one item
Groups can be used to divide marking between different markers.
The candidates are divided equally between the markers and allocated randomly and anonymously.
2. Double marking
If double marking is required, markers should be assigned to different groups.
This allows each marker to assign a mark for each candidate. A senior marker is then required to adjudicate any discrepancies.
The allowed marking discrepancy is assigned in the exam details screen in the exam setup.
Click on Exams - Assign markers - click on Add Group to create your group(s). The groups will then appear in the dropdown to enable markers to be added to a specific marking group(s).
In this example the marking of SAQ #4 is being shared by 2 markers.
Marker101 will mark 5 candidates and Marker102 will mark 5 candidates.
SAQ#5 is being double marked.
Marker101 will mark all 10 candidates and Marker102 will mark all candidates.
NB: The allocation of one marker to a group means that all candidates are marked by that one marker.
Markers can only be removed If they haven’t completed any marking.
As soon as any marking is carried out there is no longer an option to remove the marker.
NB: Resetting an exam does not remove the allocation of markers.
Import marker details
This method is recommended if you have a number of markers to add and assign.
You can add the markers details, assign them to items and also assign to groups; these can be edited manually afterwards if required.
Go to Exams and then View Markers to access the marking assignment screen
Select Upload marker assignment and import marker details from a spreadsheet (see format required below)
The Marking Process - Markers
When a Marker logs in their access is restricted to see the exams and items they have been assigned to. They are shown the number of candidates they need to mark (Expected), the number of candidates they have already marked and those that are in progress.
As markers work through the candidates they are given the option to save or confirm marks.
NB: A standard increment for marks can be set at a server level but this can be overridden at any time when creating an item.
Candidate responses are presented anonymously and in a randomised order.
The marking criteria (allocated in the item creation) are displayed along with each candidate response.
If ‘save’ is selected the marker can come back to this candidate and change the marks.
Once ‘confirm’ is selected the marker is unable to change their mark.
Marking is not shown as complete until all candidate marks have been confirmed.
NB: It is possible to go straight to ‘confirm’ without saving first.
Once candidate marks have been saved they are counted as ‘In progress’.
If they are confirmed they are counted as ‘Already marked’.
Markers are automatically moved the next candidate unless they un-click ‘mark next candidate’.
The Marking Process - Administrators
The Administrator Role
As an administrator, the progress of markers can be monitored.
This shows Marker101 has confirmed marks for 5 candidates
0 are in progress and 5 were to be completed for SAQ#4.
Marker102 has not yet commenced marking.
No marking has been completed for SAQ#5.
Double Marking and the Senior Marker Role
Allowed Discrepancy
When double marking is used, if the marks awarded by the assigned markers are different the average is taken as long as the discrepancy is within the limits defined.
The allowed marking discrepancy is defined in the exam details screen - accessed from Exams
A Senior Marker is required to resolve any discrepancies beyond the defined limit.
A Senior Marker can be added before or after an exam has been run and if marking has commenced.
NB: The allowed discrepancy is only relevant to exams that are double marked
Adding a Senior Marker
The Senior Marker is responsible quality assuring marking and will be notified of discrepancies when double marking is enabled.
If the difference between the 2 markers is outside the limits of the discrepancy a message is displayed highlighting items that need attention.
After selecting the Candidates icon, the candidates requiring attention are highlighted: (This is in the admin view)
When a Senior Marker logs in their dashboard displays the number of items and candidates that they need to review.
If the Senior Marker selects ‘Mark’ they are then taken to the relevant page to review results
The Senior Marker can view the marks allocated and then enter the final mark; it is possible to save marks and then review before confirming.
Once the Senior Marker has reviewed all results the Admin view shows the results are complete and there are no candidates that require attention.
The Senior Marker reviews the mark needing attention and confirms the final mark awarded