8.12 How to Set Up Ebel Scoring
Inviting judges
To start the Ebel scoring process you will need to create an Item Set
Leave the Item Set in Draft
On the Item Set page click "Ebel Scoring"
On the Ebel score overview page click Invite Judges to Ebel scoring
Type in the names of the judges you want to invite. These must be existing examiners in the risr/assess system. If you do not have any judges or the judges you want to invite are not in risr/assess you will need to add them in the risr/assess Users section in Settings. They will need a minimum set of permissions to be an Ebel judge including: Item sets - Score item set
Once you have entered the names of all the judges click Invite
Setting Ebel scores as a judge
The Judges will each receive an email with an invitation and a clickable link that will take them to the Item Set you want them to judge.
The Judges will click on the link and asked to log into risr/assess. If they have forgotten their password they can reset by clicking on forgotten your password
Once the judge logs in they will be taken to the Ebel scoring page which will show a list of all the items in the Item Set.
The Judges can click on the item to view the content. The Item will open in a new tab in the internet browser.
The Judge can then select the difficulty from a drop down menu (Easy/Medium/Hard)
The Judge can then select the relevance from a drop down menu (Essential/Important/Acceptable)
The Judge can say if they wrote the question by ticking the box
The Judge can add any comments (Optional)
When they have completed their judgements the judge must click "Save" at the bottom of the page
The Ebel score will automatically calculate based on the judges scores and will be visible on the Item Set page.
To finalise the Ebel score the Item set needs to be approved.
Setting the passing score with Ebel
To set the passing score using Ebel, select "Ebel method" from the "standard setting method" drop down menu on the standard setting page.
The Ebel score will be automatically calculated and displayed when you click "apply" on the standard setting page.
risr/assess will pass or fail candidates based on the Ebel score and display the results.