8.12 How to Set Up Ebel Scoring

Inviting judges

  • To start the Ebel scoring process you will need to create an Item Set

  • Leave the Item Set in Draft 

  • On the Item Set page click "Ebel Scoring"

  • On the Ebel score overview page click Invite Judges to Ebel scoring

  • Type in the names of the judges you want to invite. These must be existing examiners in the risr/assess system. If you do not have any judges or the judges you want to invite are not in risr/assess you will need to add them in the risr/assess Users section in Settings. They will need a minimum set of permissions to be an Ebel judge including: Item sets - Score item set

  • Once you have entered the names of all the judges click Invite

Setting Ebel scores as a judge

  • The Judges will each receive an email with an invitation and a clickable link that will take them to the Item Set you want them to judge.

  • The Judges will click on the link and asked to log into risr/assess. If they have forgotten their password they can reset by clicking on forgotten your password

  • Once the judge logs in they will be taken to the Ebel scoring page which will show a list of all the items in the Item Set. 

  • The Judges can click on the item to view the content. The Item will open in a new tab in the internet browser.

  • The Judge can then select the difficulty from a drop down menu (Easy/Medium/Hard)

  • The Judge can then select the relevance from a drop down menu (Essential/Important/Acceptable)

  • The Judge can say if they wrote the question by ticking the box

  • The Judge can add any comments (Optional)

  • When they have completed their judgements the judge must click "Save" at the bottom of the page 

  • The Ebel score will automatically calculate based on the judges scores and will be visible on the Item Set page.

  • To finalise the Ebel score the Item set needs to be approved. 

Setting the passing score with Ebel

  • To set the passing score using Ebel, select "Ebel method" from the "standard setting method" drop down menu on the standard setting page.

  • The Ebel score will be automatically calculated and displayed when you click "apply" on the standard setting page. 

  • risr/assess will pass or fail candidates based on the Ebel score and display the results.