8.9 How to create a new remote OSCE Exam

Owned by Miles Skinner (Unlicensed)
Sept 14, 2023
15 min read

  • Click Exams from the top menu bar

  • Click + Create then Exam

  • Fill out the basic exam details as required

    • Exam ID - the unique identifier used by risr/assess, this cannot be changed

    • Title - the name of your exam

    • Number of days - the number of days your exam will run on (if this number is more than one then you will need to specify that here as new fields are opened in exam creation)

    • Exam Blueprint - the template for your exam which will have been created at an earlier stage in Blueprints

    • Exam Venue Candidate Capacity - the number of rooms you are going to have in your circuits

    • Blocks - for OSCE this is equivalent to one Item Set (per block) within an exam, eg. Paper 1 in the morning and Paper 2 in the afternoon

    • Sessions - the method used for segregating candidates/examiners into different physical locations or different times in a day to enable easier control of your exam

    • Circuits - the number of parallel circuits running simultaneously

      • Exam Venue Candidate Capacity X Circuits = Candidates in a Session

    • Standard Setting Method - the method used for calculating cut scores and used in statistical analysis. This can be changed after the Exam is run.

    • Allowed Marking Discrepancy - this is only applicable to OSCEs where there are two examiners per station

      • Set as 20% - Item out of 10 - one marker 8/10 one marker 5/10 discrepancy is 3 which is above  20% - 2

      • Double marking is a back-end configuration and is only available upon request. Please get in touch with one of our team if you would like more information on this.

  • Click Create Exam

Exam Preparation Screen

  • Click Edit Exam Details

  • Some of the previous entered information can be edited here if required

    • Title

    • Exam Venue Candidate Capacity

    • Dates Estimate (Number of days exam will be run on)

    • Allowed Marking Discrepancy

The following Exam Configuration Settings are only applicable to Remote OSCEs

  • Candidate anonymisation - Allows to hide candidate name from the Examiner or Simulated patient P4B view

  • Enable Resource Locking - If enabled, the candidate would not be able to access the resources until a video meeting with an Examiner is started

  • Hide Rest Stations from candidates - The candidate will not be shown any rest stations when they are hidden

  • Enable Copy/Paste - Enables candidate to copy information from the question/resource and paste into notes

  • Enable Calculator - enables the use of in-built calculator function

  • Enable Answer Elimination - not applicable to OSCE

  • Enable Resource Grid view - Enables multiple resources to be simultaneously displayed in a grid format

  • Exam PIN required - When required, candidates will need to enter the exam PIN before they can begin a started session. When not required, logged-in candidates will be started automatically when they are allowed to start

  • Enable Notes? - Enables the use of in-built mini text editor for candidate to make notes only visible to them

  • Enable screen-sharing of the Candidate’s screen - Requires candidate to share their the screen as an additional video feed and extra layer of security

  • Enable Highlighting - Enables highlighting function for candidate to highlight important information in the question text

  • Enable meeting recording - Enables recording of all connected video feeds. This is only configurable upon request and requires discussion with the risr/assess team. Please get in touch with us if this is something you are interested in

Please view our comprehensive Safe Exam Browser Guidance here

  • Require Safe Exam Browser (SEB) - If enabled, candidates will need SEB to take the exam. Additional security requirements can be enforced by using either Configuration Keys (CK) or the more stringent Browser Exam Keys (BEK). Even if no keys are entered in the key lists, SEB must be set up to send configuration and browser keys

    • SEB Configuration Key list - One key per line. If set, the candidate's SEB configuration will be verified against the keys

    • SEB Browser Exam Key List - One key per line. If set, both the candidate's SEB version and configuration will be verified against the keys. Keys are required for each combination of SEB and operating system

  • Exam Timing

    • Exam start mode

      • Timer starts when Exam starts - this means that when participants log in to risr/assess and select the correct exam they will be held at a holding screen, as soon as the Exam is started by the administrator the participants will automatically enter the exam

      • Timer starts on Admin command - this means that when participants log in to risr/assess and select the correct exam they will automatically enter the exam and be held in a waiting room. This allows for any checks such as ID validations and visual/audio checks. The exam is then started by an administrator in the backend and the exam timer will begin. Participants can see this with the appearance of the yellow timer bar

    • Reading Time - Provides time before the start of the examination for the candidate and examiner to read any required information

    • Assessment time - The time during which the examination and assessment takes place. The candidate will carry out any tasks within the simulation

    • Marking Time - Time given for the examiner to mark the candidate against the marking criteria

If required you can upload a cover document. Click Upload cover document to select a file from your device to appear as the first page for your exam. You can upload your customized cover document. Any existing one can be removed with the selection of the Clear checkbox when available. If you already have a cover document uploaded within the Exam Blueprint it will be replaced with this one. We also provide a template which can be customised using the text editor, you can access this by clicking Customise cover document.

  • Click Save Changes to be taken back to the Exam Preparation screen

First Box - Candidates

  • Click Specify Exam Dates

  • Click Choose dates - if across multiple days you can use the exclusion dates settings to automatically filter out certain days (for example: weekends)

  • Click Save Changes

Second Box - Item Sets

  • Click Pre-Select Item Sets

By default the system selects an Item Set at random that fulfils the requirements of the Exam Blueprint. You can then add and/or remove Item Sets as required.

  • Click Approve Item Set when ready

Check for any warning messages ensuring Blocks and Exam Days are correctly set up in correspondence with the Item Set selection.

  • If you need to make any changes to the Item Set selection then this can be by:

    • Click Make Manual Changes

      • Click Modify to make changes to the selected Item Set

        • following this process will take you through the How to Edit an Item Set process

      • Click Replace to select a different Item Set to be used

        • Use the search bar to search for an Item Set by Item Set number

        • Click Find Item Sets to search

        • Select the desired Item Set

        • Click Select as Replacement

First Box - Candidates

  • Click Specify Candidates

  • You can allow risr/assess to allocate candidates automatically based on the use of blueprint tags or by excel spreadsheet

    • Using Blueprint maps to allocate candidates requires that a Blueprint map has been created and applied to candidates within the system already. An example could be Year of course/Graduation Date/Specialty/etc. To allocate those candidates to the Exam simply select the required Blueprint Dimension or Category from the list and then click Schedule

    • To use the Spreadsheet allocation download the template XLS file, fill in and manipulate the data in the spreadsheet and then upload the file to risr/assess (you can also drag and drop the file from your device to the choose file button)

Manual manipulation and data input is required on the spreadsheet. You can find comprehensive information regarding managing Candidates in the system here. For uploading a candidate allocation here is some useful information:

  • If the candidate already exists in the system then as a minimum only the ID of the candidate and the title of the Block for the candidate to be scheduled in to is required.

  • If the candidate is not in the system already then it is possible to use this step to create a full Candidate User account if all the required information is filled out. It would be necessary to ensure all data required for creating Candidates as Users is filled in:

    • ID (Required) - The IDs may be the same as the ID in your list of candidates (Settings > Manage Candidates). If ID does not exist yet, the candidate will be created.

    • First name (Optional) - This overrides the value in Manage candidates, if provided.

    • Last name (Optional) - This overrides the value in Manage candidates, if provided.

    • E-mail (Optional) - This overrides the value in Manage candidates, if provided.

    • Password (Optional) - If provided this will be the candidate's password used to login to this exam. If not provided and the candidate is defined in the Manage candidates section with a password already, the password in Manage candidates will be used. If there is no password, the Candidate ID will be used as a password.

    • Block (Optional) - Exam block name as displayed on View Sessions page. Usually 'Day Month Year - number' or a custom name, is defined. This field must be provided if you want to allocate candidates to rooms and circuits and there is more than one Exam block or more than one Session.

    • Session (Optional) - Session name as displayed on View Sessions page. Session 1, Session 2 etc. or custom name. This field must be provided if you want to allocate candidates to rooms and circuits for an OSCE exam and there is more than one Session in the exam.

    • Room (Optional) - See also Block and Session name above.

    • Circuit (Optional) - See also Block and Session name above.

    • Preferred variant (Optional) - Code of the preferred item variant this candidate will be using when taking an exam. If not set defaults to the Primary variant of the item set used in the exam. It overrides the value in Manage candidates, if provided. Allowed values: en / fr

    • Timezone (Optional) - The time zone where the candidate takes the exam. Usually used for a remote written exams when candidates can be assigned to different sessions taken in different time zones. Optional, defaults to the Practique server time zone.

    • Exclusion date 1 - Exclusion date 10 (Optional) - Up to 10 dates in format YYYY-MM-DD when candidate don't want to be scheduled to

    • Preferred session (Optional) - Preferred session number. This is a preference, not direct assignment. Please provide empty value or 0 for no preference or session number (1,2,..). The preference is not related to any particular day.

    • Be with 1 - Be with 5 (Optional) - Up to 5 ID/GMC numbers of another candidates. The scheduler will try to schedule candidates to the same day/session.

If after uploading a Candidate allocation you need to make manual changes the Candidate allocation there are a number of options available:

  • Clicking Make manual changes will enable you to make changes to the Candidate allocation

    • Using the drop down menus select the Block and Session you need to make changes to and click Show Candidates

    • It is possible to use the checkboxes on the left to select multiple Candidates if a bulk action is required

    • Using the drop down menus, it is possible to change the language variant for Candidates if this is configured for your system

    • Using the drop down menus it is possible to change the Circuit and Room allocation manually

    • By clicking Remove it is possible to remove a Candidate from the Session

      • If you remove a Candidate the system will not make any adjustments to the Rotation as it is ok to have empty Rooms in the Rotation

    • By clicking Add a candidate to this session you can add another Candidate to the session

      • You can use the search bar to search for a Candidate

      • Click Search

      • Click Add for the candidate you need to add

      • You can repeat this process for as many candidates you need to add

        • Adding Candidates automatically fills any empty rooms in the Rotation

  • Clicking Manage Sessions - manage sessions is only for reporting purposes?

  • Clicking Notify Candidates will enable you to send an email from risr/assess to all candidates scheduled for the Exam

    • From the dropdown menus provided you can change the following settings:

      • Notify - notify everyone participating or only previously unnotified candidates

      • Session date - filter candidates to notify by Session date

      • Session - filter candidates to notify by Session

      • Circuit - filter candidates to notify by Circuit

      • Click Populate Recipients to generate a list of Candidates emails based on your filters

    • Using the Free text boxes provided you can then write:

      • Subject - the subject for the email

      • Message - the main body for the email

    • Recipients - these will have been populated by using the above filters or you can manually write candidates emails here (separated by a comma)

    • Attachment - click Choose file if you wish to add an attachment to the email

    • Redirect to - this can be used for testing purposes to redirect all emails to another email address before sending the confirmed message to candidates

    • Click Send notification emails to send the email

  • Clicking Download Candidate Data will give you a spreadsheet download of candidate information which includes:

    • ID

    • First Name

    • Last Name

    • E-mail

    • Password

    • Block

    • Date

    • Session

    • Room

    • Circuit

    • Last Change (to candidate user profile in the system)

    • Preferance (language preference)

    • Time Zone (candidate timezone)

Third Box - Examiners

  • [Optional] Request Examiner Availability  - clicking this button will take you to our email functionality. This feature allows you to email all (or a selection of) examiners requesting that they indicate their availability for the dates identified in previous Exam preparation steps. The examiners have the opportunity to respond as a 1st Choice, 2nd Choice or Not available

  • [Optional] Review Availability - this button shows a list of Examiner’s responses to the availability request from the previous step. From here you can allocate Examiners based on their stated availability

  • Click Allocate Examiners

    • Random Allocation - randomly allocate Examiners and Marshalls into a slot. Due to the nature of running an Exam we do not recommend this step unless you are certain the Examiners and Marshalls in the system will be available or you use dummy accounts for Examiners and Marshalls so selection does not matter.

    • Upload Examiner allocation - this function works in same way as the Candidate upload as can be seen above. Examiner CSV upload guidance is as follows:

      • The data in the CSV/XLS does not have to exactly match the number of examiners/marshalls needed for your exam. For example where in the exam you will need 2 marshalls and 20 examiners, it is possible to upload a list of 1 marshall and 15 examiners. However to amend the allocation, you will need to reupload an amended complete CSV.

        You can also allocate examiners and marshals to sessions, rooms and circuits. It means you can repeat a single examiner several times in the CSV/XLS file and specify different days, sessions, rooms and circuits.

      • ID (Required) - The IDs may be the same as the ID in your list of examiners (Settings > Manage Examiners). If ID does not exist yet, the examiner will be created.

      • First name (Optional) - Used if you specify an ID of non existing examiner. If you specify First Name for existing examiner, it's ignored.

      • Last name (Optional) - Used if you specify an ID of non existing examiner. If you specify Last Name for existing examiner, it's ignored.

      • Block (Optional) - Exam block name as displayed on View Sessions page. Usually 'Day Month Year - number' or a custom name, is defined. This field must be provided if you want to allocate examiners to rooms and circuits.

      • Session (Optional) - Session name as displayed on View Sessions page. Session 1, Session 2 etc. or custom name. This field must be provided if you want to allocate examiners to rooms and circuits If omitted and role of examiner is specified, the examiner is assigned to all sessions of that day.

      • Room (Optional) - Start room number. See also Block and Session name above.

      • Circuit (Optional) - Circuit number. See also Block and Session name above.

        Role identifier (Optional) - The data for the this column is the code of the role that the examiner will be taking on the day. This is 'M' for Marshall and 'E' for Examiner and 'O' for Observer. Please note, the OSCE exams do not support Marker and Senior Marker roles.

Fourth Box - Role Players

  • Generate Requirements - this functionality incorporates the Blueprint tagging of Items with dimensions and categories for role player characteristics and attributes required for that Item (Station)

    • Click Generate requirements

    • Click the Dimension you wish to users a requirement for the Exam

    • Click Download Requirements to obtain an Xcel document contain the following information:

      • Exam Day

      • Exam Date

      • Item Number where the requirements are necessary

      • Name of Role Players that meet those requirements

      • The Type of Exam

    • Clicking Upload details works in same way as the Candidate and Examiner upload as can be seen above. Role Player CSV upload guidance is as follows:

      • You can define role players for each exam day, item and circuit. Role players will be distributed over all sessions. The column order is not important. Any extra columns are ignored.

      • Code (Optional) - Role player login ID If provided, an existing role player will be looked up in the Manage Role players list. If an existing Role player is found, the PIN and First/Last name are not required. If an existing Role player is not found, a new one is created based on the information provided. If this field is not provided a new Role player is always created. If not provided, it will be autogenerated in the form of 6 uppercase letters.

      • PIN (Optional but recommended) - Used to login to risr/assess4Browser interface. If not provided, it will be autogenerated in the form of 5 lowercase letters.

      • First name (Optional) - Role player first name, should be defined when Code is not provided

      • Last name (Optional) - Role player last name, should be defined when Code is not provided

      • Block (Required) - Exam block name as displayed on View Sessions page. Usually 'Day Month Year - number' or a custom name, is defined.

      • Session (Optional) - Session name as displayed on View Sessions page. Session 1, Session 2 etc. or a custom name. If omitted the role player will be assigned to all sessions of that day.

      • Room (Required) - Start room number. See also Block and Session above.

      • Circuit (Required) - Circuit number. See also Block and Session above. You can download the template document that contains all fields required for a full assignment with correct exam block names. Extra columns are not used during import.

  • Clicking Make Manual Changes will allow you to change any Role Players on a per Item basis

    • Using the dropdown menus provided, select the Exam Date and Session you need to make changes for

    • Click Show Role Players to provide the list of Items it is possible to change the Role Player for your selected Session

    • Using the drop-down menu provided for each Item, manually select a Role Player from the list of available Role Players (this list is defined from the Manage Role Players section in Settings)

    • Click Save Changes to save your allocation

Click Exams from the top menu bar and locate the Exam just created. A number of options and overviews are not available to view.

View Sessions

  • View Rotation - provides an overview of the rotation for the Exam

  • View Schedule - provides the same information as in View Rotation but in list format

  • View Rooms - provides you with the ability to change candidates if necessary

  • View Starting Rooms  - where you can change the ordering of the stations - this is the place where clients can design where the rest stations go - has to be done for each individual circuit per exam - can also change examiners

  • View intersect report - downloads excel spreadsheet of everything

  • Masterplan is mostly Examiner allocation

Examiners Availability report

Download Candidate Allocation

 

Exam details

  • View

  • View Seating - list of students

  • View history - audit log of administration against Exam

  • Change date

  • Download Candidate Allocation  - same as download candidate data

  • Continue Preparation - back to exam prep

  • Run session on paper - pushes exam to IN MARKING STATE. Downloads all content that you would need to run the session on paper, zip files

  • Gives pdfs of everything run

  • Publish to devices - this is the point that information is available on P4B to get into the Exam, if Exam PIN is applicable then candidate can use if they have the PIN. Otherwise Admin will click start and it will be ready. OUT risr management for candidates like emails etc

  • Available on Dashboard

 

Reset Exam Day

  • With admin secret code - will delete all data associated with that exam - all answers/responses will be removed