8.11 How to create a new written Exam

Owned by Pernille Marqvardsen
Dec 20, 2023
11 min read

  • Click Exams from the top menu bar

  • Click + Create Exam

  • Select your exam blueprint. Any information that you have completed in your exam blueprint will automatically be used to create your exam. You can correct all parameters

  • Check the basic exam details as required using the free text boxes or dropdown menus

    • Exam Title - the name of your exam

    • Number of days - the number of days your exam will run on (if this number is more than one then you will need to specify that here as new fields are opened in exam creation)

    • Blocks - for OSCE this is equivalent to one Item Set (per block) within an exam, eg. Paper 1 in the morning and Paper 2 in the afternoon

    • Sessions - the method used for segregating candidates/examiners into different physical locations or different times in a day to enable easier control of your exam

    • Circuits - the number of parallel circuits running simultaneously

    • Exam Venue Candidate Capacity - the number of rooms you are going to have in your circuits

    • Standard Setting Method - the method used for calculating cut scores and used in statistical analysis. This can be changed after the Exam is run.

    • Allowed Marking Discrepancy - this is only applicable to OSCEs where there are two examiners per station

      • Set as 20% - Item out of 10 - one marker 8/10 one marker 5/10 discrepancy is 3 which is above  20% - 2

      • Double marking is a back-end configuration and is only available upon request. Please get in touch with one of our team if you would like more information on this.

  • Click Create Exam

Exam Preparation Screen

  • Click Edit Exam Details - All details are pulled from your selected exam blueprint.

    • If your exam blueprint is set up correctly, you will not need to make any changes and can simply click “Save changes” at the bottom of the screen.

    • If you wish to correct these for the individual diet, you can do so here. If you wish to correct it for all exams created from this blueprint, please go to the blueprint section to do so.

  • Some of the previous entered information can be edited here if required

    • Exam Title

    • Exam Venue Candidate Capacity

    • Dates Estimate (Number of days exam will be run on)

  • Exam PIN required - When required, candidates will need to enter the exam PIN before they can begin a started session. When not required, logged-in candidates will be started automatically when they are allowed to start

  • Require Safe Exam Browser (SEB) - If enabled, candidates will need SEB to take the exam. Additional security requirements can be enforced by using either Configuration Keys (CK) or the more stringent Browser Exam Keys (BEK). Even if no keys are entered in the key lists, SEB must be set up to send configuration and browser keys

    • SEB Configuration Key list - One key per line. If set, the candidate's SEB configuration will be verified against the keys

    • SEB Browser Exam Key List - One key per line. If set, both the candidate's SEB version and configuration will be verified against the keys. Keys are required for each combination of SEB and operating system

Please view our comprehensive Safe Exam Browser Guidance here

  • Answer Order - for certain Item types it is possible to randomise the answer order (at Item creation stage the answer order was defined to be default or custom. For any Item where a custom order was defined, randomising the Answer Order at Exam set up will have no effect. For all Items where the Answer order was left as default (alphanumeric) then the selection here will impact those Items.

  • Question Randomisation - this setting will give each Candidate a randomised order of questions (Items)

  • Copy/Paste - Enables candidate to copy information from the question/resource and paste into notes

  • Highlighting - Enables highlighting function for candidate to highlight important information in the question text

  • Calculator - enables the use of in-built calculator function

  • Note - Enables the use of in-built mini text editor for candidate to make notes only visible to them

  • Resource Grid view - Enables multiple resources to be simultaneously displayed in a grid format

  • Backwards navigation - If enabled, candidate can navigate freely between all items and answer them in any order. If disabled, candidates are restricted to moving through the exam consecutively and are required to answer each item before they can progress to the next.

  • Answer Elimination - allow students to strikethrough an answer so they can rule out some answers they are certain are incorrect

  • Exam Timing - If enabled, the exam timing can be configured to specify the duration of the exam. If disabled, candidates have no time limits imposed.

    • Exam start mode - Timer starts when Exam starts - this is pushed by an Administrator in the backend

    • Timer starts when user logs in - started in backend at 9:00 but candidate logs in at 9:10 so timer starts from 9:10

    • Counting Up or Down - this setting changes whether the timer counts down to 0 or up to the maximum Exam time allowed

    • Exam Duration - the maximum allowed time for the Exam

    • Reading Time - Provides time before the start of the examination for the candidate and examiner to read any required information

    • Notification of time remains - the system will automatically prompt candidates with warning messages notifying them of the time remains at set intervals

  • Marking type - Allows you to preselect marking type. This can also be selected post exam.

  • Allowed marking discrepancy [%] - Only applicable for double marked items. Provide allowed discrepancy value as % of item max score.

If required you can upload a cover document. Click Upload cover document to select a file from your device to appear as the first Resource for your exam. You can upload your customized cover document. Any existing one can be removed with the selection of the Clear checkbox when available. If you already have a cover document uploaded within the Exam Blueprint it will be replaced with this one. We also provide a template which can be customised using the text editor, you can access this by clicking Customise cover document.

  • Click Save Changes to be taken back to the Exam Preparation screen

First Box - Candidates

  • Click Specify Exam Dates

  • Click Choose dates - if across multiple days you can use the exclusion dates settings to automatically filter out certain days (for example: weekends)

  • Click Save Changes

Second Box - Item Sets

  • Click Pre-Select Item Sets

By default the system selects an Item Set at random that fulfils the requirements of the Exam Blueprint. You can then add and/or remove Item Sets as required.

  • Click Approve Item Set when ready

Check for any warning messages ensuring Blocks and Exam Days are correctly set up in correspondence with the Item Set selection.

  • If you need to make any changes to the Item Set selection then this can be by:

    • Click Make Manual Changes

      • Click Modify to make changes to the selected Item Set

        • following this process will take you through the How to Edit an Item Set process

      • Click Replace to select a different Item Set to be used

        • Use the search bar to search for an Item Set by Item Set number

        • Click Find Item Sets to search

        • Select the desired Item Set

        • Click Select as Replacement

First Box - Candidates

  • Click Specify Candidates

  • You can allow risr/assess to allocate candidates automatically based on the use of blueprint tags or by excel spreadsheet

    • Using Blueprint maps to allocate candidates requires that a Blueprint map has been created and applied to candidates within the system already. An example could be Year of course/Graduation Date/Specialty/etc. To allocate those candidates to the Exam simply select the required Blueprint Dimension or Category from the list and then click Schedule

    • To use the Spreadsheet allocation download the template XLS file, fill in and manipulate the data in the spreadsheet and then upload the file to risr/assess (you can also drag and drop the file from your device to the choose file button)

Manual manipulation and data input is required on the spreadsheet. You can find comprehensive information regarding managing Candidates in the system here. For uploading a candidate allocation here is some useful information:

  • If the candidate already exists in the system then as a minimum only the ID of the candidate and the title of the Block for the candidate to be scheduled in to is required.

  • If the candidate is not in the system already then it is possible to use this step to create a full Candidate User account if all the required information is filled out. It would be necessary to ensure all data required for creating Candidates as Users is filled in:

    • ID (Required) - The IDs may be the same as the ID in your list of candidates (Settings > Manage Candidates). If ID does not exist yet, the candidate will be created.

    • First name (Optional) - This overrides the value in Manage candidates, if provided.

    • Last name (Optional) - This overrides the value in Manage candidates, if provided.

    • E-mail (Optional) - This overrides the value in Manage candidates, if provided.

    • Password (Optional) - If provided this will be the candidate's password used to login to this exam. If not provided and the candidate is defined in the Manage candidates section with a password already, the password in Manage candidates will be used. If there is no password, the Candidate ID will be used as a password.

    • Block (Optional) - Exam block name as displayed on View Sessions page. Usually 'Day Month Year - number' or a custom name, is defined. This field must be provided if you want to allocate candidates to rooms and circuits and there is more than one Exam block or more than one Session.

    • Session (Optional) - Session name as displayed on View Sessions page. Session 1, Session 2 etc. or custom name. This field must be provided if you want to allocate candidates to rooms and circuits for an OSCE exam and there is more than one Session in the exam.

    • Room (Optional) - See also Block and Session name above.

    • Circuit (Optional) - See also Block and Session name above.

    • Preferred variant (Optional) - Code of the preferred item variant this candidate will be using when taking an exam. If not set defaults to the Primary variant of the item set used in the exam. It overrides the value in Manage candidates, if provided. Allowed values: en / fr

    • Timezone (Optional) - The time zone where the candidate takes the exam. Usually used for a remote written exams when candidates can be assigned to different sessions taken in different time zones. Optional, defaults to the Practique server time zone.

If after uploading a Candidate allocation you need to make manual changes the Candidate allocation there are a number of options available:

  • Clicking Make manual changes will enable you to make changes to the Candidate allocation

    • Using the drop down menus select the Block and Session you need to make changes to and click Show Candidates

    • It is possible to use the checkboxes on the left to select multiple Candidates if a bulk action is required

    • Using the drop down menus, it is possible to change the language variant for Candidates if this is configured for your system

    • Using the drop down menus it is possible to change the Circuit and Room allocation manually

    • By clicking Remove it is possible to remove a Candidate from the Session

      • If you remove a Candidate the system will not make any adjustments to the Rotation as it is ok to have empty Rooms in the Rotation

    • By clicking Add a candidate to this session you can add another Candidate to the session

      • You can use the search bar to search for a Candidate

      • Click Search

      • Click Add for the candidate you need to add

      • You can repeat this process for as many candidates you need to add

        • Adding Candidates automatically fills any empty rooms in the Rotation

  • Clicking Manage Sessions - manage sessions is only for reporting purposes?

  • Clicking Notify Candidates will enable you to send an email from risr/assess to all candidates scheduled for the Exam

    • From the dropdown menus provided you can change the following settings:

      • Notify - notify everyone participating or only previously unnotified candidates

      • Session date - filter candidates to notify by Session date

      • Session - filter candidates to notify by Session

      • Circuit - filter candidates to notify by Circuit

      • Click Populate Recipients to generate a list of Candidates emails based on your filters

    • Using the Free text boxes provided you can then write:

      • Subject - the subject for the email

      • Message - the main body for the email

    • Recipients - these will have been populated by using the above filters or you can manually write candidates emails here (separated by a comma)

    • Attachment - click Choose file if you wish to add an attachment to the email

    • Redirect to - this can be used for testing purposes to redirect all emails to another email address before sending the confirmed message to candidates

    • Click Send notification emails to send the email

  • Clicking Download Candidate Data will give you a spreadsheet download of candidate information which includes:

    • ID

    • First Name

    • Last Name

    • E-mail

    • Password

    • Block

    • Date

    • Session

    • Room

    • Circuit

    • Last Change (to candidate user profile in the system)

    • Preferance (language preference)

    • Time Zone (candidate timezone)

Click Exams from the top menu bar and locate the Exam just created. A number of options and overviews are not available to view.

View Sessions

  • View Rotation - provides an overview of the rotation for the Exam

  • View Schedule - provides the same information as in View Rotation but in list format

  • View Rooms - provides you with the ability to change candidates if necessary

  • View Starting Rooms  - where you can change the ordering of the stations - this is the place where clients can design where the rest stations go - has to be done for each individual circuit per exam - can also change examiners

  • View intersect report - downloads excel spreadsheet of everything

  • Masterplan is mostly Examiner allocation

Examiners Availability report

Download Candidate Allocation

 

Exam details

  • View

  • View Seating - list of students

  • View history - audit log of administration against Exam

  • Change date

  • Download Candidate Allocation  - same as download candidate data

  • Continue Preparation - back to exam prep

  • Run session on paper - pushes exam to IN MARKING STATE. Downloads all content that you would need to run the session on paper, zip files

  • Gives pdfs of everything run

  • Publish to devices - this is the point that information is available on P4B to get into the Exam, if Exam PIN is applicable then candidate can use if they have the PIN. Otherwise Admin will click start and it will be ready. OUT risr management for candidates like emails etc

  • Available on Dashboard

 

Reset Exam Day

  • With admin secret code - will delete all data associated with that exam - all answers/responses will be removed