8.12 How to create Exam Blueprints

Creating an Exam Blueprint

  • From the top menu bar click Blueprints

  • Click + Create Exam Blueprint under Exam Blueprints

 

 

Blueprint General

  • Under Blueprint General in the free text box provided for Name, enter the name of your Exam Blueprint

  • Using the dropdown box for Exam Type, select whether this Exam Blueprint is for an OSCE or a Written Exam

  • Using the free text box or the arrows for Exam Size, enter the total number of Items that will be used in the Exam

  • (Written Only) Using the free text box or the arrows for Exam Maximum Marks, enter the total number of marks possible to be counted for the Exam. Do not set both Exam Size and Exam Maximum Marks.

  • By clicking the Choose File button for Cover document it is possible to upload a PDF file to act as a cover document for your exam. This option will not appear if you have set cover template content already in the backend of risr/assess

 

 

Exam Defaults

Using the arrows or by typing in the free text boxes fill out, as required, the information about your Exam. These can be changed and/or overwritten when creating an Exam, therefore unless you are using an Exam Blueprint many times we generally do not recommend filling out Exam Defaults at this stage.

Exam Default

Summary

Exam Default

Summary

Exam venue candidate capacity

This is the number of candidates per circuit. Include rest stations in this number.

Number of blocks

This is the number of blocks during an exam day

Number of sessions

The number of sessions in an exam block

Number of circuits

The number of circuits in each session

Standard setting method

Choose between Angoff, Ebel, Borderline Group and Borderline Regression, Adjustable Borderline Regression, McManus Borderline Regression .

This is the default standard setting for the exam.

 

Restrictions

If any Blueprint Dimensions (in your Master Blueprint) are applicable to Exam Blueprints then these will appear here. You can select Blueprint Dimension Categories to restrict use of the Exam Blueprint to only be applicable for Users that also have the same Category (restriction) applied to them in order to use the Exam Blueprint. For example if you have a User that can only create Year 1 Exams then you could restrict the Exam Blueprint to only be viewed, edited and used by any User that has the same Blueprint category mapped.

  • Click the Change button

  • Click all Categories that you want to restrict the current Exam Blueprint by

  • Click Create Blueprint

 

 

Editing an Exam Blueprint

  • Click View Blueprint for the Exam Blueprint you wish to edit

  • Click Edit Blueprint to edit any details entered in the creation stage

  • Once Edited click Save Changes

 

 

 

Exam Blueprint settings

To edit the settings, click the Edit blueprint exam settings button. Depending on the type of exam and the delivery platform, the relevant settings will be available for you. 

All settings are disabled as a default value. Please ensure that you go through and check the values of each setting. 

Click save exam settings to store the template and return to the exam blueprint overview page.

All exam blueprint settings can be edited at the time of creating the exam

Written exam, browser platform

Setting 

Function

Exam PIN required

Control whether candidates need to enter the Exam PIN in order to be admitted to the exam. If disabled, logged-in candidates will be started automatically when they are allowed to start

Safe Exam browser

Control whether candidates will need SEB to take the exam. Additional security requirements can be enforced by using EITHER Browser Exam Keys (BEK) or Configuration Keys (CK). 

SEB Browser Exam Key

Please refer to SEB configuration documentation.

SEB Configuration Key

Please refer to SEB configuration documentation.

Proctoring

Controls the proctor provider for integration

Answer order

Control how the answer options are presented where answer ordering is set to default at item level. Items are unaffected where item level answer ordering is set to custom.

Question Randomisation

Control how the items are sorted within the sections on the item set.

Copy and Paste

Control the ability to copy and paste.

Highlighting

Control the ability to highlight text. This does not apply to resources.

Calculator

Control access to a calculator.

Notes

Control access to take on-screen notes at an item level. Notes are not submitted as part of the response and are not saved.

Resource grid view

Control the ability to toggle to a 2x2 resource layout. If disabled, only the 1x1 layout view will be available.

Backwards navigation

If enabled, candidates can navigate freely between all items and answer them in any order. If disabled, candidates are restricted to moving through the exam consecutively and are required to answer each item before they can progress to the next.

Answer elimination

Control the ability to cross out options in items or sub-items of type EMQ, MAA, MTF, RRQ, SBA. This does not have any impact on submitted responses

Exam timing

If enabled, the exam timing can be configured to specify the duration of the exam. If disabled, candidates have no time limits imposed.

Exam start mode

'Timer starts when session is started' will start the candidate's exam timer on session start, regardless whether the candidate is logged in or not. 'Timer starts when the user enters exam' will allow the candidate the full exam time from the time they log in. The candidate will not lose time if logging in after the session has started.

Counting

What the candidate sees on the Timer (counting up or down)

Exam duration (in minutes)

Period during which candidates can submit responses. Does not include Reading Time

Reading time

Period during which candidates can browse the exam content. They are not able to submit any responses.

Notification of the Remaining Time (in minutes)

Timestamps in which a message "X minutes remaining" will be displayed to candidates. Enter values separated by a comma (example: 30,10,5) to specify multiple timestamps.

Marking type

Pre-sets marking type for the exam

Allowed Marking discrepancy (%)

Only applicable for double marked items. Provide allowed discrepancy value as % of item max score.

Passing score override (%) (requires configuration)

Overrides exam passing score. Provide value as % which is required to pass exam. This can be used where an absolute predetermined passing standard is in place and bypasses any on-platform standard-setting.

Written exam, On paper

Setting 

Function

Answer order

Control how the answer options are presented where answer ordering is set to default at item level. Items are unaffected where item level answer ordering is set to custom.

Question Randomisation

Control how the items are sorted within the sections on the item set.

Marking type

Pre-sets marking type for the exam

Allowed Marking discrepancy (%)

Only applicable for double marked items. Provide allowed discrepancy value as % of item max score.

Passing score override (%)

Overrides exam passing score. Provide value as % which is required to pass exam.

OSCE, browser platform

Setting 

Function

Exam PIN required

Control whether candidates need to enter the Exam PIN in order to be admitted to the exam. If disabled, logged-in candidates will be started automatically when they are allowed to start

Name visibility

Hides candidate name from the Examiner or Simulated patient P4B view.

Rest stations

Control whether the candidate will be shown rest stations. If enabled, the candidate will see a station without any content and an indication 'this is a rest station'. If disabled, the candidate will have early access to the next round's content station and await the examiner.

Resource locking

If enabled, the candidate would not be able to access the resources until a video meeting with an Examiner is started.

Copy and Paste

Control the ability to copy and paste.

Highlighting

Control the ability to highlight text. This does not apply to resources.

Calculator

Control access to a calculator.

Notes

Control access to take on-screen notes at an item level. Notes are not submitted as part of the response and are not saved.

Resource grid view

Control the ability to toggle to a 2x2 resource layout. If disabled, only the 1x1 layout view will be available.

Share candidate’s

Control the requirement for candidates to share their screen as an additional video feed for Examiners / Observers / Simulated Patients to view.

Image synchronisation

Control the ability for users to broadcast and observe in real time how another user is interacting with a resource

Exam timing

If enabled, the exam timing can be configured to specify the duration of the exam. If disabled, candidates have no time limits imposed.

Exam start mode

'Timer starts when session is started' will start the candidate's exam timer on session start, regardless whether the candidate is logged in or not. 'Timer starts on admin command - Waiting Room access' will allow the users into the Waiting Room, however, exam timing will only start on admin command

Reading time

Duration of time that is allotted for the candidate to review station content before the start of the assessment interaction. If no reading time is required set as 0.

Assessment Time Duration (in minutes)

Duration of time that is allotted to the interaction. Must be greater or equal to 1 minute.

Marking Time Duration (in minutes)

Duration of time that is allotted for examiners to complete marking. If no marking time is required set as 0.

Allowed Marking discrepancy (%)

Only applicable for double marked items. Provide allowed dsicrepancy value as % of item max score.

Passing score override (%)

Overrides exam passing score. Provide value as % which is required to pass exam. This can be used where an absolute predetermined passing standard is in place and bypasses any on-platform standard-setting.

OSCE, iPad delivery

Setting 

Function

Allowed Marking discrepancy (%)

Only applicable for double marked items. Provide allowed discrepancy value as % of item max score.

Passing score override (%)

Overrides exam passing score. Provide value as % which is required to pass exam. This can be used where an absolute predetermined passing standard is in place and bypasses any on-platform standard-setting.

OSCE, on paper

Setting 

Function

Allowed Marking discrepancy (%)

Only applicable for double marked items. Provide allowed discrepancy value as % of item max score.

Passing score override (%)

Overrides exam passing score. Provide value as % which is required to pass exam. This can be used where an absolute predetermined passing standard is in place and bypasses any on-platform standard-setting.

 

 

Blueprint exam grade settings

To pre-set the grade banding for an exam 

  • Click the Add blueprint exam grade setting.

  • Use the grade setting function to create and save the bands.

  • Click save and close

 

 

 

 

 

 

 

Exam Blueprint Constraints

  • Click View Blueprint for the exam Blueprint you wish to edit

  • Click Add constraint group

  • Using the free text box for Group Name, type in the name for your constraint group

  • Using the free text box or arrows for Total, enter the value for the maximum number of Items to be generated by this constraint group

  • Click Add constraint group

  • Click Apply filters

  • Using the dropdown menu for Dimension, select the Dimension to be used as a constraint filter

  • Using the search box or drop down menu, select the category from the selected Dimension to be used as the constraint

  • Click Add constraint

  • Using the free text box or arrows, you can select the number of days between Item use and Item Set generation to have passed for an Item to be applicable in the Item Set generation

  • Using the free text boxes, enter the value allowances for each category in the constraint group as required, only either the Total Values or Percentage values can be used

    • Lower Bound - Minimum number of items

    • Upper Bound - Maximum number of items

    • Category Target - Target number of items

  • OR

    • Lower Bound % - Minimum percentage of items to match criteria

    • Upper Bound % - Maximum percentage of items to match criteria

    • Category Target % - Target percentage of items to match criteria

  • You can click in the check box under Delete if you wish to delete one or more of the created constraints

  • Click Save Page to save and continue working

  • Click Save page & Close to save and go back to the Blueprint