This covers how to relate your blueprints directly to the creation of exams, i.e. sets of questions.
Clicking on Create a new blueprint from the dashboard will take you to the creation screen.
The idea for creating exam blueprints is to allow you to create templates for common exams that might reoccur in the same format regularly. For example an OSCE exam that is always run in the summer with the same number of stations and to the same year group. All of the information that you enter here will influence the questions/stations that the system tries to automatically generate for this exam. Setting the Exam defaults will also determine the configuration and placement of candidates into circuits and sessions.
Exam defaults terminology
Exam venue candidate capacity
This is the maximum number of candidates that your exam venue can accommodate per circuit
Number of blocks during an exam day
Useful to have more than one if you intend to run different Item sets on the same day. One block can accommodate one Item set.
Number of sessions during an exam block
Sessions are run sequentially and usually multiple sessions will allow different groups of candidates to take the same exam on the same day.
Number of circuits per one session
Circuits are run concurrently so this field determines the number of circuits that your exam centre can accommodate at the same time. The number of candidates each circuit will contain is set within the field for Exam venue candidate capacity.
Default standard setting method
Choose between Angoff, Ebel, Borderline Group and Borderline Regression. Read more about this here
Supported Standard Setting Methods
Angoff
Each item in the item-bank can be assigned an Angoff percentage, representing the percentage of minimally competent candidates that would be expected to know the correct answer to the item.
Ebel
Practique has full support for Ebel standard setting, allowing each exam paper to be standard set by a group of invited judges. The Ebel matrix can be customised, and difficulty and relevance can be input by each judge, and the Ebel scored automatically calculated for each item in the exam. Ebel score histories are kept for each item in the item bank, allowing Ebel scoring trends over different exams and exam papers to be tracked separately in the item bank.
Borderline Group
The borderline group method only applies to examiner marked exams. For each item the examiner can indicate a global mark for each item answered by the candidate, which can have multiple levels, but minimally pass/borderline/fail. The pass mark is determined by analysing the borderline group of candidates.
Borderline Regression
Similar to borderline group, the examiner specifies a global mark for each item answered by the candidate, which minimally has pass/borderline/fail levels. The pass mark is determined by constructing the regression line through all of the groups.
Cronback’s Alpha
For each of the standard setting methods, the Cronbach’s alpha reliability metric is also calculated for the exam, and is given for the whole exam, as well as what it would be if each item in turn were omitted from the analysis. This allows items that are lowering the reliability of the exam to be excluded from the results.
Standard Error of Measurement
The SEM is calculated from the Alpha, and is multiplied by an user-input number (the SEM-multiplier) allowing a single-sided confidence interval to be added to the pass mark. For example an SEM-multiplier of 1.64 is equivalent to a 90% confidence that people given a pass should have passed (based on the reliability of the exam), and 1.96 corresponds to 95% confidence.
Related articles