Standard setting Setting is the methodology process used to determine the proficiency of candidates and the related cut scores for each examexam outcomes for candidates. Practique includes several common Standard Setting methods, each of which can be customised, and additionally allows you to use custom Standard Setting methods.
Supported Standard Setting Methods
Angoff
Each item in the item-bank can be assigned an Angoff percentage, representing the percentage of minimally competent candidates* (borderline) that would be expected to know the correct answer to the item.
...
McManus Borderline Regression
Similar to borderline regression, the examiner specifies a global mark for each item answered by the candidate, which minimally has pass/borderline/fail levels. The pass mark is determined by constructing the regression line through all of the groups and combining with a negative confidence interval.
| Info | ||
|---|---|---|
| ||
Related termsCronbach’s Alpha Standard Error of Measurement The SEM is calculated from the Alpha and is multiplied by a user-input number (the SEM-multiplier) allowing a single-sided confidence interval to be added to the pass mark. For example an SEM-multiplier of 1.64 is equivalent to a 90% confidence that people given a pass should have passed (based on the reliability of the exam), and 1.96 corresponds to 95% confidenceStandard Error of Measurement (not to be confused with the Standard Error of the Mean) gives an indication of the spread of the measurement errors, when estimating candidates' true scores from the observed scores. It is calculated from the reliability coefficient (Practique uses Chronbach's alpha). It is assumed that the sampling errors are normally distributed. The SEM is calculated as SEM = S(1 – rxx)0.5 where S is the standard deviation of the exam, and rxx is the reliability coefficient (Chronbach's alpha). The key application of SEM in Practique is to apply a confidence interval to the cut score. For example, if you would like to be 68% sure of the pass/fail decision, the SEM indicates that the candidates within 1 SEM of the cut score may fluctuate to the other side of the cut score should they take the exam again. For example, if you wanted to be 95% sure of your decision on outcomes, an SEM multiplier of 1.96 can be applied. These figures are based on the Normal Distribution. Practique applies this on the positive side for most Standard Setting methods, as we are dealing with competency exams. In practice, what this means is that you are 95% certain that the passing candidates scores represent their true scores. |
Score Normalisation
It is possible to normalise the marks for each item in an exam to a consistent number. You may want to do this so that each item in an exam is equally weighted, despite having an inconsistent number of marks for each item. Note that the normalisation is on an item basis, and not on an exam basis. It is possible to revert to non-normalised scores.
Notes
*It is important that the assessment function has a clear definition of what a borderline or minimally competent candidate is, and whether these are equivalent
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...