8.6 Item Response Model

Report showing two facet IRT model and Item characteristic curve. Note: assumptions are made around size of cohort.

Item Response Theory

Item

Description

Useful links

Item

Description

Useful links

Difficulty

3Pl model

curve_fit — SciPy v1.14.1 Manual

Item response theory

Discrimination

3PL modle

curve_fit — SciPy v1.14.1 Manual

Item response theory

Pseudo-guess

This is only showed if it is more than 1. 3PL model

Item response theory

Chi-squared test

 

curve_fit — SciPy v1.14.1 Manual

Chi-squared test

In addition to the Scipy links, here is the wiki page that describes the 3 parameters above for IRT.

Item Characteristic curve (Passing Probability)

 

Classical Test Theory

Item

Description

Useful links

Item

Description

Useful links

Facility

facility = mean_score of the station / max_score of the station

 

Discrimination (point-biserial)

The item discrimination index is a point biserial correlation coefficient. Its possible range is -1.00 to 1.00. A positive result indicates that there is a high correlation between higher performing candidates giving a correct response to the item.

scipy.stats.pearsonr — SciPy v1.9.3 Manual

Point-biserial correlation coefficient

Frequency

In SBA item type frequency of answers is calculated. If candidate have not responded it is included in calculation. Facility and Frequency of most chosen answer should be the same. From Practique 5.4.0 > , beside answer letters columns for Frequency there is No Response column as well to show the whole picture.