Definition:
Item Response Theory (IRT) is a family of statistical models used in test design and scoring that models the probability of a correct response as a function of both the test-taker’s latent ability and the characteristics of each test item (difficulty, discrimination, guessing). Unlike Classical Test Theory which focuses on total scores, IRT analyzes the interaction between individual items and individual test-takers.
In-Depth Explanation
IRT is the psychometric framework behind most modern high-stakes language assessments, including TOEFL, IELTS, JLPT, and many CEFR-aligned tests.
Core concept — the Item Characteristic Curve (ICC):
Each test question has a characteristic curve showing the probability of a correct response at different ability levels. The curve is typically S-shaped (logistic): low-ability test-takers have a low probability of answering correctly, and higher-ability test-takers have a higher probability. The curve’s shape is defined by:
| Parameter | Symbol | What it measures |
|---|---|---|
| Difficulty | b | The ability level at which there’s a 50% chance of a correct answer |
| Discrimination | a | How well the item separates high from low ability (steepness of the curve) |
| Guessing | c | The probability of a correct answer from pure guessing (floor of the curve) |
IRT models by number of parameters:
- 1-Parameter (Rasch model): Only difficulty varies between items. See Rasch Model.
- 2-Parameter: Difficulty and discrimination vary.
- 3-Parameter: Difficulty, discrimination, and guessing parameter.
Why IRT matters for language testing:
- Adaptive testing: IRT enables Computer Adaptive Testing (CAT), where the test selects harder or easier items based on your running performance. This is how the TOEFL iBT works — each test-taker gets a different set of questions tailored to their level.
- Score comparability: IRT allows scores from different test forms to be placed on the same scale (equating), meaning you can compare scores across test administrations even when the questions differ.
- Item banking: Test developers can maintain large pools of calibrated items with known difficulty and discrimination parameters, drawing from them to create equivalent test forms.
- Fairness analysis: IRT-based Differential Item Functioning (DIF) analysis can detect items that are biased — working differently for test-takers of equal ability from different groups.
Related Terms
See Also
Research
- de Ayala, R. J. (2009). The Theory and Practice of Item Response Theory. Guilford Press. — Comprehensive textbook covering IRT models, estimation, and applications.
- Bachman, L. F., & Palmer, A. S. (1996). Language Testing in Practice. Oxford University Press. — Applies IRT and other measurement frameworks to language assessment.