Free · Runs in your browser · No data leaves your device

Psychometrics you can move

Four small tools for students, researchers and test developers. Drag a slider, enter your own numbers, and see how the models behave. Each tool lists the formulas it uses so you can check the results.

Rating scale categories Adaptive testing simulator Mantel–Haenszel DIF Alpha vs omega

Do your response categories work?

Rasch rating scale model

In a well-functioning rating scale, each response option is the most likely answer somewhere along the trait. Move the thresholds and watch for categories that are never chosen, or thresholds that come out of order.

Number of categories

Categories work

How it is computed. Andrich (1978) rating scale model with item location δ = 0: P(X = x | θ) ∝ exp Σ_k=1..x (θ − δ − τ_k). Adjacent curves cross at θ = δ + τ_k. The spacing check follows Linacre's (2002) guideline that thresholds should advance by at least 1.4 logits for a 3-category scale (about 1.0 for 5 categories) and by less than 5 logits. These are guidelines for judging a scale, not strict tests.

Watch an adaptive test find your level

CAT · 2PL · EAP

A computerized adaptive test picks each next item to be as informative as possible at the current ability estimate. Set a true ability and run it, or answer the items yourself, and see how quickly the estimate settles.

estimate θ̂ ± 1 SEtrue θ● correct · ○ incorrect

Who answers?

True ability θ 1.00

Stop when SE is below Or after 30 items, whichever comes first. Bank of 80 simulated items.

Ready

Items given0

Ability estimate θ̂0.00

Standard error1.00

#	a	b	answer	θ̂

How it is computed. Items follow the two-parameter logistic model (Birnbaum, 1968) P = 1 / (1 + exp(−a(θ − b))); the bank has 80 items with a between 0.7 and 2.0 and b between −3 and +3. The next item is the unused one with the most Fisher information a²·P·(1 − P) at the current estimate (Lord, 1980). θ̂ is the expected a posteriori (EAP) estimate with a standard normal prior (Bock & Mislevy, 1982); the standard error is the posterior standard deviation. Real programs add exposure control and content balancing, which this demo leaves out.

Mantel–Haenszel DIF calculator

ETS A / B / C classification

Enter, for one item, how many people in each group answered correctly and incorrectly within each total-score group. The calculator returns the common odds ratio, the ETS delta (MH D-DIF), the chi-square test and the ETS category. The table opens with example numbers; replace them with your own.

Example data

Score group	Reference group		Focal group		remove
Score group	correct	incorrect	correct	incorrect	remove

—

Common odds ratio α_MH—

MH D-DIF (Δ_MH)—

SE of Δ_MH—

MH χ² (continuity corrected)—

p-value—

Score groups used—

How it is computed. For score group k with reference correct A, reference incorrect B, focal correct C, focal incorrect D and total N: α_MH = Σ(A·D/N) / Σ(B·C/N) and Δ_MH = −2.35 · ln α_MH. Negative Δ means the item favors the reference group. The SE uses the Robins–Breslow–Greenland (1986) variance of ln α_MH. ETS categories: A if |Δ| < 1 or χ² is not significant at .05; C if |Δ| ≥ 1.5 and |Δ| is significantly greater than 1 (one-sided .05); otherwise B. Score groups where either group is empty are skipped. References: Mantel & Haenszel (1959); Holland & Thayer (1988); Zwick (2012).

When does α underestimate reliability?

Cronbach · McDonald

Cronbach's α assumes every item measures the construct equally well. When factor loadings differ, α falls below ω, the reliability of a one-factor model. Change the loadings and compare.

Cronbach's α—

McDonald's ω—

—

How it is computed. Six standardized items load on one factor with loadings λ_i and uncorrelated errors (error variance 1 − λ_i²). Implied total variance V = (Σλ)² + Σ(1 − λ²). ω = (Σλ)² / V (McDonald, 1999) and Cronbach's (1951) α = k/(k − 1) · (1 − k / V) from the implied correlation matrix. With equal loadings (tau-equivalence) α = ω; otherwise α < ω.

References

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick, Statistical theories of mental test scores (pp. 397–479). Addison-Wesley.
Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6(4), 431–444.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel–Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Erlbaum.
Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Erlbaum.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4), 719–748.
McDonald, R. P. (1999). Test theory: A unified treatment. Erlbaum.
Robins, J., Breslow, N., & Greenland, S. (1986). Estimators of the Mantel–Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics, 42(2), 311–323.
Zwick, R. (2012). A review of ETS differential item functioning assessment procedures: Flagging rules, minimum sample size requirements, and criterion refinement (Research Report No. RR-12-08). Educational Testing Service.