Scale development · Adaptation · Validity

Can we trust what this scale measures?

A questionnaire score is only useful if the items measure the construct they claim to measure, measure it consistently, and work the same way for the people who answer them. Scale development builds a new instrument from a clear definition of the construct. Scale adaptation carries an existing instrument into a new language or culture, and then has to show again that it still works.

I work with researchers in health, sport and education who need an instrument that will hold up in review: planning the study, choosing the analyses, and building the reliability and validity evidence that goes into the paper.

How many items does a scale need?

Spearman–Brown formula
How strongly the items hang together. Higher values mean each item carries more of the same signal.
Longer scales are more reliable, but every extra item costs respondents time.
Good
Expected α for this scale—
Items needed for α = .80—
Items needed for α = .90—

Standardized α under equal inter-item correlations, from the Spearman–Brown formula: α = k·r̄ / (1 + (k − 1)·r̄). The .70 / .80 / .90 lines are common rules of thumb (Nunnally & Bernstein, 1994), not fixed cut-offs. Real scales also need evidence on factor structure and validity; α alone is not enough.

See when α underestimates reliability: α vs ω →

From construct to validated scale

The development sequence

  1. Define the construct

    Literature review and a written definition of what the scale should and should not cover.

  2. Write the item pool

    More items than needed, with a clear response format and wording rules.

  3. Expert review

    Content validity ratings from subject experts, then revision of weak items.

  4. Pilot study

    A first sample to check item statistics, clarity and response patterns.

  5. Factor structure

    Exploratory and confirmatory factor analysis to test the intended dimensions.

  6. Reliability

    Internal consistency, composite reliability and, where needed, test–retest.

  7. Measurement invariance

    Checking that the scale works the same way across gender, age or other groups.

  8. Report and interpret

    Scoring guidance and a write-up that reviewers and users can follow.

For adaptations, steps one and two become forward and back translation, expert review of cultural fit and language equivalence, and a check that the original factor structure holds in the new sample.

Instruments and publications

Evidence

Scale development and adaptation studies

2026

Development of the Esports Orientation Tendency Scale (ESOTS) New scale

Birol, S. S., Birol, E., Sirganci, G., et al. · Discover Psychology, 6, 107

A new instrument measuring individuals' orientation toward esports.

Read article
2023

Determination of the psychometric properties of the Digital Addiction Scale for Children Psychometric properties

Çimke, S., Gürkan, D. Y., & Sirganci, G. · Journal of Pediatric Nursing, 71, 1–5

Reliability and validity evidence for a scale measuring children's digital addiction.

Read article
2023

Psychometric properties of the Sport Imagery Ability Questionnaire (SIAQ) in Turkish culture Adaptation

Araç Ilgar, E., Sirganci, G., & Cihan, B. B. · CBÜ Journal of Physical Education and Sport Sciences, 18(2), 455–467

Turkish adaptation of a questionnaire on athletes' imagery ability.

Read article
2022

Reliability and validity study of the Rational Drug Use Scale Validity study

Göçer, Ş., Sirganci, G., & Günay, O. · Türkiye Klinikleri Journal of Health Sciences, 7(3), 856–864

Reliability and validity evidence for a scale on rational medicine use.

Read article
2021

A study on validity and reliability of the COVID-19 Anxiety Scale Validity study

Sirganci, G., Ceyhan, O., & Kilic, H. · Kocaeli University Journal of Education, 4(1), 123–136

Reliability and validity evidence for a scale measuring anxiety related to the COVID-19 pandemic.

Read article
2019

A study on validity and reliability of the Sports Engagement Scale Validity study

Cihan, B. B., Ilgar, E. A., & Sırgancı, G. · Journal of Youth Research, 7(17), 171–191

Reliability and validity evidence for a scale measuring engagement in sport.

Read article

In press and under review

2026

Examination of the psychometric properties of the Digital Vaccine Literacy Scale Psychometric properties

Çimke, S., Gürkan, D. Y., & Sirganci, G. · In press, Osmangazi Journal of Medicine

2026

Adaptation and validation of the Employee Intrapreneurship Scale in the Turkish sport management context Adaptation

Kahyalar, F., Sirganci, G., Araç Ilgar, E., & Cihan, B. B. · Under review, minor revision requested · Humanities and Social Sciences Communications (Nature Portfolio)

Methods behind the instruments

2022

Measurement invariance in the scale development process

Book chapter, in Turkish · In M. Acar Güvendir & Y. Özer Özkan (Eds.), Tüm yönleriyle ölçek geliştirme süreci (pp. 155–182). Pegem Akademi

2021

Determining the factors affecting the psychological distance between categories in the rating scale

Sırgancı, G., & Uyumaz, G. · International Journal of Contemporary Educational Research, 8(3), 178–190

Examines what affects the spacing between response categories in rating scales.

Read article
2020

What is the required sample size for confirmatory factor analysis? Bayesian approach and maximum likelihood estimation

Uyumaz, G., & Sırgancı, G. · OPUS International Journal of Society Researches, 16(32), 5302–5340 · in Turkish

Compares Bayesian and maximum likelihood estimation for CFA across sample sizes.

Read article
2018

Analyzing the psychometric distance between categories of the Behavioral and Emotional Rating Scale-2 (BERS-2)

Sali, G., & Sirganci, G. · Kastamonu Education Journal, 26(4), 1217–1226 · in Turkish

Read article