The Impact of Raters on Teacher Behavior Rating Scale Scores

This session is from the NASP 2021 Annual Convention

  • PMD: 1,9
  • Skill Level: Intermediate

Learner Objectives

This session will help participants…

  1. learn about how rater effects impact teacher behavior rating scale scores for individual students and the relation between teacher behavior rating scale scores and theoretically linked external variables.
  2. learn about prior research on the scope of rater effects on teacher behavior rating scales.
  3. be exposed to advanced analytic techniques for cataloguing and monitoring rater effects.
  4. be presented with an empirical example of rater effects on a brief teacher behavior rating scale.


Despite the ubiquitous use of rating scales, scores on these measures are affected by the unique perspectives of those who complete them. Yet, research has not evaluated whether statistically correcting rater effects might improve score validity/utility. Thus, the purpose of this study was to evaluate this possibility using advanced psychometric techniques. Results indicated that rater effects had potentially large impacts on individual scores, but correcting them did not improve score validity/utility. Implications are discussed.



Kara M. Styck, Northern Illinois University
Christopher J. Anthony, University of Florida


Katherine E. Matthews, Justin R. Martel, Erin Cooke


Letter to Supervisor

Looking for a concise way to pitch the convention to your employer? This letter and the corresponding talking points are here to help!