The Impact of Raters on Teacher Behavior Rating Scale Scores (OS209)

  • PMD: 1,9
  • Skill Level: Intermediate

Learner Objectives

This session will help participants…

  1. learn about how rater effects impact teacher behavior rating scale scores for individual students and the relation between teacher behavior rating scale scores and theoretically linked external variables.
  2. learn about prior research on the scope of rater effects on teacher behavior rating scales.
  3. be exposed to advanced analytic techniques for cataloguing and monitoring rater effects.
  4. be presented with an empirical example of rater effects on a brief teacher behavior rating scale.

Description

Despite the ubiquitous use of rating scales, scores on these measures are affected by the unique perspectives of those who complete them. Yet, research has not evaluated whether statistically correcting rater effects might improve score validity/utility. Thus, the purpose of this study was to evaluate this possibility using advanced psychometric techniques. Results indicated that rater effects had potentially large impacts on individual scores, but correcting them did not improve score validity/utility. Implications are discussed.

_____________________________

Presenter(s)

Kara M. Styck, Northern Illinois University
Christopher J. Anthony, University of Florida

Contributor(s)

Katherine E. Matthews, Justin R. Martel, Erin Cooke

Letter to Supervisor

Looking for a concise way to pitch the convention to your employer? This letter and the corresponding talking points are here to help!

Preliminary Brochure

Share our NASP 2021 preliminary brochure with your supervisor along with the above letter.