Short abstract:

Situational judgment tests (SJTs) are psychometric tests used in medical school admissions in the U.S., Canada, and elsewhere. I argue that SJTs reinforce the prominence of psychometric expertise by both constructing notions of character and proposing methods of evaluating character.

Long abstract:

I investigate a new testing technology used in medical school admissions processes in the U.S., Canada, and other countries. It is a called a situational judgment test, and is a type of test that uses challenging everyday scenarios to learn about the courses of action an individual would take if confronted with them, thereby drawing inferences about the individual’s behavioral traits and procedural knowledge. These tests were developed in the past 15 years, and grow out of the use of similar, less sophisticated tests in non-professional job screening. SJTs are important to investigate because they introduce quantification practices to the assessment of character and fit in admissions processes. While studying the success of educational tests is generally the province of educational researchers and test developers, given the understandings of the consequences of quantification developed by scholars in STS, a narrow psychometric frame is insufficient for understanding the implications of these new tests for admissions processes. The critical approach to psychometrics I articulate investigates the ‘social life’ of situational judgment tests by examining how they are actually used in admissions processes. I combine interview data with a rhetorical analysis of test development and marketing materials to produce a critical analysis of validity claims and underlying theories of behavior in SJTs. I argue that the introduction of situational judgement tests into admissions processes consolidates and reinforces the prominence of psychometric expertise by both constructing notions of character and proposing methods of evaluating character.