All Categories

The Opposite of Right Isn’t Just Wrong: Using Statistical Analyses to Interpret Incorrect Responses

Research-based Assessment Instruments (RBAIs) have become ubiquitous for scientists engaged in physics education research (PER) as well as instructors interested in evaluating their courses. RBAIs consist of a series of multiple-choice questions that are designed to test a student’s conceptual understanding of a particular topic. Typical analyses of RBAI data involve determining how many questions each student answered correctly at the beginning and end of a course and reporting a measure of growth or learning; however, little attention is paid to which incorrect responses students choose, why someone might choose one answer over another, or whether or not all incorrect answers are equally bad. We are using statistical analyses to try to answer some of these questions for one particular RBAI: the Force and Motion Conceptual Evaluation. Data come from over 7,000 students at multiple institutions across the US.