Beyond Multiple Choice: Using Automated Analysis to Evaluate Student Writing about Biology

Presented by Kevin Haudek

Constructed-response assessments, in which students write their own answers, can provide greater insight into student thinking, but are challenging to use in large-enrollment courses due to resource constraints.  We are investigating the use of lexical analysis software as a means to provide instructors timely formative feedback from constructed-response items. In this project, we examine student explanations of basic chemistry concepts in an introductory biology course.  Lexical analysis has revealed common problems in students’ explanations, even when those students are able to make a correct multiple choice selection.  By using a combination of lexical analysis and statistical classification, we achieve computer-to-expert interrater reliability (IRR) that approaches expert-to-expert IRR.  Such automated techniques may help overcome the barriers to use of constructed-response items in a wide range of large enrollment