By Tonya Kowalski from Washburn University School of Law
This summer, I taught Professional Responsibility to about 60 students, a large number of whom were scheduled to take the August MPRE. While I was committed not to let the class turn into a bar prep course, I saw it as a great opportunity to help students improve their multiple-choice testing skills for the MPRE and Multi-State Bar Examination. This idea of the month concerns how to take the leap into multiple-choice question design with the understanding that it is a growth process.
I knew that I had a lot to learn, as I normally teach only paper courses. I’d written what seemed to be effective, straightforward multiple-choice questions for in-class assessment in a previous doctrinal course, so I thought it would not be too hard to improve my skills. I was wrong. I read some of the excellent guides below, but found that effectively putting that advice into practice still required a lot of trial and error. Fortunately, I did grow slowly but steadily in multiple-choice skills, and I believe the students did, too. I hope that they also learned from some of my errors and our discussions about them.
Despite reading some of the excellent literature below, I still fell into some newbie traps, particularly in the first half of the term.
» My answer selections often tried to test too many things at the same time: applying the correct rule, applying the rule to facts, distinguishing from related concepts, and so on, all in the same question.
» I had a hard time finding the appropriate line between what students should have to read into the question for basic legal context and common knowledge, versus what I needed to supply in order to avoid multiple correct answers.
» Overreliance on “all of the above” and similar answers when they didn’t serve an additional purpose, such as to test whether students knew all three elements of a rule.
» Using fact patterns from in-class discussion in testing, but with nuanced alterations to the facts, which seemed to increase the risk of improper recall more than it tested for understanding.
» Omitting facts that would make the rule application clearer to students who understood the rules. I had a tendency to hold back a little on explaining the facts to make the questions harder. What I found from the students’ challenges was that they understood the rules well but struggled to choose from among two best answers because the facts could be read multiple ways. The increased difficulty decreased, rather than increased, their confidence in knowing the rules.
Meanwhile, I managed to do a few things right (at least most of the time).
» Keep fact patterns as short as possible. Do not include extra “story-telling” facts that do not have a clear purpose in the problem, which is usually to eliminate a grey area in the rule application.
» Avoid silly names and outrageous scenarios. They overload the senses and distract from the purpose of the question, which is to test critical thinking and/or recall. Most of my actors were named “Lawyer,” “Client,” etc.
» When there are two actors, consider making them opposing genders so that the problem is easier to follow.
» When testing about an area with unsettled authority, it’s best to center answer choices around which rule to apply, or the “best argument” one party can use.
» Remember that dozens of intelligent students more likely to find the troubling ambiguities or omissions in your questions than a single, intelligent professor. Consider implementing a multiple-choice challenge procedure on your course website. It should include firm deadlines and a requirement to cite and argue authority, where appropriate.
Short Bibliography
» Susan M. Case & Beth Donahue, “The One-Page Guide to Writing Multiple-Choice Questions” (developed for the National Conference of Bar Examiners).
» Susan M. Case & Beth Donahue, “Developing High-Quality Multiple-Choice Questions for Assessment in Legal Education,” 58 Journal of Legal Education 372 (2008) [Read fulltext in HeinOnline; subscription required].
» Mike Dickinson, “Writing Multiple-Choice Questions for Higher-level Thinking,” Learning Solutions Magazine (December 5, 2011).
» The eLearning Coach, “10 Rules For Writing Multiple Choice Questions.”
» Janet Fisher, “Multiple-Choice: Choosing the Best Options for More Effective and Less Frustrating Law School Testing,” 37 Capital University Law Review 119 (2008).
» Steven Friedland, “Test Builder, 7:1 The Law Teacher, 6-7 (Fall 1999).