By Cynthia Ho, Loyola University Law School, Chicago
Failure, Fun and Formative Assessment: Lessons from Click & Learn: Civil Procedure
The title is not a typo. Rather, it describes what can happen when you blend the best of learning theory, legal education goals and technology. Intrigued? Keep reading!
Formative Assessment is now widely acknowledged as important to legal education as reflected in ABA standards. This makes sense based on decades of learning theory that show active learning promotes both short and long term memory. There seems to be great interest in promoting formative assessment based on comments I’ve seen among participants in the CALI Online Teaching Mini-Course, as well as a Civil Procedure Pedagogy & Distance Learning workshop. Many seem to recognize that in-class polling can provide key formative assessment. As studies have shown, such polls close a “feedback loop” by showing whether class and/or specific students understand material, such that any gap can be remedied long before the final. But, how to incorporate more formative assessment, and especially find questions, often seems daunting.
There is actually an easy way to incorporate more formative assessment for Civil Procedure faculty – a new online and interactive tool: Click & Learn: Civil Procedure, of which I’m a co-author with Angela Upchurch and Susan Gilles that is premised upon these principles of learning theory. This is true whether class is entirely in person, entirely online, or some combination. In fact, we have successfully done both.
Click & Learn can provide formative assessment throughout the course. There are over 2000 questions at all levels of difficulty across all topics, as well as sections that cover multiple topics such as SMJ, PJ and Venue simultaneously. These questions permit repeated formative assessment, consistent with learning theory that recommends multiple opportunities for formative assessment, as well as assessment that promptly provides feedback. Since the tool is completely online and provides explanations to questions, including wrong answer choices, students can obtain feedback immediately and on their own pace. Students at our three different law schools, as well as students who have adopted it on their own all tell us they found this fun and effective. Usage outside of class can constitute ABA minutes regarding class time whether the class is designated as fully online or not. In addition, it can even satisfy ABA requirements for distance classes by including feedback and promoting monitoring of student efforts. The questions can also be used with small groups to promote student interactions.
This tool is fundamentally built upon active learning principles, as well as studies that show the benefit of failing on student learning. Although less well known among legal educators, formative assessment can include more than testing students after material is introduced. Studies show that “failing” with a pre-test can actually improve learning. There is a “generation effect,” which basically means that generating a guess on an answer promotes long term recall. Perhaps counter-intuitively, studies find students learn better this way than being told the answer first. Why? Basically, generating a guess, students prime their brain to the concept such that they are more likely to remember it. This means that formative assessment of law students can ideally begin in conjunction with learning material for the first time.
Click & Learn tool incorporates many questions that can be used as pre-tests to promote learning. In particular, each topic includes questions that can be used as a pre-test that studies show can enhance final exam performance. So, for example, a student might be shown FRCP and asked what they think it relates to based on headers. Or, alternatively, a student might be asked why they think a rule has a particular policy.
Click & Learn is designed to supplement any Civil Procedure casebook, regardless of the order in which topics are taught. Each major topic is introduced without any assumption that students have covered other material.
Click & Learn can be flexibly used. Students can do it on their own without any faculty involvement. Or, faculty can adopt it to require the entire class benefit from formative assessment. Faculty of course retain freedom in deciding how to use the formative assessment opportunities. A faculty could require students to answer medium-level questions – in class or outside of class – to test student understanding after covering a topic in class. Or, a faculty could assign, or at least recommend the most basic questions as a way to actively engage students in learning and gain the learning best from such a pre-test. There are also more challenging questions that can be used in class, in office hours, or as supplemental review. In addition, faculty can assign questions to previously covered material to promote interleaving of concepts that studies show further promotes class learning.
Each topic is introduced through several interactive FAQ that students can easily revisit as well. Here is how the beginning of 1332 looks like:
The icons under the “Q#/Level” indicate the number and difficult level of questions provided. Most of these are the easiest questions. The section with the trophy icon is a review section with more challenging questions. A student can do these sections in conjunction with class material. In addition, students could return to past sections that they previously performed poorly on. This would also be consistent with interleaving and lead to better learning outcomes. Indeed, we found our students often repeated questions they initially got wrong even though we did not require them to do so.
To help make it easy to incorporate Click & Learn, there is a downloadable table of contents that shows all the topics, as well as questions, including difficulty level. There is also a way to match it to casebooks. Both of these are shown under the Support tab online:
“Matching to my Casebook” enables faculty to match Click & Learn with major textbooks including Freer & Perdue, Glannon and Yeazell. Faculty adopters can easily cut and paste from these documents to create something like this:
Class 1: Introduction to SMJ and Diversity
Read F&P pp. 175 – 207
Class 2: Diversity
Read F&P pp. 207 – 212
WEEK IN REVIEW Complete by ____________2020
C&L Unit 3, Part 4: Diversity (& Alienage) SMJ
Required: Ch V. 1332 SMJ Synthesis 15 Qs Synthesis
Optional: Ch II. D. Review of “Citizenship” for SMJ – 10 Qs Synthesis
AND Ch IV. F. Review of Amount in Controversy (AIC) 4 Qs PMP
Click & Learn is also designed to promote more student learning. How? It provides a unique faculty dashboard rich with data. For example, the image below shows that question 2 is the one that most students had trouble with, as well as that choice “C” was the most common wrong answer. Checking what choice C says (which is easily done with one click) can then indicate what issue should be reviewed. A faculty member can thus use this information to close the feedback loop and address the issue(s) of confusion.
The Dashboard also provides information on individual students. It highlights late submissions, and also provides the score of each student. You can see this below with a fake class (to avoid FERPA issues). Here, the fifth student was late. In addition, this easily identifies students 1 and 4 as the lowest performing students – but, only to the individual faculty member. Students see their own scores, but not those of other students. This enables a faculty member to easily identify students who need more support and provide that support.
There is a lot more I could say about this, but it would probably be more engaging to try it out yourself! Any faculty member interested in learning more can register for a free complimentary copy. In addition, if you want more of an overview, check out this zoom session: https://www.youtube.com/watch?v=-fmugIa-oUA