First Monday Musings By Dean Vik Amar: What Can Law Profs Learn -- From Grading Exam Answers -- About How To Write Good Questions?

Four thoughts for law teachers to consider as they construct future exams.

(Image via Getty)

This time of year, as we law professors assess final exams and assign course grades, we naturally reflect on the ways some students could have done better in their test answers.  But as I finished grading in my own course over the past few weeks, it occurred to me that it might also be helpful to think — and share my thoughts — about how to improve in creating law school tests.  So below I offer — based on my two-and-a-half decades as an exam giver, my several years as an Associate Dean dealing with issues arising from exams given by my entire faculty, and my stint as a dean, that affords me the chance to hear much about what happens not just at my own school but at other law schools — four thoughts for law teachers to consider as they construct future exams.

Frequency

In my first 20 years of teaching, I never gave any exam other than a final.  The historical explanation for why law professors — as contrasted with undergraduate instructors — generally eschew midterm exams is, I think, largely resource-focused; unlike undergraduate professors, law professors ordinarily do not have Teaching Assistants who can help grade exams, and in a large course the time commitment involved for a law professor to carefully read and grade all the midterms as well as finals would be quite significant.  But I have come to believe in the past handful of years that midterms (and other, shorter assignments prior to the final exam) are well worth the time investment.  Regardless of how much weight I attach to midterm performance in the final course grade, and even if I use question types that are faster to grade than traditional issue spotting/analyzing questions — e.g., short answer, multiple-choice questions, modified true/false questions in which I tell students that particular statements are false but ask them to explain in a few sentences precisely how so — the feedback I get, and the feedback the students get, is invaluable.

Open- or Closed-Book

Just as I never used to give midterms, I also never used to give closed-book exams.  My thinking was that the real world isn’t closed book — any lawyer worth her salt always confirms her legal instincts by doing up-to-date research before drafting any documents — so why should my exams be artificially constrained?  But one answer that has become increasingly important in recent years is that bar exams (or at least important parts of them) are closed-book.  And virtually every state and nearly every law school has seen bar passage rates fall over the last five or six years.  There are many possible explanations for this decline (and no one knows for sure all that is going on), but surely having students experience some closed-book testing on various legal subject matters prior to the high-stakes bar exam is an idea worth pondering.  And if, as I suggest above, professors move beyond the single, final exam tradition, then there is more opportunity to employ a variety of test types.  For example, a course might feature a closed-book midterm and then an open-book final.

The Real World

Sponsored

Speaking of the ways law school exams can sometimes be artificial, let me say a word about the natural tendency for a professor to use an actual case or news story as a basis for an exam question.  I think this is generally a great idea — it simulates what law students will be doing when they graduate, and also helps engage students during the exam itself by making them feel that they are weighing in on important issues of the day.  But as with many great ideas, this one needs to be executed carefully.  One big problem is that ripped-from-the-headlines exam questions are often very convoluted and messy, and can prod students to think about and address a lot of issues that the professor didn’t cover in class and doesn’t really want to test on.  To be sure, law students need to learn how — and be evaluated on their ability — to discern red herrings from real McCoys, but an exam also needs to be fair, and thus to track the content of the course.  So using real-world-based questions can be very effective, but only if exam drafter properly adapts and hones them, which often takes a good deal of time and care.

Putting Students on Proper Notice

And finally, when it comes to putting in sufficient time and care in drafting to be fair to students, attention need be paid to the clarity of the instructions the students are given.  If answers to some questions will be weighted differently than answers to others, instructions should make that fact, and the relative weights, explicit.  If word limits for answers are prescribed, call them limits (not guidelines), and don’t offer a word-number range but rather a specific maximum.  And also specify the sanctions triggered when a student goes over (e.g., “I will stop reading when the answer hits the word limit,” or “I will downgrade an answer by a full letter grade for every X words over the word limit,” etc.)  Finally, instructions should guide students as to what they should do if they feel they don’t have sufficient information to properly answer a question.  Should they assume information and make their assumptions explicit?  Should they describe the additional information they would need or want, and explain its relevance?  I am not suggesting that there is one right way to address this and other foreseeable reactions students might have to particular questions; I am saying only that significant time that drafters spend on anticipating potential wrinkles students may encounter when they take the exam often saves more time (and hassle) during the grading and explanation-to-students process afterward.


Vikram David Amar Vik AmarVikram Amar is the Dean of the University of Illinois College of Law, where he also serves the Iwan Foundation Professor of Law. His primary fields of teaching and study are constitutional law, federal courts, and civil and criminal procedure. A fuller bio and CV can be found at https://www.law.illinois.edu/faculty/profile/VikramAmar, and he can be reached at amar@illinois.edu.

Sponsored