Exam Anti-Patterns

There is no such thing as the perfect university exam, but if we agree that its purpose is to give an objective score of an individual’s comprehension of the covered topics, then there is clearly a way to be less wrong when creating exams. This post lists a few DON’Ts that can be easily avoided. If you think there is an anti-pattern missing from this post, you can write me an email and I will add it here.

No past exams #

Clearly, coming up with good new questions every year or semester is hard. So why not reuse the questions from past exams and keep them secret? Because you can’t keep something secret that you show to many students! Not publishing exams shifts the most effective preparation from learning and understanding to asking around if anyone remembers the question. This is unfair, random, and does not measure comprehension. The defense against it for students is to publish all questions immediately after the exam. Fortunately, the computer science department at KIT, with a few exceptions, publishes all old exams with good reference solutions. This is great!

Curving exams #

Applying curves to exams discourages studying together and publishing useful resources. It’s a bad incentive if helping out other students gets you a worse grade. In other courses of study such as law or accounting, people go as far as tearing out relevant pages from books in the library. Instead, having useful blogs, git repositories and online tools from students for students is a better spirit. We want to study together, not against each other.

If the grade distribution is Gaussian, the exam is good, right? #

You can design exams in many ways and still arrive at a Gaussian grade distribution. But exams measuring writing speed, IQ or literal memorization miss the point. Of course, you can expect a reasonable amount of any of them. A better way to design exams is to put in a good mix of basic, advanced and hard questions, by thinking about what a student should be able to do in order to pass the exam, and likewise for a good and a very good student.

Open-ended questions with fixed answers #

Asking a question about which whole books could be written and then grading it by expecting a fixed number of predefined keywords to appear in an answer limited to a small box is an anti-pattern. Make clear what you want and what the student is expected to do, use clear task verbs (e.g., name, explain, compare, derive), and think about alternative solutions and ways that the question might be misunderstood.