What to do with the questions people got wrong
Feedback after a quiz: why showing the correct answer matters, why a short delay can beat instant marking, and what to do with the misses before people leave the room.
A quiz at the end of a session is only half the job. The other half is what you do with the answers. If people leave knowing they got question four wrong, and not knowing what four actually was, you have measured the miss and left the miss in place.
The useful sequence is retrieval, then the correct answer, then another look at the items that failed.
What the evidence says
Butler, Karpicke and Roediger ran two experiments, published in the Journal of Experimental Psychology: Applied in 2007, that pulled apart two things teachers often bundle: how you give the answer, and when.
Participants read twelve short passages, then sat a six-option multiple-choice test. Some items got standard feedback (the correct answer on the screen). Some used answer-until-correct (keep choosing until you hit it). Some items were tested with no feedback. Some were never tested. Feedback was either immediate, after each question, or delayed — ten minutes in the first experiment, a day in the second. The final test was cued recall, a day later or a week later.
Three results are the ones that travel.
A test beats no test. A test with the answer beats a test without one. In the week-later experiment, previously tested items were recalled at .42 against .26 for items that had not been on the first test. Items that had received feedback reached .65. The quiz is doing work. The answer after the quiz is doing more.
How people arrive at the answer does not matter much, once they actually see it. Answer-until-correct and standard feedback finished almost identically (.64 against .65). Earlier classroom studies that favoured “keep guessing” had often given that method immediately and the ordinary mark-sheet a day later, so the method and the timing were tangled. When Butler’s group separated the two, discovery added nothing over being shown the answer.
A delay can beat instant marking. Delayed feedback produced better final recall than immediate feedback (.70 against .60 after a week). The advantage was smaller after a one-day final test and larger after a week, which is the usual shape of a spacing effect. After a correct response, delayed feedback is a second look with a gap. After an error, the delay may also let the wrong option fade before the right one is stored.
They also tracked leftover wrong answers. A multiple-choice test with no feedback left people producing more of those lures later (.26) than if the item had never been tested (.18). Feedback brought that down to .13. If you use options, you owe people the key.
Butler is careful about the classroom version of “delayed”. In the lab, everyone had to look at every answer for a fixed time, right or wrong. In a room, a mark-sheet returned next week is easy to scan for the red ink and ignore. The delay only helps if the second look actually happens.
What to do in the session
You do not need a teaching machine. You need the correct answer to be seen, including on the items people already had.
- Ask from memory. Notes closed.
- Show the answers. A slide, a walk-through, or a marked sheet — the method is less important than the content.
- Make people look at the ones they got right as well as the ones they missed. That is the restudy they will otherwise skip.
- Restudy only the misses if time is short, then ask those again if you can.
- If you can return to the same questions at the next meeting, do. That delay is useful, provided they will actually open the answers.
A tool can show the key while the room is still together — WrapUp Quiz is built for that last-five-minutes pass — but a printed answer list is the same mechanism.
What this does not mean
It does not mean you should withhold the answer until next week as a point of principle. If people will not come back to the sheet, immediate feedback in the room is the better of the two real options.
It does not mean answer-until-correct is harmful. It was simply no better than showing the key, and it exposes people to more wrong options on the way.
It does not mean feedback is a lecture. The point is the correct response to the question they just tried, not another tour of every slide.
And a quiz whose answers are still on the wall was not a test in the first place.
Sources
- Butler AC, Karpicke JD, Roediger HL 3rd. The effect of type and timing of feedback on learning from multiple-choice tests. Journal of Experimental Psychology: Applied. 2007;13(4):273-281.
- Cepeda NJ, Pashler H, Vul E, Wixted JT, Rohrer D. Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychological Bulletin. 2006;132(3):354-380.
- Roediger HL 3rd, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychological Science. 2006;17(3):249-255.
Questions people actually ask
Should I tell people the answers straight after a quiz?
Yes, they need the correct answer. Butler, Karpicke and Roediger found that a test with feedback beat a test with none, and beat items that were never tested. Instant marking is not the only way to deliver that answer.
Is delayed feedback better than immediate feedback?
In their laboratory studies, yes. Feedback after ten minutes, or the next day, produced better later recall than feedback after each question. The delay is a form of spacing. People still have to actually read the answer.
Do people need to keep guessing until they get it right?
Not for the sake of the method. Answer-until-correct and simply showing the right option produced the same later recall once both groups had to look at the feedback for a set time.
What if I only mark the wrong answers?
That is the usual classroom habit, and it is the weak point. Feedback after a correct response is also useful. If people only glance at the misses, they miss a spaced restudy of what they already had right.
Will a quiz without feedback still help?
A bit. Testing alone beat not testing. Without the correct answer, though, wrong options from a multiple-choice question can stick. Feedback cut those leftover lures roughly in half in the week-later test.