Ask them to produce the answer, not pick it

The summary slide is a kindness that does the remembering for the room. The dose, the criterion, the name of the next step: it is all still up there while people nod. Producing that point, with the slide down, is a different close from ticking it in a list that already contains it.

What the evidence says

The generation effect is the memory benefit for something you produced over the same thing you only read. McCurdy, Viechtbauer, Sklenar, Frankenstein and Leshikar pooled 126 articles (310 experiments, 1,653 estimates) in 2020. Across those generate-versus-read comparisons, the mean proportion remembered was 0.581 for generated material (95% CI 0.549 to 0.612) and 0.478 for material that had only been read (0.446 to 0.511). The comparison included healthy adults and excluded children and clinical samples. It is mostly word lists in a lab: half the items are a fragment such as open–cl___, half are the intact pair, and a later test asks for the target. That is not a ward round. It is the cleanest evidence that producing the word beats reading it.

The benefit is for the item, and it is smaller for the surroundings. Item memory was 0.591 for generated words against 0.468 for read words. Memory for the context around the item was 0.557 against 0.525, a reliable but thin gap. Perceptual context, such as the colour or the typeface, went the other way: 0.541 after generation against 0.594 after reading. Producing the dose does not store which column it sat in. Words showed the effect (0.603 against 0.471). Nonwords did not (0.443 against 0.439, p = 0.770). A string nobody could have known is not a generation effect. Older adults still showed it (0.473 against 0.344), and age did not reliably change the size. The long-delay tests, more than a day later, still favoured generation (0.608 against 0.444).

How much of the answer you print matters more than how difficult you make the target. The reviewers coded constraint from the cues on the page. Low constraint was a cue and a blank, with no rule. Medium constraint added a rule, such as "generate an antonym". High constraint added part of the target, as in open–cl___. Lower-constraint generation was remembered at 0.642, medium at 0.586, and higher at 0.528. On a free recall test the spread was wider: 0.484, 0.354, and 0.295. On a recognition test, constraint made no reliable difference. A separate check of difficulty, only in studies that actually varied it, found no extra benefit for the harder version. The advantage over reading was 0.177 for low-difficulty generation and 0.171 for high-difficulty generation. Ask for the term you taught. Do not print the stem, and do not swap in a more obscure synonym in the hope that strain will help.

A passage is a different material, and the tasks are still modest. Schindler and Richter, reviewing text generation in 2023, found that completing letters or words, or putting scrambled sentences back in order, beat reading the intact text (Hedges' g = 0.41, 95% CI 0.31 to 0.52, 129 effects). The same pattern held when learning time was capped (g = 0.45 with a cap, g = 0.37 without one; the difference was not reliable), so this is not only "they spent longer on the worksheet". University samples (g = 0.43) and other samples (g = 0.35) did not differ reliably. People who knew they were meant to learn the text showed the smallest benefit (g = 0.29), which is the situation in a teaching room, and that figure was still reliable. Studies published after 2000 were smaller (g = 0.28) than earlier ones (g = 0.63). The pooled 0.41 includes that older literature.

Length and task both change what you should ask. Texts of 301 to 600 words showed the largest benefit (g = 0.63). Texts over 900 words did not (g = 0.05, p = 0.738), on 12 effects, which the authors say to treat cautiously. For expository text, the kind a briefing is, putting sentences back in order (g = 0.77) beat filling in missing letters (g = 0.23). For a narrative the letter task was the larger one, and that difference was not reliable. These are worksheets on a passage someone has in front of them. They are not a request to write a case note from a blank page.

The lecture comparison is the one that matches a close. Greving and Richter ran the last ten minutes of five sessions in an existing psychology lecture. Fifty-three undergraduates sat the unannounced test in a later session (aged 19 to 47, mean 23.51). Each week they were assigned at random to eight short-answer questions, eight multiple-choice questions, or eight one-sentence summaries, with four minutes a page and no correct answer given. The questions were not taken home. Across items, the short answers raised the probability of a correct response on the surprise test by 17 percentage points (95% CI 5.63 to 27.00, odds ratio 2.06). The gain was clearest for the items that had been hardest to produce in the room, a 27-point rise (95% CI 7.27 to 43.66). Those hard items were still answered correctly somewhere between 4 and 29 percent of the time in practice. The medium and easier bands pointed the same way, and their intervals included zero.

Their multiple-choice arm did not beat reading the summary. The change was 1 percentage point (95% CI −9.48 to 11.48). Those items were not a standard single-best-answer question: more than one option could be correct, and "none of the above" was available. The authors note that a review of classroom testing treats simple, one-correct-option questions as the format that does produce a testing effect, and that their own earlier lecture study, with a simpler multiple-choice format, also found no benefit over restudy once feedback was withheld. Read the null as "this kind of list, with no key, did not beat the summary". Do not read it as "every four-option question is inert".

A health-professions course is the nearer setting, and it did not crown the short answer. van Wijk, de Jonge, van Blankenstein, Janse and Langers compared very short answers (at most four words) with four-option questions, one of them correct, in an extracurricular lifestyle course. Forty-five bachelor and master students completed the quizzes, 71 percent of them medical students, and people already working in healthcare were excluded. Each student did both formats, with no answer and no slides afterwards. Weeks later, content practised as a very short answer scored 51.2 percent on the final test and content practised as multiple choice scored 57.1 percent. The difference was not reliable, F(1, 44) = 3.23, p = 0.08, eta-squared = 0.02, and there was no interaction with the format of the final question. On the night, the short answers were right 58.6 percent of the time and the multiple-choice questions 73.9 percent. There was no group who skipped the quiz, so this study cannot say the quiz itself helped. It can say that, without a key, producing the answer did not beat picking it. Most of the students (31 of 45) still thought the quizzes helped them remember the lectures.

What to do in the last five minutes

Nothing below was tested as a five-minute dose. Greving and Richter used ten minutes for eight items. The five minutes are the slot a recap already occupies.

1. Take the summary sentence down. The comparison that favours a short answer is against reading that sentence. If the words stay on the wall, the room is in the read condition.
2. Ask for the term, the figure, the criterion, or the name of the next step, in a few words. Leave the letters off. A stem that already shows half the word is the high-constraint end, and that end remembered less on a later free recall.
3. If the session was a short sequence, an escalation ladder or a three-step check, ask them to put the steps back in order. For expository text, that reordering beat filling in missing letters. Keep it to the passage you just taught. A handout past 900 words is the length band that lost the benefit.
4. If the blank stays empty, show the answer before they leave. The medical course that withheld the key, and whose short answers failed more often on the night, found no later advantage for producing over picking. One or two misses are compatible with a benefit. A room that cannot reach the target is a different close.
5. Stop at a few items. Eight was the lecture dose. Two is enough when the session was one decision.

A short quiz that asks for a typed answer, and shows the key only after the attempt, is this close. WrapUp Quiz runs that check. A slip of paper, collected before you read the key, is the same activity.

What this does not mean

Producing beats reading. It does not, on its own, beat a well-built single-best-answer question. The lecture null used a list that could have several correct options. The medical course, with one correct option in four, found the two formats close, and the numbers leaned toward the list.

A harder target is not a better target. Difficulty, where studies actually varied it, did not change the size of the generation effect. Fewer cues on the page did. Those are different dials.

The benefit does not spread to the incidental detail. Colour, typeface, and which version of the slide it was are the kind of context generation can make worse. If the teaching point is "which box on the chart", producing the label is the wrong drill.

It does not apply to a string people could not have known. Nonwords showed no generation effect. Asking a room to invent a protocol they were never taught is not this finding.

It does not license a missing-letter worksheet on a long policy. The text tasks are letter completion, word completion, and putting sentences back in order, on passages of a few hundred words. Over 900 words the average was indistinguishable from zero, on a small set of effects. Intentional study, which is what a course is, showed a smaller benefit than the pooled figure.

These studies do not show that the close installs a procedure. The lab tests ask for the word. The text tests ask for the passage. Greving and Richter's surprise test asked the same concepts again, sometimes in a reworded question. van Wijk's questions were recall and understanding. None of them is a later performance on the ward.

The rooms are still mostly students. The age range in the lecture study went to 47, and the meta-analysis included older adults, but the health-professions sample excluded people who had already left university for practice. A one-off CPD day is an extrapolation.

Nothing here supports matching the question to a preferred learning style.

Sources

• McCurdy MP, Viechtbauer W, Sklenar AM, Frankenstein AN, Leshikar ED. Theories of the generation effect and the impact of generation constraint: a meta-analytic review. Psychonomic Bulletin & Review. 2020;27(6):1139-1165.
• Schindler J, Richter T. Text generation benefits learning: a meta-analytic review. Educational Psychology Review. 2023;35:44.
• Greving S, Richter T. Practicing retrieval in university teaching: short-answer questions are beneficial, whereas multiple-choice questions are not. Journal of Cognitive Psychology. 2022;34(5):657-674.
• van Wijk EV, de Jonge M, van Blankenstein FM, Janse RJ, Langers AMJ. The battle of question formats: a comparative study of retrieval practice using very short answer questions and multiple choice questions. BMC Medical Education. 2024;24:1547.