The case you taught is not the next patient

The room can name the effusion on the film you have just taken down. Put up a second film of the same condition, with the landmarks moved, and fewer hands go up. The patient on the ward that evening is not a film at all. Transfer is that gap: using a concept already learned to solve a problem not already seen. Castillo, Park, Harris and colleagues use that definition in a 2018 review of basic-science teaching. A correct answer on the case in the room is not, by itself, evidence the idea will travel.

What the evidence says

The drop from the taught case to a new one is large even when the diagnosis does not change. Rosby, Schmidt, Tan, Low-Beer, Mamede, Zwaan and Rotgans trained 36 year-2 medical students, mean age 20.7, on four chest X-ray conditions. During training each student saw the same four films over and over, picked the diagnosis from a list of eight, and was shown the annotated film again after an error. At test they typed a diagnosis. The trained films scored 3.75 out of 4 and took 4.95 seconds. Near-transfer films, the same conditions with the features rearranged (about 2.87 feature differences from the original), scored 2.08 out of 4 and took 15.77 seconds. Far-transfer films, the same conditions missing one or more of those features (about 3.88 differences), scored 1.31 out of 4 and took 18.80 seconds. Conditions they had never been trained on scored 0.72 out of 4 and took 19.46 seconds. Trained beat every other band (p < 0.001). Near beat far (p < 0.001). Far still beat untrained on accuracy (p = 0.03), and the times for those two bands did not differ (p = 0.69). The near-versus-far time gap was marginal (p = 0.06).

Two weeks later, 35 of them sat the same 16 films. Accuracy had fallen but the order held: trained 2.29 out of 4, near transfer 1.57, far transfer 1.17, untrained 0.89. Trained remained higher than the other three (p < 0.01). Near versus far was marginal (p = 0.055). Far versus untrained was not reliable (p = 0.20). The training had never varied the film. The authors note that a mix of versions during practice might have transferred further. What the study shows is narrower: speed and accuracy on one image do not survive a second image of the same condition, and a fortnight later the more distant image is not reliably better than a condition nobody taught.

A second context helps when the test really is a new context, and a third adds little. Kulasegaram, Chaudhary, Woods, Dore, Neville and Norman taught three physiology ideas (laminar and turbulent flow, Laplace's law, Starling's law) to first-year psychology students who had not studied them. Ninety were recruited and two were excluded for not finishing the immediate test, leaving 88, with 15 planned per cell. Everyone had to score at least 8 out of 10 on a true-or-false check of the ideas before the transfer cases. Explanations of a new clinical vignette were marked from 0 to 3. Overall, cases in a familiar organ system scored 1.09 (SD 0.51) and cases in an unfamiliar organ system scored 0.96 (SD 0.45), F(1, 82) = 5.92, p < 0.017, partial eta-squared = 0.07.

Where they had practised changed which of those two scores held up. One organ system produced the highest near-transfer score, 1.28 (SD 0.57), and the lowest far-transfer score, 0.78 (SD 0.34). Two organ systems scored 0.94 (SD 0.41) near and 1.03 (SD 0.48) far. Three scored 1.04 (SD 0.47) near and 1.07 (SD 0.46) far. The authors report that adding contexts improved far transfer, and that two and three contexts did not differ. An everyday analogy for the underlying idea did the same kind of work. Far transfer was 1.06 (SD 0.50) with the analogy and 0.86 (SD 0.37) without it, F(1, 82) = 4.6, p = 0.035. The analogy group kept a similar score when the organ changed. The no-analogy group was higher on the familiar organ and dropped on the new one. A week later, 75 of the 88 returned. Far transfer fell from 0.98 (SD 0.41) to 0.85 (SD 0.38), F(1, 69) = 8.06, p < 0.006. Near transfer did not fall reliably. The pattern of who dropped, and who did not, was the same as on the day.

A recall score did not tell those groups apart. In a second experiment, 40 further psychology students, 10 in each of four groups, sat a 15-item multiple-choice test of the same explanations. The four means ran from 8.45 to 10.22 out of 15, and the groups did not differ. They then picked which of three new cases was most like a target case, across 10 classifications. The group with no analogy and one organ system made 2.89 matches on the surface (same organ, different idea) and 2.5 matches on the idea (different organ, same idea). The group with the analogy and two organ systems made 1.79 surface matches and 5.58 idea matches. The authors' own reading is that novices default to the surface, and that naming the structure, or showing the idea in a second context, shifts that default. They also say the studies are small, the learners were novices, and the results should not be treated as a prescription to attach an analogy to every lecture.

The health-professions literature has very few tests of this at all. Castillo and colleagues screened 9,803 papers on undergraduate basic-science teaching and included 15. Fourteen of the 15 studied transfer out, applying a taught idea to a new problem. One studied transfer in, using old knowledge to learn a new explanation. The teaching approaches that documented transfer were links between the basic and clinical sciences, commonsense analogies, and several clinical problems in more than one context. Performance on recall tests did not reflect transfer to clinical reasoning. Diagnostic accuracy on a new case was one of the measures that did.

Practice testing can carry to a new question, and it often does not. Pan and Rickard pooled 192 transfer effects from 122 experiments in 67 articles (N = 10,382), testing against a restudy or reread control rather than against no practice. The random-effects estimate was d = 0.40 (95% CI 0.31 to 0.50). Transfer was greatest when only the test format changed, when the final questions asked for application or inference, when the problems were medical diagnosis and treatment, and when the cue was a mediator or a related word. It was weakest when the cue and the response were rearranged, when the final question asked about material that had been seen but never tested, and when the problems were worked examples. Three features strongly influenced whether the benefit appeared: the new question still calling for the same answer (they call this response congruency), elaborated retrieval practice, and performance on the first test. Allowing for publication bias, the estimate often fell to no positive transfer when none of those three was present. The medical-diagnosis results in that pool are new problems inside the same studies. They are not a later shift on a ward.

The feedback on a new question is a separate, smaller effect, measured in medical students. Ryan, Judd, Swanson, Larsen, Elliott, Tzanetos and Kulasegaram recruited year-2 students in Melbourne (41) and Toronto (25). Two of the Toronto students missed the test a week later. Each sat 17 single-best-answer questions, then saw one of three keys: right or wrong plus the correct option; a short note on why each option was right or wrong; or a longer note on the concept underneath. They then answered near-transfer items, which changed surface details such as age and sex and kept the decision, and far-transfer items, which used the concept for a different decision. Across the immediate test and the one-week test, items scored 51.3 percent after right-or-wrong feedback, 56.3 percent after the option notes, and 59.4 percent after the concept note, F(2, 330) = 7.4, p < 0.001. The two longer forms beat right-or-wrong (effect sizes 0.2 and 0.4) and did not differ reliably from each other. Near-transfer items scored 59.1 percent and far-transfer items 52.2 percent, F(1, 330) = 15.61, p < 0.0001. Feedback type did not interact with near versus far. The one-week scores were not reliably different from the immediate scores. Median time on the key was 3 seconds for right or wrong, 25 seconds for the option notes, and 52 seconds for the concept note. Melbourne averaged 57 percent and Toronto 53 percent, with no site difference. Both cohorts were early in clinical work or still pre-clinical. This is a transfer between written questions, one week apart, not a patient.

A simulation is the place people hope the ward has been rehearsed. Lavoie, Lapierre, Maheu-Cadotte, Fontaine, Khetir and Bélisle mapped quantitative studies of clinical decision-making after simulation in nursing or medicine, searching in May 2020. From 5,969 records they included 61 articles. Seven (11 percent) assessed transfer to clinical practice. The other 54 assessed a later simulation only. Of the seven, five looked only at patients whose conditions were similar to the scenario, and two mixed a later simulation with practice. In the discussion, the reviewers note that patients in these studies had problems resembling the simulation, with one study not specific on that point, and that there is too little evidence, and too much heterogeneity, for a synthesis. There is less still on licensed professionals: registered nurses were the sample in 4 of the 61 articles. Sample sizes across the 61 ran from 11 to 266, median 59. The review did not estimate an effect. It estimated how rarely the ward is the outcome.

What to do in the last five minutes

Nothing below was tested as a five-minute dose. Rosby's training was a repeated image, not a close. Kulasegaram's practice was three cases per idea. Ryan's test ran to two hours. The five minutes are the slot a recap already occupies.

1. Take the taught case down. Another look at that film is the condition Rosby's students scored 3.75 out of 4 on. The rearranged film of the same condition scored 2.08. If the close uses the same chart, you have not asked for transfer.
2. Put up one new case that keeps the decision and changes the surface: a different organ, a different set of numbers, a different shift. In the physiology study, far transfer was 0.78 out of 3 after one organ system and 1.03 after two. The third organ system was 1.07, and two and three did not differ. The familiar organ was already the high score after a single system (1.28). The second case is for the patient who does not match the example, not a second rehearsal of the example.
3. Name the conserved idea in ordinary words before they attempt that case. Far transfer was 1.06 out of 3 with an analogy for the structure and 0.86 without it. The authors say this is not a licence to decorate every session with an analogy. It is evidence that the score held up when the structure had been named, and fell when only the organ had been practised.
4. After the attempt, give the reason, not only a tick. Transfer questions scored 51.3 percent after right or wrong, 56.3 percent after a note on each option, and 59.4 percent after a note on the concept. The two longer keys did not separate. Near-transfer questions still beat far-transfer questions. A tick leaves the room in the weaker arm.
5. Stop at that one new case. If they cannot do the case you taught, the new one is the wrong next step: Pan and Rickard found that transfer of practice testing depended on how the first test had gone. Show the taught answer, then stop.

A short quiz that asks the same decision in a second case, and shows the reason only after the attempt, is this close. WrapUp Quiz runs that check. A second vignette on a slip, collected before you read the key, is the same activity.

What this does not mean

A pass on the original case is not transfer. Castillo and colleagues found that recall tests did not reflect transfer of basic science to clinical reasoning. In Kulasegaram's second experiment the four groups did not differ on a 15-item recall test, and they still sorted new cases differently. Rosby's students were near ceiling on the film they had drilled and well below it on the next film of the same condition.

Drilling one image until it is fast is not a new patient. The training used the identical film, ten times and then under a time cap. Two weeks later the far-transfer films, 1.17 out of 4, were not reliably better than conditions never trained, 0.89 out of 4. The authors themselves treat a mix of versions during practice as an untested improvement.

Two organ systems in a laboratory are not a ward round. The learners were first-year psychology students, chosen because medical students at that school had already met the physiology. Means sat near 1 out of 3. The authors say the results may not translate straight into a course, and that they are not a stable estimate of the effect of analogies.

A realistic simulation is not evidence of ward performance. Seven of 61 studies looked at patients, and those patients usually had problems like the scenario. The review offers no pooled effect. Licensed staff are the thinner slice of an already thin set.

Practice testing is not a guarantee that the learning will travel. The d = 0.40 is against restudy, across forty years of studies, and the bias-adjusted estimate often showed no transfer when the new question did not still want the same answer, the practice was not elaborated, and the first attempt had gone badly. Worked examples were among the weakest categories. Watching a fully worked case and then receiving a harder one is the comparison that did least, not a model for the close.

Nothing here supports matching the question to a preferred learning style.

Sources

• Rosby LV, Schmidt HG, Tan GJS, Low-Beer N, Mamede S, Zwaan L, Rotgans JI. Promotion of knowledge transfer and retention in year 2 medical students using an online training exercise. Advances in Health Sciences Education. 2021;26(3):1059-1074.
• Kulasegaram KM, Chaudhary Z, Woods N, Dore K, Neville A, Norman G. Contexts, concepts and cognition: principles for the transfer of basic science knowledge. Medical Education. 2017;51:184-195.
• Castillo JM, Park YS, Harris I, Cheung JJH, Sood L, Clark MD, Kulasegaram K, Brydges R, Norman G, Woods N. A critical narrative review of transfer of basic science knowledge in health professions education. Medical Education. 2018;52(6):592-604.
• Pan SC, Rickard TC. Transfer of test-enhanced learning: meta-analytic review and synthesis. Psychological Bulletin. 2018;144(7):710-756.
• Ryan A, Judd T, Swanson D, Larsen DP, Elliott S, Tzanetos K, Kulasegaram K. Beyond right or wrong: more effective feedback for formative multiple-choice tests. Perspectives on Medical Education. 2020;9(5):307-313.
• Lavoie P, Lapierre A, Maheu-Cadotte MA, Fontaine G, Khetir I, Bélisle M. Transfer of clinical decision-making-related learning outcomes following simulation-based education in nursing and medicine: a scoping review. Academic Medicine. 2022;97(5):738-746.