You Didn’t Learn It. You Only Read It.

By Martin Bergmann | AI Project Lab | November 2026

Before you read on, try this. Without scrolling back, name the three main points of the last document an AI summarized for you. If you hesitated, that hesitation is what this post is about.

The experiment that should worry you!

In 2006, two psychologists, Henry Roediger and Jeffrey Karpicke, asked students to learn a short passage. One group read it four times. The other read it once and then took recall tests.

Five minutes later, the rereaders did better, as you would expect with the text fresh in their minds. A week later, the result flipped. The tested group remembered 61 percent of the passage. The rereaders remembered 40 percent, even though they had read it about four times as often.

The twist is what the rereaders believed. They were the confident ones. They predicted they would remember more, and they were wrong.

Rereading feels like learning because it is smooth. That smoothness is the problem.

Your AI summary is smooth too.

An AI summary is the smoothest rereading ever invented: fast, tidy, complete, and you receive it without retrieving anything. That is my analogy, not a research finding, and the nearest evidence is not about summaries.

In a 2025 experiment in PNAS, researchers led by Hamsa Bastani gave nearly 1,000 high school math students in Turkey different kinds of practice. Students who practiced with a ChatGPT-style assistant did better during practice. When the AI was taken away for the exam, they scored 17 percent worse than students who had only their textbook. A version designed to give hints rather than answers left students at the same level as the textbook group.

It was one school and one subject, so I would not stretch it to the office. But the direction deserves attention. It is also in the same family as my post on prompt libraries: what you work out yourself tends to stay with you better than what you are handed.

Make the AI quiz you.

The fix costs nothing. After an AI summarizes something, flip the roles: “Ask me five questions about this document, one at a time. Wait for my answer before you correct me.” Do it again two days later with the ones you missed. Testing yourself and spacing it out are the two techniques that top a 2013 review of study methods by John Dunlosky and colleagues. Rereading sits near the bottom.

Use it before a steering meeting, when you are learning a new process, or when your team picks up a new tool. Two cautions: AI can write a wrong question, so check surprising ones against the source, and I have not checked whether AI-written questions work as well as human-written ones.

The summary tells you what is in the document. Only the quiz tells you what is in your head.

Now try the opening question again. How many of the three points can you name?

---

Martin Bergmann is the Director of an IT PMO and founder of the AI Project Lab. He writes about AI, project management, and the future of intelligent work.