All posts
22 MARCH 2023 · · 5 MIN

Open-book exams, AI, and what learning is actually for

Open-book exams, AI, and what learning is actually for
There's a specific memory I have from university where the open-book exams were the ones I learned the most from. The closed-book exams I crammed for and forgot. The open-book exams I prepared for differently - knowing I could look things up changed what I bothered to memorise and what I tried to actually understand. AI is the latest version of the same dynamic at a much wider scale, and most of the pedagogical establishment is still working out how to respond to it.

The cognitive-science literature on open-book learning has been consistent for decades. Open-book exams produce deeper retention than closed-book ones, because they shift cognitive load from memorisation to synthesis. When you can look up the formula, the answer to 'do I memorise this' becomes 'no, I memorise the conditions under which it applies'. That conditional understanding is a different thing from rote recall, and it's what makes the knowledge usable later when the exam isn't there to define what you should remember.

What changes when AI is the open book

AI extends the open-book principle in a way that doesn't quite fit the traditional category. A textbook is fixed; you can look things up but the things you look up are bounded. AI can synthesise across vast corpora, answer questions in ways that no static reference can, and adapt to your specific question rather than the chapter you happen to be reading. The reference becomes interactive in a way that changes what the user has to do to use it well.

The skill that gets selected for in this environment is asking the right question. AI rewards good prompts and exposes the limits of bad ones. A student who has formulated their question precisely gets a useful answer. A student who is fishing gets something that sounds like an answer but doesn't help them learn. The new skill is therefore a kind of meta-cognition: knowing what you need to know in order to use the tool effectively. This is harder to teach than the old skill of memorisation, and the curriculum has barely begun adapting.

What gets lost

There's a real cost to the shift, and it deserves more honest discussion than it usually gets. The pre-AI knowledge floor is what makes AI-as-open-book actually useful. Without enough background to evaluate what the AI tells you, the AI becomes a black box that produces answers with no learning attached. Junior engineers using AI as a study aid score higher on synthesis questions and lower on conceptual understanding than control groups, in the research that's emerging. The shift in skill profile is real, and the loss in conceptual depth is also real.

This isn't a reason to ban AI from learning environments. It is a reason to think carefully about what's worth teaching with the AI present and what's worth teaching with it absent. The conceptual scaffolding has to come from somewhere, and if students never struggle with the unaided version of a problem, they don't develop the intuitions that make the AI-aided version useful. The right pedagogical response is probably some mix - closed-book foundational learning, open-book practical application, AI-aided synthesis - rather than a single mode applied uniformly.

The institutional response

Educational institutions are split, often along lines that reveal what they're really trying to do. Institutions banning AI use in assessments are usually optimising for backward-compatible signal - the marks need to mean what they used to. Institutions integrating AI openly are usually optimising for forward-relevant skill - the graduates need to be useful in environments where AI is already standard. Both are defensible positions for different parts of the educational ecosystem. The single right answer doesn't exist.

The institutions integrating AI most successfully are the ones that already had students with strong foundational skills coming in. Cambridge, MIT, top-tier schools generally have students who developed knowledge floors before AI was in their hands, and AI use accelerates rather than replaces the foundational work. Schools whose students don't have the foundational layer when AI is introduced tend to produce graduates who can use AI fluently for surface tasks but can't evaluate or extend its outputs. The gap is widening rather than narrowing, and the equity implications of this divergence are concerning.

AI is an open book. You still need to read enough other books to know what to ask.

The professional analogue

What I've seen on engineering teams matches the educational picture. People who treat AI as an open book - to look things up, verify, synthesise - develop differently than people who treat it as an answer dispenser. Same tool, different mental models, different professional trajectories. The mental model matters more than the tool. The senior engineers I respect most use AI heavily, but always in the context of their existing technical understanding. Junior engineers who skip the foundational work and rely on AI for synthesis often produce code that works in the demo but breaks in production, and the gap shows up around month nine.

The practical implication for engineering managers: invest in your team's conceptual foundations explicitly. Code reviews that explain not just what to change but why. Design discussions that surface the underlying principles. Mentorship that's about building the knowledge floor that AI rests on. The teams that do this produce engineers who use AI as a force multiplier on their own thinking. The teams that don't produce engineers whose thinking has been quietly outsourced to a tool whose outputs they can't reliably evaluate.

The broader frame

Every generation reinvents the open-book debate with the technology of its era. Calculators in maths classes. Internet search at the start of the 2000s. AI now. The pattern is the same - initial panic, period of bad practice, gradual integration as curricula adapt, eventual recognition that the new tool requires different but not lesser learning. The current AI panic is following the same cycle as previous waves. The honest position is to take the long view: AI will become a normal part of learning, the curriculum will adapt, the deeper question of what's worth knowing in an AI-augmented world will resolve over a decade rather than overnight.

For individual learners - and this is where the original observation lands well - the right mode is to use AI as your open book, but to also do the harder work of reading enough other books, thinking through enough hard problems, and developing the conceptual scaffolding that lets you ask AI good questions. The synthesis between strong foundations and good tool use is where the real learning happens. Either alone is insufficient. Together they extend what's possible considerably beyond what closed-book learning ever did.

← Previous
If you're reading this on a weekend
Next →
Leadership communication isn't a soft skill - it's the work

Discussion

Email used only for your avatar. Never shown, never stored in plain text.