Every AI Brainstorm is a List of Survivors

During World War II, the American military kept losing bombers, and they wanted to know where to add armor. You can't armor the whole plane. Armor is heavy, and a plane wearing too much of it never gets off the ground. So the question was narrow and practical: given a limited amount, where does it go?

They had good data. Planes came back from missions over Europe peppered with bullet holes, and when analysts mapped where the holes clustered, a clear pattern emerged. The wings were full of holes. The tail was riddled. The engines, oddly, were mostly intact. The obvious move was to reinforce the places taking the most fire, and if you had handed me the maps, that is exactly what I would have done. It's umm, where all the holes are.

A statistician named Abraham Wald looked at the same data and said no. Armor the engines.

His reasoning was brilliant, actually. The holes weren't showing where bombers got hit. They were showing where a bomber could get hit and still make it home. The planes they were looking at were the survivors. The ones hit in the engines weren't in the data at all, because they were at the bottom of the English Channel. The single most important fact in the entire study was sitting inside the planes that never came back, and not one of them was in the room.

I've been thinking about Wald a lot lately¹, because we've built a machine that does to ideas what survival did to those bombers.

When a student opens a chat window and asks for ten ways into an essay, or five concepts for a project, or the strongest counterarguments to their thesis, the tool returns something genuinely good. Reasonable, organized, accurate. The student picks from the list.

What never appears on the list is the strange idea, the half-formed one that surfaced from a dinner-table conversation three days ago, the one that doesn't quite make sense yet (and might have become the best thing they ever made).

Those ideas are the bombers in the Channel. They got filtered out before anyone saw them, and because they were never on the screen, nobody mourns them. I've been calling this the Unseen Unseen.

What makes it worse than Wald's problem is that his data was incomplete in a way you could reason about. He could look at a clean engine and infer the missing planes. A student looking at a tidy list of ten ideas has nothing to infer from. The list looks finished. It has the texture of completeness. That's the trap. The answer is accurate, so we stop looking, and the stopping feels like critical thinking.

So here is what I actually do, and it costs nothing. Before anyone opens a laptop, students discuss and brainstorm at their tables. Then they do a five-minute freewrite in a blank Google Docs. No AI. Five minutes of human guessing, bad guesses included (bad guesses especially.)

Then they can use AI as instructed, for the activity.

Those five minutes generate the planes that wouldn't have come back.² You make the missing data yourself, before the filter has a chance to remove it. When the AI list finally shows up, the student already has something human to hold it against, and more often than not the odd thing scrawled on the page is more interesting than anything on the screen. Sometimes it isn't, and they go with the AI version, and that's fine. The win is that they chose against a real alternative instead of picking from a list that only ever shows you the survivors.

Wald could only study the planes that made it back. He had to imagine the rest. The blank page is the one place where, for five minutes, every plane comes home.