What Fluency Conceals
The most dangerous answer is the one that feels complete.
I moved to the United States at sixteen, and for the first several months, I was fluent in almost nothing that mattered.
Not the language. I spoke English. But the operating system was different, and no one had thought to hand me the manual.
My first week, I missed the school bus three times. Not because I couldn’t read the schedule, but because no one told me there was one, or which of the fifty yellow buses waiting outside after school was mine. My homeroom teacher and the driver had simply said see you in the afternoon, expecting a sixteen year old to navigate a hundred page district document to find a bus number, a time, and a pickup location. The information existed. The context did not.
At basketball tryouts, I knew the game but not the language. I had played varsity in Addis Ababa, but the positions I had learned were named differently; spatially, relationally, built around the geometry of the net in ways that made sense to students who had grown up learning them that way. The coaches calling plays in American shorthand might as well have been running a different sport. I did not make the team. Not because I couldn’t play. Because the system had no mechanism to recognize what I knew in the language I knew it.
On my first day of class, I did not stand for the pledge of allegiance. I didn’t know what was happening. Someone scolded me before anyone thought to explain.
The system was not lying to me. It simply never considered that I might not already know.
And here is what I have come to understand about those months: the system felt complete to everyone it was built for. The bus routes made sense. The plays were obvious. The pledge was automatic. I was the only one who could see the gap. And the system had no mechanism to make that gap visible to anyone else.
I have been thinking about that structure again lately. Because I have encountered it a second time. Different technology. Same design.
The issue is not that AI is inaccurate. It is that AI makes partial knowledge feel sufficient.
That distinction is worth sitting with. An inaccurate system can be corrected. Errors surface. People push back, verify, find the mistake. But a system that is mostly accurate, confidently presented, and structurally incomplete in ways that are invisible to the user — that system is harder to question, because it rarely gives you a reason to.
This is what I mean by epistemic incompleteness. Not bias in the conventional sense, a skew that might be fixed with better data or more diverse training sets. Something more structural. These systems are built on documented and digitized knowledge, which means they exclude, by definition, forms of understanding that are oral, contextual, embodied, or relational. The incompleteness is not incidental. It is constitutive. And it does not announce itself.
At a recent AI forum in Addis Ababa, the city I grew up in, someone suggested that as little as six percent of the world’s knowledge has been digitized in forms that make it into training data.
The room went quiet.
Not because the number was verified. But because no one could confidently say it was wrong.
Six percent. Which means the systems now mediating how billions of people learn, decide, and understand the world may be drawing on a fraction of what human beings actually know. The oral traditions are not there. The community practices are not there. The regional histories, the knowledge held in relationships and seasons and repetition rather than in documents — largely not there. And the systems built on what remains do not signal the gap. They answer with the same confidence regardless of whether they are drawing on centuries of accumulated knowledge or the thin edge of an archive that was never designed to be complete.
That silence in the room was not about the number. It was about what the number implied. We are building systems of extraordinary consequence, and we have not been asked to find out what they do not contain.
Consider a scene that is playing out millions of times a day.
A person sits on their couch, planning something that matters. A trip. A move. A decision about their health or their child’s education. Instead of searching through multiple sources, they turn to an AI system and ask for guidance. The response comes back quickly. Clear. Structured. Confident. It outlines options, offers recommendations, presents a path forward. It feels thoughtful. It feels complete.
There is no visible indication of what is missing. No sense of which perspectives were included, which were excluded, or how the answer was shaped by the data it draws from. The person does not treat the response as one input among many. They act on it. Not because it is perfect, but because it is sufficient.
In that moment, a useful approximation quietly becomes the basis for a real world decision.
Nothing in the interaction signals that this is what happened. The system did not flag uncertainty. It did not name its limits. It delivered its answer the same way it always does — fluently, completely, as if the question had been fully resolved.
This is not a malfunction. It is the system working exactly as designed. That is the part worth examining carefully.
The stated objectives of most major AI systems cluster around usefulness, safety, and alignment. These are reasonable goals. But in practice, users frequently experience these systems as something different: authoritative, complete, good enough to act on. That gap between what the system was designed to optimize for and what users actually experience is not explicitly built in. It emerges. And once it emerges, it is self-reinforcing.
A system that signals its own limits feels like a worse product. It introduces friction. It creates doubt. It asks the user to do more work at the exact moment they came to the system to do less. So the incentive — not malicious, not conspiratorial, simply structural — runs toward fluency. Toward confidence. Toward the answer that feels complete rather than the answer that accurately represents what the system does and does not know.
The result is a quiet collapse of approximation into authority.
AI systems were not designed to be the final word. But they are increasingly experienced as exactly that. Not because users are credulous or careless. Because nothing in the interaction suggests they should be anything else.
This is what I could not articulate at sixteen, standing outside a row of fifty identical yellow buses with no idea which one was mine. The system was functioning correctly. It was just functioning correctly for people who already knew how it worked. My not knowing was not accounted for. And because it was not accounted for, it was invisible to the system, to the people running it, to everyone except me.
At scale, with AI, that invisible not-knowing belongs to most of the world.
There are things that can be done.
More representative training data. Investment in language inclusion that goes beyond translation. Systems built in partnership with communities rather than extracted from them. Models designed to surface uncertainty rather than paper over it. These are real steps and they matter.
But they do not resolve the deeper question.
If a system is designed to feel complete, and users have no reliable way to know when it isn’t, the problem is not only what the system contains. It is what the system does to how people relate to knowledge itself. Over time, a generation of users habituated to receiving confident, frictionless answers may lose the instinct to question, to verify, to seek what the answer did not include. The system does not just reflect how knowledge is held. It reshapes how knowledge is sought.
That is a consequence no training update fully addresses.
AI systems should not collapse approximation into authority. That is not a technical requirement. It is an ethical one. And it raises a question the field has not yet seriously answered.
If the system gives a confident answer based on incomplete knowledge, and the user has no way of knowing that, does the system bear any responsibility for how that perception is formed?
I do not think that question has a clean answer. But I think we are past the point where we can avoid asking it.
The system felt complete to everyone it was built for. That was always the problem. And it is still the problem now. Only the system is larger, the users are more numerous, and the answers arrive faster than anyone can think to question them.