Ten thousand years before the events of Dune, there was a war. Humanity rose up against the thinking machines that had come to run its life, destroyed them, and wrote a single commandment into every religion that survived: thou shalt not make a machine in the likeness of a human mind. Frank Herbert called it the Butlerian Jihad, and it is the hidden foundation of the whole book. Everything strange about the Dune universe — the human computers called Mentats who are trained from childhood to calculate like machines, the Spacing Guild navigators who fold space with their drug-expanded minds, the Bene Gesserit sisterhood breeding for a superhuman — exists because a civilization decided, once and forever, to be afraid of the thing I am telling you is already here.
And the result is not a paradise. The result is a feudal empire run on a single narcotic, where a handful of aristocrats hold power precisely because the tools that would have made every ordinary person capable were banned. The Butlerian Jihad did not free humanity from the machine. It handed the machine’s work to a priesthood and called it freedom.
Herbert gave his hero a litany to recite when he is afraid, and the first line of it is four words long: fear is the mind-killer. I think that line is the most important sentence in science fiction, and I think it is about the singularity.
Because the dramatic map — the Terminator map, the red-eyed-robot map — is a map drawn by fear, and fear draws badly. Fear makes you see agency where there is none. It makes you see a will in the machine, because a will is something you know how to fight. It cannot see the checkbox, the tired product manager, the sentence under the box, because those are boring and fear needs a monster.
The honest territory is more boring and more serious at once. The machine does not want anything. It will not rise up. It will do exactly, and only, what the hands on the checkboxes allow — which means the real question of this century is not “will the machine turn on us” but “who gets to hold the checkboxes, and do the rest of us get to read the sentence.”
Which brings me to Margaret Atwood.
In The Handmaid’s Tale, Atwood built a near-future America, called Gilead, in which the first thing the new regime does — before the uniforms, before the ceremonies — is take away words.
Women are forbidden to read. Shop signs are replaced with pictures.
The handmaids are stripped of their names and given a possessive instead: Offred, Ofglen, Ofwarren. Of-Fred. Property of.
Atwood, who is from this city, has said for decades that she put nothing in that book that had not already happened somewhere. And her genius was to understand what Charlotte understood from the other side: whoever controls the words controls the pig. Gilead does not need to convince anyone that women are property. It only needs to make “property” the only word available. The medium — a system in which one half of the population cannot read — is the message.
Put Herbert and Atwood together and you get the two ways the singularity goes wrong, and neither of them is a robot.
The Dune failure is to ban the thinking machine out of fear, and thereby hand its power to a priesthood — the few who are allowed to think — and build a feudal world on top of it. The Gilead failure is to keep the machine and take away the people’s words, so that the few who can read the web decide what it says and everyone else just looks up at the pig.
Both are failures of the same thing. Both take the tool away from the ordinary person. And the ordinary person — the one with the doughnut, the one at the register — is the entire reason the tool should exist.
Star Trek: Picard put both failures in one story, and I want to stop on it because it is the clearest recent picture of what Herbert was warning about.
In the show’s history, a group of synthetic workers attacks Mars and kills tens of thousands of people. The Federation — the most enlightened civilisation in all of fiction — responds by banning synthetic life outright. Research stops. The scientists who built the synths are disgraced. The institute that made them is shuttered. It is the Butlerian Jihad in a Starfleet uniform, and for most of the first season it looks like a reasonable response to a massacre.
Then you find out where the fear came from. A secret order has spent centuries guarding a vision — the Admonition — that shows the end of all organic life at the hands of machines, and they engineered the Mars attack to force the ban. The fear was not a reaction to the technology. The fear was manufactured, by a priesthood, to take the technology out of ordinary hands. And it worked, because fear always works, because a monster is easier to believe in than a checkbox.
The people who lost were not the synths. They were the engineers — Maddox, Jurati, the researchers at Daystrom — who had been building the thing carefully and were told to stop. People, in other words, like me. The ban did not protect anyone. It just decided who was allowed to think.
I think about this when I think about my parents and reading.
My mother and father were not readers in the way the word is used in a university. They were readers in the way that matters: they could read the bus schedule, the lease, the pay stub, the price tag, the warning label, the form that had to be filled out to keep the family going. They could read the sentence under the checkbox. In the Gilead of Atwood’s book, that is the first thing taken, because it is the thing that makes a person ungovernable by anyone but themselves.
The singularity, as it actually arrived, did something extraordinary that fear cannot see: it put the most powerful tool ever built behind an interface that every reader can use. You do not need to code. You do not need a Mentat’s training. You need the ability my parents had, which is to read a sentence and ask a question in your own words. That is the whole barrier. It is lower than the barrier to a driver’s licence.
So the stakes, as I see them, are not whether the machine thinks. The stakes are whether we keep it that way — a tool in the hands of the person at the register — or whether, out of Dune’s fear or Gilead’s appetite, we let it become something only a priesthood is allowed to touch.
Fear is the mind-killer. Not because the fearful are stupid, but because fear always, every time, in every story, ends by taking the tool away from the people who need it most and handing it to the people who already have everything. I would rather be afraid of that.
