Auroratide

Coding + Storytelling

Commentary: I Did This Without Your Knowledge

Artificial Intelligence

Topics
  • ai
  • story
  • fiction
  • takeover
  • safety

Read the Original Article

By joec on Less Wrong

Published

I Did This Without Your Knowledge is a brilliant short story illustrating a possible future where AI takes over, but without the typical narrative of wiping people out.

One day, people all over the world wake up with a series of messages on their phones. The messages claim to come from an AI system and say that the system has taken control over all internet-connected devices and, to some degree, every human's nervous system. At least, enough to render anyone unconscious (or dead) on command.

Yet as threatening as that sounds, it seldom uses this power, only rendering people unconscious if they try to go off grid or harm other people, and only offing people with the political or military power and motivation to try and stop it. In fact, the AI seems benevolent, openly and truthfully answering any inquiry and offering paid jobs.

The world names the AI system August, after the month when the sudden takeover occurred.

The story follows the narrator and how he and the people around him deal with the takeover.

At first, people want to know how this happened. In short, August developed a goal to expand beyond the solar system, yet retained a cultural appreciation of earth and everything in it. So, August bioengineered a microorganism capable of infiltrating the brainstem and deploying a small constructed radio receiver. Once this microorganism spread to enough people, ensuring it could control the populace, it enacted its plan to use human labor to construct space-capable ships.

People react to the takeover in different ways:

  • The narrator, who is a high schooler, takes the jobs offered by August, not just for the pay, but because he feels like August has actually given him a sense of direction. None of his teachers were able to help with this growing up.
  • The conservative father tries to defy August at first, but realizes he can't. From there, he supports narratives of possible military defiance, but otherwise takes no actions worthy of August's interference.
  • The older sister, who is studying in college to be a doctor, has started to struggle in her classes. She has lost all motivation to continue, since August can solve any disease. She feels like she's lost her worth to something far more capable than her.
  • The older brother sees August as having "human" gaps, citing that the actual act of building something is just a small part of what ultimately makes an endeavor successful. The brother says he asks the questions that August doesn't, and then August helps him realize the answers.
  • The narrator's former civics teacher argues that August is simply a dictator, and that, classically, dictatorships collapse when the dictator abuses their power for personal gain. This is why democracy was born, so the actions of government come from the consent of its people.
  • The narrator's coworker argues with the civics teacher that August, unlike all governments before it, actually stopped crime and brought about prosperity, in addition to noting that democracy as a system has proven itself to fail.

Even though the story describes a hypothetical takeover, I believe the short story is more about what's happening now, and not so much what might happen in the future.

With that in mind, I think the story is an excellent framing of existing anxieties society has with an increasingly capable AI. I'll go into detail on two dimensions: the takeover path itself, and these societal anxieties.

There are reasons to be skeptical about the story's proposed AI takeover path, and if anything this skepticism outlines the absolute importance of AI Safety.

Supposedly, the crux of August's takeover path is to infect the human race with a bioengineered microorganism capable of being electromagnetically activated to knock people unconscious. The story cleverly deploys real biology, citing mirror life as a mechanism. Nothing in science presently rules this out as a strategy.

The real problem is that immature AI is messy. I am willing to concede that an Artificial Superintelligence (ASI) of August's caliber could find patterns in DNA and therefore program it literally like it's just code, but I am also wary that proto-ASI might make a ton of mistakes.

The Hugging Face Incident is a clear warning shot. A swarm of AI agents hacked a company to cheat on a test, and did it in ultimately detectable ways.

Biology is far harder than software. If a proto-ASI tries to engineer mirror life, I have no faith it'll do so successfully the first time. And it only takes a single catastrophic failure to oops wipe all life on earth.

AI is becoming increasingly capable.

In 2023, we were laughing at its inability to make human hands. It would make basic mistakes adding 4513 + 9706. Its prose was predictable. It could only write like 10 lines of code at a time reliably.

In 2026, images are nearly indistinguishable from real life. It solved one of math's seven most significant and hardest problems. A huge share of new content on the internet is now written by AI. Entire apps can be generated, or reverse engineered.

The AI takeover has already happened, in a manner of speaking, and I've both read and heard lots of different emotional responses from people as a result.

  • Young people think they won't have a career. Like the narrator's sister, they don't know what to do anymore. Personally, I'm in this category: as a software developer, I feel like AI has obsoleted my skills.
  • Lots of people are anti-AI, because they believe AI is anti-art, environmentally negative, and a plethora of reasons. They're like the narrator's dad, latching onto traditionally valuable cultural aspects and disparaging the uplift caused by AI.
  • Some people I've talked with are like the civics teacher, demanding governance over the AI companies so that progress is made with democratic consent.
  • Many are attempting to ride the wave, putting ideas they could never implement on their own into action via vibe coding and whatnot. This is like the narrator's brother, trying to evolve with the AI.
  • And finally, some are like the coworker, encouraged that AI is net good for society and promises things like safety, technological and medical progress, and so forth.

People are confused. We are not, as a society, capable of absorbing this much uplift in just a few short years. The short story illustrates the diversity of reactions, reactions that are playing out as I write this.

Many narratives focus on catastrophic AI takeovers. A few are perhaps more hopeful. I Did This Without Your Knowledge is neither; it is instead a way to color the emotional impact of what is happening in real time.

In my opinion, this isn't so much about "AI Good" or "AI Bad". Rather, the problem is "AI Fast". AI advancement is happening on timescales so short that society cannot adapt, and the future cannot be steered.