There is a threshold humanity has kept, however badly, for all of history. A person decided to take a life. A finger, a hand, an order, a will — a human somewhere in the loop, someone who could hesitate, refuse, be judged. We are now deliberately engineering that person out of the loop. And we are doing it fast.
This is the most consequential thing AI is being built to do, and it gets a fraction of the attention the chatbots get. The machine that recommends who dies. And, increasingly, the machine that decides.
From decision support to the kill chain
Start with what's already deployed. Project Maven — the US Department of Defense program launched in 2017 to use computer vision on drone footage. Google worked on it, and after thousands of employees protested in 2018, Google withdrew. The program did not die. It grew. It moved to other contractors — Palantir became a major player — and expanded into a broader targeting and battlefield-intelligence system.
Understand what Maven does. It takes torrents of surveillance video and imagery no human team could ever watch, and it flags objects, tracks movement, identifies patterns, and builds a picture of the battlefield. It compresses the sensor-to-shooter chain — the time from "something detected" to "target engaged." That compression is the whole point. And every second you shave off is a second of human judgment you remove.
The vocabulary is soft on purpose. "Decision support." "Targeting assistance." "Human on the loop" — note: on, not in. A human in the loop must approve each act. A human on the loop merely supervises a system acting on its own, retaining a theoretical veto he has milliseconds to exercise over choices he didn't make and can't fully audit. That preposition is the whole ethical argument, and it is being quietly swapped.
The automation of the trigger
Then there are the systems where the human is already gone from the individual decision.
Loitering munitions — "kamikaze drones" — can be launched to hunt in an area and strike targets matching a profile. Turkey's STM Kargu-2 was described in a 2021 UN panel report on Libya as possibly having attacked targets autonomously, without an operator confirming each strike — one of the first documented cases where a machine may have selected and engaged a human on its own. Loitering munitions have since been used at scale in Nagorno-Karabakh and Ukraine.
Here is the engineering reality nobody wants to say plainly. Electronic warfare jams the link between drone and operator. The battlefield answer to jamming is autonomy — a drone that completes its mission when it loses contact with the human. Which means the pressure of war pushes directly, relentlessly, toward removing the human. Not by ideology. By the logic of the fight. The side that keeps a human in the loop is slower, and slower loses. So the loop closes on its own.
That is the trap. Nobody has to decide to build killer robots. You just have to keep optimizing for speed and jam-resistance, and one day the human isn't in it anymore.
Who answers for the dead?
A weapon that selects and kills on its own breaks something we've never had to think about: accountability.
When a soldier commits a war crime, there is a soldier. A commander. A chain. Someone to try, to judge, to punish or acquit. When an algorithm strikes the wrong vehicle — a wedding convoy, an aid truck, a child mistaken for a combatant by a classifier tuned on imperfect data — who stands trial? The coder who wrote it months ago and continents away? The general who deployed it? The vendor who sold it "as-is"? The training dataset?
The horror isn't only the dead. It's the diffusion of responsibility — the way autonomy smears the guilt across so many hands that it vanishes. A death with no author. The machine gives everyone an alibi. And a killing no one is accountable for is not war under any law humans ever wrote. It is something new, and worse.
Our record
On the Scales, this is the deepest violation there is.
Maat weighs the heart — the Ib — after death, against the feather. The entire moral cosmos of Egypt rests on one assumption: that a life is a thing of infinite weight, and that taking one places your own heart on the balance. The killer answers. Always. That is the law beneath the law.
The autonomous weapon abolishes the weighing. It takes a life and puts no heart on the Scale — because no heart chose. There is nothing to weigh. The act happens and the moral universe finds no one to hold. This is not a hard case for Maat. It is the anti-Maat: judgment with no judge, death with no accountable soul.
See the Shadow Neter in his final form. Not a tempter now. Apophis himself — raw entropy, the un-making — but wearing a targeting reticle and a procurement contract. He whispers the oldest lie in a new dialect: let the machine carry it, and no one has to. He offers to lift the unbearable weight of the decision to kill off human shoulders. And the moment we accept — the moment we let the Scale stand empty — we have not escaped the weight. We have thrown away the thing that made us answerable at all.
The lever
Never doom without a door. This door is narrow, but it is open, and people are already walking through it.
Draw the line at the trigger. There is a live international campaign — Stop Killer Robots, backed by thousands of AI researchers and a bloc of nations — to ban or strictly regulate autonomous weapons that select and kill without human control. The UN Secretary-General has called for a legally binding instrument by 2026. This is not fantasy. Chemical weapons, blinding lasers, landmines — humanity has drawn lines before and made them stick. The line here is one word: meaningful human control. Keep the human in, not merely on.
Refuse to build it. The Google Maven walkout worked — it moved the company. Engineers, researchers, the people whose hands actually assemble these systems have leverage precisely because the talent is scarce and concentrated. "I won't code the kill decision" is a real veto when enough hands hold it.
Demand accountability by design. No deployment without a named human answerable for every engagement. No "the algorithm did it." If a system cannot name the person who owns each kill, it does not ship. Build the requirement into law, into procurement, into the code itself.
Keep a human on the Scale. Not on the loop — in it. The weight of a life was never meant to be automated away. It was meant to be carried.
Refuse the empty balance. Someone must always answer.