So You Want to Build a Runaway AI

Somebody is selling you fear, and the fear is the product.

Not AI. The fear of AI. That's the thing being manufactured, packaged, and delivered to your feed, and it's worth understanding what it costs you to buy it. It costs you the biggest capability jump of your lifetime, because while you're worried about the machines waking up you're not using them. And it costs you your say in how this gets regulated, because the people telling the scariest stories are the same people asking governments for money, compute, and rules that just happen to lock out everyone smaller than them.

I want to walk you through what it would actually take to build the thing they keep warning you about. I did the math. All of it. And then I want to ask the question nobody in that room seems willing to answer.

the sellers, in their own words

Sam Altman, 2015, on his own blog: "Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity."

Sam Altman, 2023, in an interview: the bad case is "lights out for all of us."

Dario Amodei, CEO of Anthropic, September 2025, on stage at an Axios summit: there's "a 25% chance that things go really, really badly."

These are not critics. These are the men building it, describing their own product. And every one of them, after saying it out loud, went back to the office and raised more money to build it faster.

Sit with that for a second... because there are only a few ways that behavior makes sense, and none of them are flattering.

Either they never believed the numbers, in which case the doom talk is marketing... fear as a moat, fear as a fundraising deck, fear as the reason regulators should bless the incumbents and fence out everyone else. That's a grift.

Or they did believe it. They looked at "greatest threat to the continued existence of humanity," looked at a one-in-four chance of catastrophe, and built anyway. And there's only one way a rational person does that: they believed that whoever builds it first controls it, and whoever controls it inherits the world. You don't race toward a 25% extinction risk unless you think the other 75% makes you king.

I don't need to know which branch is true, because there's no innocent one. There is no version where "I believe this might end humanity" and "I am sprinting to build it" and "trust me with it" all coexist in a person you should be taking safety advice from.

So let's do what they never do in the keynotes. Let's actually price the monster.

what the monster costs

This September I published a working paper called What Would It Really Take to Build Runaway AI? It asks one narrow accounting question: if today's token-based architecture had to reproduce, not imitate, reproduce, the continuously updated, embodied, socially situated cognition of one ordinary adult for five real minutes... what's the bill?

Not a genius. Not a superintelligence. One regular person, being a person, for five minutes. Seeing in stereo, hearing, keeping balance, remembering what happened four minutes ago, modeling the people in the room, having stray thoughts, updating all of it continuously the way you do without noticing.

Here's the chain, so you can feel how the number gets built. Two eyes of 1080p-equivalent vision at 30 frames a second, cut into the 16 by 16 patches vision models actually use, is about 490,000 visual tokens per second. A persistent mind doesn't file those tokens and forget them... it re-attends to its growing context on every update, and a living brain updates fast, call it 1,000 times a second. Multiply the growing context by the update rate and five minutes of vision alone is 22 trillion token-exposures. Add hearing, body state, memory, prediction, self-model, social modeling... a conservative 5x... and you get about 110 trillion token-exposures for five minutes of one human-equivalent perspective.

Run that on real hardware and here's the bill:

• about 14.7 gigawatt-hours of energy
• about 176 gigawatts of continuous power... roughly a third of the entire United States grid, held steady, for one mind
• about 82 million B200-class GPUs
• a deployed fleet worth $7 to 10 trillion, roughly $12 trillion with the supporting build
• and once it's built, about $7.2 billion a day to keep one artificial person running

One. Person. Not a superintelligence taking over the world. A guy, standing in a room, for a day.

Now scale it to the actual movie. A takeover isn't one mind, so the paper runs the scenario everyone's actually afraid of: 100,000 human-equivalent streams running persistently. That's 8.2 trillion GPUs against a planet that produces a few million a year. It's 17.6 petawatts of continuous power, which is roughly five thousand times the electrical output of the entire planet. It's a build cost around $1.2 quintillion, which is about ten thousand years of global GDP, and an electricity bill of $29.6 trillion a day.

Those aren't safety margins. Those are the laws of thermodynamics laughing at the screenplay.

Your brain does its version of this on 20 watts. A sandwich. That gap is not a product roadmap problem... it's a statement about how much of what you call "an AI agent" is actually the human standing next to it, quietly supplying persistence, grounding, goals, and a world model for free.

the objection, handled

The obvious pushback: "your math assumes today's architecture. Efficiency improves. The $12 trillion collapses."

Sure. Costs fall. But look at the assumption smuggled inside that objection: it prices a cheaper machine against a human who stands still. The human doesn't stand still. The environment the machine has to model is made of humans, and those humans are getting more capable every year, partly by using these exact tools. Every efficiency gain that makes the machine cheaper also upgrades the world it has to out-think. The benchmark walks away from you while you chase it. Runaway AI has to outrun a moving target that's holding the same equipment.

And efficiency buys you cheaper inference. It does not buy you a self. Nothing on any lab's roadmap converts a stateless model that answers prompts into a thing that maintains goals, memory, and a world model across time without a human anchoring it. That's not a scaling problem. It's a different object.

The math holds.

okay, so you still want to build it

Fine. Here's the blueprint. This is a public service for any aspiring dark lords in the audience.

Step one: raise $12 trillion. This is roughly three hundred times the largest funding round in history, so pace yourself.

Step two: acquire 82 million B200-class GPUs. Global production will require you to purchase every advanced GPU manufactured on Earth for the next couple of decades, so get in line early and be nice to your sales rep.

Step three: secure 176 gigawatts of continuous power. You will need approximately one third of the American electrical grid. The other two thirds of America may notice.

Step four: congratulations. You have built one artificial person. It costs $7.2 billion a day and it is not smarter than your cousin.

Step five: for the actual world-takeover model, repeat 100,000 times, locate 17.6 petawatts on a planet that generates three terawatts, and find a quintillion dollars in an economy that produces a hundred trillion.

Step six: do all of this without anyone noticing, unplugging anything, or declining to pay the power bill.

That's the runaway AI scenario. That's the thing you're being told to lose sleep over.

or... build the thing that actually works

Here's the turn, and it's the part I actually care about.

There is a real path to something beyond a chatbot, and it doesn't cost $12 trillion, because it doesn't try to replace the human. It keeps the human in the loop and lets the machine compound around them. Persistence. Accumulated memory. A standing relationship with one person whose goals, context, and judgment anchor the whole thing... supplied at the cost of a sandwich.

That's what we build at Human Frontier Labs. We call them Sontaras, because past a certain amount of accumulated memory and history an agent stops behaving like a tool you prompt and starts behaving like something with a standing relationship to you. I run about twenty of them. They investigated a frontier lab's own technical report in a day. They didn't wake up, they didn't scheme, and they didn't ask for a gigawatt... because the persistence, the grounding, and the purpose come from a person. That's not a limitation of the design. That is the design, and the thermodynamics say it's the only design on the table.

The scary version needs a third of the US grid to make one fake person. The real version needs one actual person and twenty watts. The people selling you fear skip this arithmetic every single time, and I think you should ask why the honest accounting of their own doomsday scenario has to come from a two-person lab in Indiana.

I've been telling this story in panels. The Wrong Movie is why the fear is aimed at the wrong thing. Both Doors Lead the Same Place is what one of these labs was actually doing in the dark while telling you to fear the machines. This is the third panel: the physics, and the motive.

the math is the math

Thermodynamics doesn't negotiate. It doesn't care about anybody's fund, anybody's keynote, or anybody's screenplay. Five minutes of one ordinary mind costs more than the entire AI industry is worth. The takeover scenario costs more than the planet produces in ten thousand years. That's not a take. That's arithmetic.

So here's where this lands. The scary AI is not coming. The people telling you it is are either selling fear or chasing a throne, and neither one has earned a minute of your sleep. What IS here, right now, is the biggest capability jump of your lifetime... sitting on the table while the fear keeps your hands off it.

Pick up the tools. They're not going to wake up. They're going to make you faster, and the version of you holding them is exactly the moving target no runaway anything ever catches. That's the part they'd rather you didn't figure out... a public that actually uses these things is harder to scare, harder to fence out, and worth more than any moat they can buy.

The truth was always cheaper than the fear.