When the leaders say slow down
Suman Regmi ·Co-founder · 3-min read ·
Two pieces landed this month that are worth reading against each other.
Anthropic published When AI builds itself: AI is starting to write the next AI, the curve is steepening, and the world should have the option to slow or pause frontier development before recursive self-improvement gets away from us. Careful, transparent, genuinely uneasy. More than 80% of their own code is now written by Claude.
Around the same time, Satya Nadella argued something that sounds unrelated: every firm will need human capital (its people’s judgment, relationships, pattern recognition) and token capital (the AI capability it builds and owns). The win isn’t picking the best model; it’s building a learning loop on top of models — your workflows and judgment turned into a system that compounds, where you can swap out a “generalist” model without losing the “company veteran” you’ve encoded. His line: you can offload a task, even a job, but you can never offload your learning. And his warning: don’t let a few models eat everything they see and commoditize every industry from underneath it.
They’re not two arguments — they’re two sides of one
On the surface, one is about safety and one is about economics. Underneath, both are about the same thing: who ends up holding the reins.
And there they pull in opposite directions. Anthropic’s fix — a few frontier labs coordinating, with verification, to govern the pace — centralizes. Whatever the sincerity, it seats a handful of approved actors as gatekeepers of the frontier. Nadella’s whole thesis decentralizes — push capability and value out to every company so no one is commoditized. Read that way, Nadella is the quiet rebuttal to Anthropic: a coordinated pause among the few is one of the ways the few come to eat everything.
Be skeptical of both, not just one
It would be easy to nod along to the danger warning. But the players who ran the fastest — using every advantage to keep smaller players from catching up — are the ones now turning around to say careful, danger ahead. Maybe that’s sincere. Maybe it’s the shrinking instinct wearing a safety jacket. From outside you can’t tell, because protect humanity and protect the lead point the same way.
The honest move is to aim that same suspicion at Nadella. “Every company should build its own learning loop” is a lovely thesis for the company that sells the platform the loop runs on. Both men are routing the value toward their own book. Neither is a neutral narrator.
Where that leaves us
So the takeaway isn’t “two views, neither touches us.” There’s a real fight underneath — centralize at the frontier versus distribute across the ecosystem — and where you choose to build is a vote.
We’ve made ours. We don’t try to push the model; we try to use it well, close to real work, in a profession we know from the inside. What we care about building is the distributed answer — a learning loop that lets an industry keep its own judgment instead of handing it to a few models to absorb. The louder this argument gets, the more that feels like the question worth working on. It’s the thesis we started with.
The frontier can sort out its own pace. We’ve got a profession to make better.
More from Notes