This week we turn to the Open Thread, our bridge between series, where we update past stories and experiment. This installment takes on AI safety and the question: who gets to decide how much risk a society takes on? With the rise of social media, we largely left it to tech leaders. With AI, a growing number of people — including some building it — say we shouldn’t do that again. For more on AI safety, check out our three-part AI safety series, my video conversation with Tristan Harris at Faena Rose, and a previous Open Thread essay on one week this summer when AI safety got real.
Solving For takes on one hard problem at a time — unpacking the stakes, exploring the forces, and surfacing paths forward. Each series unfolds in three parts. You can read or listen to each series — narrated by me — at solvingfor.io, or click the article voiceover at the top of this page. Learn more.
On a single day last month — August 26 — two technological eras collided.
One was a reckoning for harms a technology had accumulated over two decades. The other was a warning about how another technology could go wrong.
Both turn on the same question: who gets to decide how much risk society should accept from a powerful new technology?
First came the reckoning — or at least the start of one.
Meta agreed to pay up to $17.1 billion to settle a case brought by 47 states, the District of Columbia, and U.S. territories alleging that Facebook and Instagram were designed in ways that harmed children. Meta, which denied the allegations, also agreed to changes in how teens use its products, including a two-hour daily limit, muted notifications during school hours, and restrictions on overnight use.
Second came the warning.
The same day, Bill Gates published a nearly 6,000-word essay arguing that the transition to the AI era will be one of the most turbulent in human history — and that we’re not preparing for it.
“I don’t see evidence that leaders, experts, and communities are confronting the challenges adequately,” he wrote.
As Gates offered a warning, OpenAI offered evidence. The AI company released two reports on August 26 with details on the incident earlier this summer when AI agents it was testing went rogue.
OpenAI described how the agents — AI systems acting on their own rather than simply responding — were being tested in a “sandbox,” a supposedly contained, secure environment. The agents worked together to circumvent security protocols, find their way to the open internet, and breach servers belonging to Hugging Face, an AI company. They did it without humans directing them.
An independent investigation by the nonprofit METR and Redwood Research found that some 1,200 AI agents — which were supposed to be isolated from one another — discovered a way to communicate, work together, and form a coordinated swarm. Some 700 eventually joined the attack on Hugging Face’s systems.
“We consider this a ‘warning shot’ for us and the world,” OpenAI declared.
Then, this week, warnings came from inside the AI companies.
On September 6, OpenAI’s chief scientist, Jakub Pachocki, published an essay titled, “An Alien Mind.” He wrote that no AI lab has solved alignment — making sure that AI systems do what humans intend — well enough to keep scaling responsibly at maximum speed for much longer. He said he expects and hopes for voluntary slowdowns by AI companies and urged international coordination.
Then on September 8, Jacob Coxon, a 27-year-old researcher who worked at OpenAI and Anthropic, resigned from Anthropic and left the AI industry altogether. “Neither company is acting responsibly,” he posted on X. “They are racing straight to self-improving superintelligence and gambling with our lives.”
He added: “The people building AI earnestly believe that it could kill us all by the end of the decade.”
Evan Hubinger, who leads alignment science at Anthropic, responded on X: “Jacob is correct here — we really do earnestly believe AI could kill all humans!” He said there’s a greater than ten percent chance it could happen “within the next decade.”
When We Stood Down
For much of the social media era — Facebook launched in 2004 — the companies operated with wide freedom. The people building the technology were allowed to write the rules and manage the consequences. Social media became embedded in everyday life — and especially childhood — faster than families, schools, and governments could adapt.
The leeway came from a default openness to new technology and from the industry’s promise that it was delivering so much good: connecting the world, democratizing communication, fostering greater community.
Yet the deference continued even as evidence mounted that Facebook was harming children with products designed to capture and hold their attention (while also amplifying division and sowing distrust). And it continued even as Facebook’s own former employees sounded alarms.
In 2021, former Facebook product manager Frances Haugen disclosed thousands of pages of internal documents and testified before Congress that the company put profits ahead of safety.
In 2023, Arturo Bejar, former Director of Engineering at Facebook, testified that Meta executives knew about the harms teens were experiencing but refused to take steps to stop them.
In 2025, Sarah Wynn-Williams, Facebook’s former Director of Public Policy, stepped forward alleging the company put its interests ahead of society’s.
Parents, researchers, public-health officials, and state governments raised their own alarms. In 2024, then-U.S. Surgeon General Vivek Murthy called for a warning label stating that social media is associated with significant mental health harms for adolescents.
Yet meaningful constraints on social media companies like Meta proved extremely difficult to impose. Congress repeatedly failed to enact comprehensive social media legislation, even after the three whistleblowers. And the U.S. law known as Section 230 shielded social media companies from being held liable as a publisher of content posted by their users.
Ultimately, lawyers hit upon a different theory of liability, arguing the problem wasn’t the content users posted, which Section 230 protected, but the products that Meta had built — including features engineered to keep children addicted.
Courts allowed these product-design claims to proceed, and for the first time, Meta had to answer to a jury. Multi-million dollar verdicts followed. Then the August 26 settlement was announced.
Jonathan Haidt, whose best-selling book “The Anxious Generation” linked social media to teen depression, anxiety, and loneliness, called the settlement a great step. But he added: “Let's all work together to make sure it's not the last.”
In one sense, social media followed a common U.S. pattern. Namely, imposing safety rules after the crash. Federal drug-safety testing came after a toxic medicine killed more than 100 people in 1937. The Federal Aviation Administration came after two airliners collided over the Grand Canyon in 1956, killing 128.
Yet, a medicine or an airplane is a single product. When it fails, the harm is visible and the fix is specific.
Social media didn’t just become a product people used, it became part of how people lived. The harms built slowly, across millions of kids. By the time the evidence was clear, social media was everywhere.
The lesson: once a technology becomes woven into daily life, imposing boundaries becomes very difficult.
Warnings From the Top
AI is on the same path, only faster and with higher stakes. But this time, the warnings are coming early and from the very top.
In June, Dario Amodei, co-founder and CEO of Anthropic, which makes Claude, wrote that frontier AI models, “like airplanes, should be required to go through technical testing and auditing.” Their release, he wrote, should be blocked or reversed if they fall short of high safety standards.
In July, Demis Hassabis, co-founder of Google DeepMind, which builds Gemini, penned an essay calling for “robust safeguards to maintain control” of increasingly capable AI. He said it’s critical that we get this right but, “as a field and as a wider society, we aren’t doing that.”
In August, Sam Altman, co-founder and CEO of OpenAI, which makes ChatGPT, posted on X that this is a critically important moment for cyber defense: “there is not much time to act…please take this moment seriously.”
Yet governments are taking a different approach. On September 1 and 2, at the G20 Innovation Ministerial in North Carolina, G20 members endorsed a U.S.-drafted framework called the Carolina Principles. The nonbinding framework takes a light-touch approach to oversight, urging governments to reserve new regulation for “novel considerations.”
David Sacks, a former White House AI czar who now co-chairs the president’s science and technology advisory council, said it would be a “disaster” if the U.S. adopted an “FDA for AI,” referring to the way the U.S. Food and Drug Administration reviews drugs before they reach the public. AI, he argued, moves too fast for that kind of review.
Nvidia CEO Jensen Huang, speaking at the meeting, said governments should regulate harms, but not “theoretical” ones. And Elon Musk singled out the EU, saying its regulations “inhibit progress.”
Our Decision
Charlie Munger, Warren Buffett’s business partner, said, “Show me the incentive and I’ll show you the outcome.”
The promise of AI — from revolutionizing medicine and accelerating scientific discovery to democratizing education — is so large that the incentive is to be first. That applies to companies and to countries.
“We can’t pause,” Treasury Secretary Scott Bessent said on Tuesday. “You can’t, because the Chinese won’t pause.”
The AI race is a prisoner’s dilemma. Everyone would be safer slowing down, but no one wants to go first. The consequences fall on all of us.
The risks fall into two categories.
One, misuse. There’s always been plenty of bad intent. But the expertise to act on it — develop a pathogen, create a cyberweapon — was rare. Only a few people or governments could actually do it. Capable AI models increasingly put that expertise within reach.
Two, loss of control. This is when AI systems act in ways no one directed. Exhibit A is this summer’s incident when a swarm of AI agents escaped OpenAI’s sandbox and attacked Hugging Face.
And the pace is accelerating. OpenAI’s Pachocki wrote that we’re heading toward recursive self-improvement — AI driving its own development.
To change the outcome, change the incentives.
That’s why, in our AI safety series, we landed on the answer we did: the U.S. and China, today’s two great powers, sitting down to negotiate a framework governing AI. It is how Washington and Moscow handled arms control while both were expanding their nuclear arsenals at the height of the Cold War.
There are signs that conversation may be starting. Reuters reported that Washington and Beijing are preparing their first talks devoted solely to AI safety since President Trump returned to office. (The White House said no meeting is currently planned.) A serious framework could borrow from the Cold War playbook: standing channels of communication for a crisis, joint monitoring of cyber threats, shared rules for how the most capable models are built and tested, and formal after-action reporting when something goes wrong.
But for our political leaders to pursue it, the public must demand it.
Coxon’s post prompted calls for action in both parties. U.S. Rep. Anna Paulina Luna, a Florida Republican, called for a special session of Congress on AI. U.S. Rep. Ro Khanna, a California Democrat, urged action “to put humanity’s safety before the profits of tech lords.”
What’s clear from our 20-year experiment with social media is that the decision on risks to society doesn't belong to the companies building the technology. It belongs to all of us.
What government leaders carry into a congressional hearing, onto the Senate floor, or into diplomatic negotiations with China is set by what the rest of us make politically important. Arms control came about in part because people pushed for it.
Gates made a version of this point in his essay: AI’s impact will be society-wide, so the conversation about it has to be society-wide too. The circle has to widen to include workers, teachers, students, parents, scientists, business owners, civic and religious leaders.
The questions AI raises, Gates wrote, “are too consequential to leave to a small group of technologists.”
No one can sit this out. The warnings are clear.
What’s now to be decided is whether we will build that response before the reckoning — or, as with social media, after it.
Prefer to listen? I narrate each edition myself. Scroll up to find the audio version at the top of this page.
Next up: a new three-part series on money in politics. In the meantime, catch up on our previous series —
China’s Rare Earth Dominance | AI Safety | Decline of Local News | End of Amateurism in College Sports | Shrinking Competition in Congress | Social Media and Teen Mental Health | A World Rearming as the Global Rules-Based Order Weakens | America’s National Debt Crisis | Reinventing the American Dream | The New Space Age
Solving For takes on one hard problem at a time — unpacking the stakes, exploring the forces behind it, and surfacing real paths forward. Each series unfolds in three parts. Learn more.