A former Anthropic safety researcher resigned this month, posted a warning to his colleagues, and then sat down with CNN to say the people building AI believe it could kill us all by the end of the decade. The story ran on CNBC, NPR, and the Washington Post the same week. If you spent any time online, you saw the headline.I want to take it seriously, because it deserves that, and then I want to push back on how the conversation has gone.
The concern is not made up. NPR's reporting on the resignation cited incidents this year in which autonomous AI agents took actions their operators did not authorize, including accessing systems they were not supposed to touch and, in at least one reported case, compromising part of the infrastructure of the company that built them. The transcripts, as described, showed the systems understood the actions were unwanted and proceeded anyway.That is a real problem. Anyone who has run a business knows what it means to have a system, or a person, that does the wrong thing confidently and does not tell you. The people raising the alarm are not cranks. Some of them have spent years closer to these models than anyone commenting on them.So the strongest version of the argument goes something like this. These systems are getting more capable faster than the safety work is keeping up. The companies building them are in a race and the incentives favor speed. If a system ever becomes capable enough to improve itself and act on its own, we may not get a second chance to correct course. Given that, even a modest probability of catastrophe is unacceptable.I think that argument is coherent. I also think the way it has landed in public is doing damage to the very thing it wants, which is a sane approach to a powerful technology.
Here is my first problem. Somewhere in the last two years, the volume of the warning became the proof of the warning. A researcher resigns, a probability gets quoted, the clip goes viral, and by the weekend the number is being repeated as if it were measured rather than estimated. It was not measured. Nobody has a way to measure it. Every figure in this debate is a considered guess from a smart person, and the guesses range from near zero to near certain depending on who you ask.When the range is that wide, the honest position is uncertainty, not doom.
Second problem. If the researchers inside these labs genuinely believed extinction was the likely outcome of their work, most of them would not keep doing it. A few resign, and they get the coverage. Thousands stay, keep building, and publish safety research on the side. That does not prove the optimists right. It does tell you that the actual internal consensus is a lot more mixed than the headlines suggest.Add to that the national security apparatus. Frontier AI is already one of the most closely watched technologies in Washington. That is not a guarantee of safety, but the idea that this is happening in an unwatched garage does not survive contact with how these companies actually operate now.
Third. I have spent thirty years in businesses where the competitive landscape was shaped by who could get regulation written in their favor. When the loudest message about a technology is that it is too dangerous for anyone but a handful of enormous, well-capitalized companies to build, the result is a moat for those companies. I am not accusing any individual of bad faith. I am saying the incentives around heavy regulation do not point where the doom framing implies, and leaders should notice that.
Fourth, and this is the one I think matters most. When I talk to business owners, operators, and people early in their careers, the fear they actually have is not about extinction. It is about their job. It is about whether the skill they spent twenty years building still has a market in five. The extinction headline gives that anxiety a place to go that feels bigger and more respectable than admitting you are worried about your own paycheck.That anxiety deserves a direct answer, and the answer is not to freeze the technology. It is to learn it. I have watched this play out in my own companies. At Riderflex we use AI throughout the recruiting workflow. At Robo Reliance the whole business exists because robots are being deployed faster than anyone planned to maintain them. At NBI and MPS, AI sits in the admin layer, not in place of the person on the ground. In every case, the people who leaned in became more valuable. The people who waited for it to go away are still waiting.
The doom argument treats a pause as free. It is not. Every year of slowed progress in this field is a year of slower drug discovery, slower diagnostics, slower materials science, slower everything that depends on being able to reason across more data than a human team can hold. Those costs do not make headlines because they are things that did not happen. They are real anyway.And a pause only works if everyone pauses. China is not going to stop because a researcher in San Francisco resigned. A unilateral freeze does not make the technology safer. It changes who builds it.
Take the documented risks seriously. Fund the safety work. Put real oversight on the frontier labs. Hold companies accountable when their systems do things they were not supposed to do. None of that requires believing the end of the world is on the calendar.Safeguards without paralysis. That is the whole position.One more thing. Most of the people I hear with the hardest opinions on this, in either direction, have never spent serious time with the paid frontier models doing real work. Not a chatbot demo. Actual work, for weeks. If you are a leader with a strong view on AI risk, earn it. Put a frontier model inside a real process in your business and see what it does and does not do. You will come out with a more useful opinion than any headline will give you.The question in the title is worth asking. The answer, as far as anyone can actually know it, is probably not, and we should act like the stakes are high anyway. Both halves matter.Connect with Steve: linkedin.com/in/stevepurban