On September 8, a 27-year-old British researcher sat on a park bench in San Francisco and quit his job. Jacob Coxon had spent three years training AI models at OpenAI and Anthropic, and his resignation post claimed that neither company was acting responsibly and that both were racing toward self-improving superintelligence. Within 36 hours, roughly 153 million people had seen it.
Fear of new machines is nothing new. English weavers smashed looms in the 1810s, and teachers panicked when calculators arrived in classrooms in the 1970s. Each time, work changed instead of disappearing. What makes this moment strange is who is sounding the alarm. Some of the loudest warnings are coming from the people building the technology.
The builders ask for brakes
Coxon was not alone for long. Four days later, Anthropic’s CEO Dario Amodei published a long essay arguing that the risks justify a slowdown. On September 14, OpenAI’s Sam Altman said no amount of competitive pressure should excuse recklessness or let AI’s abilities race ahead of our ability to monitor them. By September 23 he was at the UN Security Council, warning that humanity could lose control of AI’s future and that the biggest decisions should not be left to San Francisco labs alone.
Both men came with proposals. Altman asked for international standards to measure what AI can do and whether safeguards are enough. Amodei suggested narrow global bans, such as one on AI-made bioweapons, along with ways for countries to verify each other’s promises and a shared system for reporting AI incidents.
The trap: Who slows down first?
Good ideas are one thing, and getting rivals to adopt them is another. President Trump has called fears of AI destroying humanity a hoax and argued that aggressive regulation would only help China. Vice President Vance said the administration wants to manage the risks without weakening America’s position in the race. Meanwhile, about 20 countries led by Finland and Norway backed a global oversight declaration. The United States and China, the two front-runners, stayed out, and experts see little chance that either will accept major limits soon.
This is a classic arms-race trap. If one country slows down and its rival does not, the slow one falls behind, so both keep sprinting even if both would rather stop. Countries have escaped this trap before, most famously with nuclear weapons, and it was not trust that got them out. It was inspections and verification. That is why Amodei’s proposal for countries to check one another matters more than any dramatic warning. A rule nobody can verify is only a promise.
Should we trust the messengers?
Fair enough, but should we trust the people delivering the warnings? Suspicion runs in both directions. Critics of Coxon, including former White House AI adviser David Sacks, note that it was his first ever post, that safety groups boosted it within minutes, and that he offered no hard data. A later report said a communications firm linked to AI-safety figures had helped arrange his media appearances, though it did not show that his claims were false. On the other side, skeptics of Altman argue that a company that built the technology may be asking for rules mainly to protect its lead.
Endorsements from inside the industry cut both ways too. Anthropic’s Evan Hubinger publicly backed Coxon and put the odds of AI killing all humans within a decade above 10 percent. That is one researcher’s estimate, not a scientific consensus.
Motives are worth checking, but they do not settle the question. Who says something has no bearing on whether it is true. The better test is whether the claim rests on evidence that others can examine.
What the evidence shows today
So let us look at some evidence. The extinction debate is about the future, but the research already available paints a steadier picture. AI is powerful, uneven, and remarkably easy to over-trust.
Take programmers. In one randomised trial, experienced developers believed AI made them 20% faster, when in fact they were 19% slower. It was a small study, but the gap between feeling and fact is the point. Consultants show a similar pattern. In a study of 758 of them, AI made people faster and better on tasks within its ability. On a task just beyond it, AI users were 19 percentage points less likely to reach the right answer, because the wrong answers looked so convincing.
Classrooms tell the same story with a twist. In Turkey, students given an unrestricted ChatGPT-style tool scored 17% worse on the exam than students who never had AI, and they did not realise they had learned less. In Nigeria, a World Bank trial used AI with teacher supervision and prompts designed to encourage reasoning, and the results were striking: roughly 1.5 years of typical learning in six weeks. The tool was similar, but the outcomes were opposite, and the difference was how it was used.
Jobs may be where the effects show up first, and they are landing at the bottom rung. Stanford economists found that employment of 22- to 25-year-olds in AI-exposed jobs is now 19% below where it would have been, while experienced workers show no comparable gap. The same researchers see no widespread, economy-wide job loss, and other economists argue that a general hiring slowdown explains part of the pattern. Even so, the risk may not be that AI takes your job tomorrow. It may be that fewer beginners get the first one.
Across all of these studies, the lesson is that “human oversight” is not only a job for governments and labs. Every user needs to practise it: check the answer, and keep learning without the machine.
What Nepal should take from this
Nobody will write AI rules with Nepal in mind, and the stakes here are high. Unemployment among 15- to 24-year-olds stood at 22.7 per cent in 2022-23, and around a million young Nepalese leave the country each year. AI has to become a tool for them, not another closed door.
Three steps follow from the evidence. First, schools should teach students to check AI’s work, not just use it. Second, classrooms should favour guided, teacher-led AI, as in the Nigeria trial, over a free-for-all. Third, Nepal should push for proper Nepali-language support, since researchers testing Nepali have found it takes more tokens, the units of text AI tools charge for, than the same content in English. A small country lacks power, but it can join coalitions, as those 20 nations did.
The bottom line
AI may be genuinely dangerous, and the people warning us may have motives of their own. Both can be true. The race with China means no country wants to break alone, so the real question is whether countries can build ways to check one another. Until they do, the sensible rule for everyone is a simple one: use AI, but never hand over your judgment.
