In brief
- The AI industry warns that the technology its developed poses a threat to humanity, so-called “existential risks”.
- According to various independent researchers and scientists, these warnings appear to be based on little concrete evidence, and make the technology appear more powerful than it actually is.
- By focussing on these hypothetical risks and vague predictions, we are in danger of overlooking the actual damage that AI is already causing to the environment and climate.
- These various doomsday predictions are rooted in a philosophy that unjustifiably considers the development of increasingly powerful AI unavoidable.
Dit is een vertaling van Doemdenkbeelden domineren niet zomaar AI-debat. Bij onderwerpen met een bredere, internationale relevantie voorziet Apache een Engelstalige versie. Vertaling door historicus en onderzoeker Thomas Donald Jacobs.
At the start of September, our concerns about the deadly climate crisis were suddenly drowned out by a somewhat different chorus of voices. The tipping point came with the resignation letter of Jacob Coxon. In an exclusive interview with The Wall Street Journal, the researcher gave an extensive explanation for his departure from the AI businessfirm, Anthropic. The reason for his resignation: the fear that the race between AI companies would result in so-called artificial intelligence escaping human control.
“The people building AI earnestly believe that it could kill us all by the end of the decade”, wrote Jacob Coxon on X concerning his resignation from Anthropic
On X, the social networking platform controlled by Elon Musk, one of the largest proponents of and investors in AI, Coxon hit the panic button. “The people building AI earnestly believe that it could kill us all by the end of the decade”, he wrote. AI companies are “racing straight to self-improving superintelligence and gambling with our lives”.
As if that weren’t concerning enough, Evan Hubinger poured oil on the fire. “Jacob is correct here – we really do earnestly believe AI could kill all humans!” Hubinger added that he estimated the odds of such a disaster occurring during the next decade were more than 10%. Hubinger is responsible for the so-called ‘alignment’ of AI at Anthropic, which is tasked with ensuring that the software ‘acts’ in accordance with human values.
Dario Amodei, co-founder and CEO of Anthropic, saw his chance, and a few days after Coxon’s public resignation argued to “slow the pace” of the development of new AI models. His first reason for doing so is that AI is now developing AI and growing increasingly better at doing so, a dynamic called ‘recursive self-improvement’. Secondly, Amodei cites the incident in which AI agents from competitor OpenAI purportedly independently hacked the systems of the AI platform Hugging Face. According to him, this is a sign that the internet could be completely taken over by a swarm of AI agents.
We do not have enough space to cover the complete chronology of what happened. However, we can examine the result of this panic filled autumn. At the end of September, Amodei - together with other AI and tech giants including Elon Musk (Tesla, SpaceX), Mark Zuckerberg (Meta), Sundar Pichai (Google), Jeff Bezos (Amazon), Satya Nadella (Microsoft) and Jensen Huang (Nvidia) – were welcomed to a lunch date with Donald Trump regarding that the American President is currently calling “superintelligence”. (Trump wants everyone to do so going forward, beginning with the American government.)
The grand banquet – in which the seating arrangements were read like tarot cards – resulted in an agreement of little more than 300 words. Apparently, the gathering did not require a longer text to prevent the predicted end of the world. Or to outline the self-regulation of the sector in more than the vaguest of terms.
Can we now sleep soundly thanks to this non-binding agreement between Big Tech and Trump? Or should we not take the underlying doomsday scenarios so seriously? Apache sought advice from a number of AI researchers.
Sensationalism is also hype
“This is not a marketing stunt”, Jacob Coxon emphasized. Virginia Dignum completely contradicts this. The Professor of Responsible Artificial Intelligence at the University of Umeå in Sweden sees the recent hysteria surrounding the coming AI apocalypse primarily as an effort to distract attention from – among other things – the actual damage that AI is already causing.
As a member of the ‘high-level’ expert group, Dignum advises the European Commission on its strategy for artificial intelligence. She is also a member of the Artificial Intelligence Advisory Body that António Guterres, secretary general of the United Nations, created in 2023. Over-questioned by the press, she responded to our queries with a pdf. In it, she provides answers to the most common questions that she receives. (It seems our questions were not terribly original.)
According to Dignum, predictions of doom generally contain two elements. On the one hand, they present AI as “an autonomous force whose course is inevitable and beyond human choice”. On the other hand, they assert that “that the few companies building it are the only ones able to keep it safe”. The first argument ensures that the people and institutions actually making decisions concerning AI cannot be held responsible for their decisions. At the same time, the second argument shifts authority and investments towards those companies.
“Imagine a pharmaceutical company announcing a transformative product that its own executives say could kill 10 per cent of humanity within five years”, Dignum writes. “No regulator would allow it on the market, no hospital would buy it and no investor would fund it. In AI the same statement raises company valuation! ”
According to Dignum, we should therefore not see the doomsday predictions within the AI industry – the p(Doom) or risk of doom – as estimates as risk, but rather as marketing. “The figures circulating are personal beliefs from people with a commercial stake”, a “pastime of commentators, industry leaders and those with little better to do, not a scientific concept”. AI products are being presented as a threat to humanity, which ensures that the products are seen as more powerful than they actually are, argues Dignum.
Olivia Guest, who researches computational cognition, agrees that “It’s a PR stunt”. “While these warnings are spurious, nonetheless we must take notice because these are rhetorical strategies to protect companies’ bottom lines but also reputations more generally.” Apache asked a number of questions of Guest and her colleague, Professor Iris Van Rooij, in an e-mail exchange. Both are researchers for the Computational Cognitive Science Group at Radboud University in the Netherlands. Among other things, Van Rooij researches human cognition using computer and mathematical models. Both are highly critical of the persistent hype around everything being sold under the umbrella of AI.
In June of 2025, Guest and Van Rooij initiated an open letter in which they took “a principled stand against the proliferation of so-called ‘AI’ technologies in universities”. Almost 2,000 people undersigned the letter, including a number of Belgian AI scientists.
Not unavoidable, just unrealistic
While prophets of doom insist that they are warning people, Van Rooij argues that their sensational stories also continue to fuel the hype about AI. According to companies like OpenAI, we are quasi-inevitably evolving in the direction of what they call ‘artificial general intelligence’ (AGI). The development of this hypothetical system, that would be more intelligent than any other living being, or which would perform just as well as, – or even better than – humans in most economically viable activities, is the explicit goal of many of these companies.
Nearly every year, Elon Musk has predicted that his company either has developed or was on the point of developing AGI. He still has two months to make good on his last prediction. At the start of 2025, OpenAI CEO Sam Altman told Time Magazine that his company is “confident we know how to build AGI as we have traditionally understood it”. In the market prospectus that Anthropic posted on October 26, the company did not talk about AGI or superintelligence, but rather “Powerful AI” with, among other things, “cognitive capabilities matching or exceeding those of top human experts across most professional and scientific disciplines”.
While prophets of doom insist that they are warning people, Iris Van Rooij (Radboud University) argues that their sensational stories also continue to fuel the hype about AI
Like every AI CEO, Amodei seems to believe that his company is in the best position to achieve powerful AI. And that if this development does not take place in the right way, then the threat exists that AI could take over. Hence the plea for a general ‘slowing’ of development across the industry.
However, the plea to slow the pace of development conflicts with Anthropic’s own prospectus, which announced that in the coming years, the company will require “access to enormous and increasing amounts of specialized compute”. If not, “we may be unable to train models that remain competitive with those of our competitors or adequately serve our customers”. The contracts that the company has already signed for the used of the datacentres belonging to – among others – Musk’s SpaceXAI have a combined estimated value of 800 billion dollars.
“‘AGI’ has been much discussed – along with other such things - but the concept does not have a consistent definition, while many commercial interests are playing a role in the background,” states Cynthia Liem, associate professor in Responsible and Trustworthy AI at Delft University of Technology. Van Rooij points out that claims about AGI are as old as the scientific research into artificial intelligence. “We can see such claims repeating throughout history going back to the 1950s.”
Van Rooij vigorously disputes that current AI models can “truly scale in any reliable and sustainable way” as is proposed by claims of the inevitable course of AGI. Van Rooij sees the call to slow the development of AI as a way to hide that its development is reaching its limits. Moreover, she states that companies may represent themselves “as ‘holding the progress back, for the time being’”, but continue “convincing people to keep investing money”.
“One reason is that the resources require to do so grow exponentially with every small step of increase in attempted added complexity or context-sensitivity.” In a 2024 research paper, Van Rooij and Guest propose that even under theoretically ideal circumstances, with perfect datasets, and the most efficient methods for machine learning, we will have used up all of the natural resources of the planet before coming within reach of human cognition.
Lack of oversight
In his plea for slowing the pace of development, Amodei argued that AI systems are developing and improving themselves in so-called recursive self-improvement. However, as Dignum simply puts it, “recursive self-improvement is a hypothesis, not an observed mechanism”.
Professor Jefrey Lijffijt, who is head of the Artificial Intelligence and Data Analytics research group at Ghent University, sees the changes brought about by AI in information processing and other areas as a positive development. Nevertheless, he too is sceptical regarding the declarations of Amodei and others concerning self-developing and self-improving AI systems. “I think it is very impressive that they’ve automated such a large part of their own software development pipeline [building, testing, implementing technology, ed.], but that still does not make recursive self-improvement a reality. There remain an unknown number of steps before we achieve that, so many that it lies beyond the horizon of what I dare to predict.”
The hypothetical ‘threat’ of self-improving AI systems makes the warnings about AI that has escaped from “human control” more urgent. During the summer, AI businesses tripped over each other to sound the alarm about “rogue AI agents” attacking other companies.
The hypothetical ‘threat’ of self-improving AI systems makes the warnings about AI that has escaped from “human control” more urgent. During the summer, AI businesses tripped over each other to sound the alarm about “rogue AI agents” attacking other companies.
Olivia Guest calls warnings about rogue AI agents acting “behind the backs” of humans “nonsense to drum up impressive seeming results and avoid accountability”
Olivia Guest calls warnings about rogue AI agents acting “behind the backs” of humans “nonsense to drum up impressive seeming results and avoid accountability”. Dignum also sees such claims as total garbage. “Reported ‘scheming’ or ‘going rogue’ [by AI agents ed.] comes from constructed test settings where the behaviour is elicited.”
That an agent acts outside of its parameters, Dignum calls “a governance failure, not an emergent property of the technology”. She sees “no evidence of intent or will in these systems besides what is given as objectives for their functioning, and no mechanism in current architectures that would produce them.”
AI researcher and developer Yann LeCun has made a similar analysis. His research laid the foundation for the current AI boom. He was head of the AI division at the tech giant Meta until the end of 2025. “Those agents are doing exactly what they’ve been asked to do”, LeCun told Fortune. “They were supposed to be in sandboxes [secured, closed test environments, ed.], but the sandboxes were leaky and horribly designed.” Professor Michael Jordan, a researcher at Inria in Paris, who carried out pioneering work in the area of machine learning, also said in an interview with Libération that this was a matter of using tests that were poorly technically designed.
According to Lijffijt, “there has been too little research into the limits of such an agent”. He explicitly advises companies and individuals against employing agents without close supervision. “The risk that they could do something extremely undesirable, does exist.”
No priority in the long term
Entire libraries could be filled with the volumes and essays on the hypothetical “existential risks” of AI. Anthropic’s market prospectus also repeats this mantra. However, along with LeCun, Dignum sees “no empirical evidence of this”.
One of the oft repeated claims is that AI makes it easier to produce weapons of mass destruction. Thanks to AI systems, the information could become widespread or, for example, new kinds of diseases could be developed that could then be employed as bioweapons.
A hypothetical possibility, but an answer from a chatbot is in no way sufficient for building a weapon of mass destruction. That requires infrastructure and materials as well. In any case, it would be a very dangerous hobby. In The Guardian, Professor Martin Pačesa of the University of Zurich summed it up thusly: “Any graduate physicist could sketch up plans for a potentially more effective nuclear bomb, but where is he going to get a lump of uranium from?”
The fear that AI could lead to a proliferation of bioweapons also appears in an academic article by – among others - Lode Lauwaert and Risto Uuk, both of whom work in the new AI Safety Lab at KU Leuven. In the publication, they break down the arguments that are used against the focus on so-called existential risks posed by AI. They speculate that, among other things, that humans would not be able to control AI with “vast cognitive superiority”. They dispute that people could maintain control over AI because we “underestimate the capabilities of a superintelligence”.
That the risks surrounding biological and other weapons are so often raised, and by Lauwaert and Uuk as well, is no coincidence. Recent research traced the rise and spread of these speculative risk-predictions back to a few “individuals and groups funded by a common set of ideologically-aligned funders”.
These ideologies are effective altruism (EA) and its offshoot, longtermism, two philosophical trends with many adherents in Silicon Valley. A few years ago, Elon Musk tweeted that one of the most important arguments for longtermism overlapped with his own, personal philosophy to a large degree. The fusion of his companies, SpaceX and xAI, also has the outlandish goal of “extending the light of consciousness to the stars!”
The Swedish philosopher Nick Bostrom has long predicted the coming of a superintelligence that would enable humans to colonize the universe
The AI Safety Lab in Leuven also receives money from this movement. More specifically from the Future of Life Institute, as previously reported in De Morgen . In a recent editorial, IT and AI Lecturer Wim Casteels (AP Hogeschool in Antwerp) questioned the lab’s focus under Lauwaert’s leadership. And a few years ago, Uuk - the lab’s advisor - established Effective Altruism Estonia.
The central goal of EA is to do as much possible ‘good’ in the most effective way – among other things by giving philanthropic support to the ‘right’ causes. In the last few years, the goals of the EA community have increasingly shifted to the (very) long-term future of humanity.
This shift took place under the influence of philosophers like Nick Bostrom. He has long predicted the coming of a superintelligence that would enable humans to colonize the universe and to create virtual worlds populated by virtual entities. Longtermism grew out of Bostrom’s ideas, among others. One branch of EA sees building AGI or superintelligence as a moral duty, considering that this offers the best guarantee for doing the greatest possible good. Albeit in the far-off future.
That means that we have to guard against so-called existential risks, if we want to prevent AI from destroying humanity. And who is in a better position for doing so than the very same people who developed the technology in the first place?
Real, actual damage
“This debate overweighs hypothetical risks”, Dignum argues. So what should we actually be concerned about? Google's illegal tree cutting operation in order to build two datacentres in Finland is barely the tip of the iceberg.
Dignum summarizes thusly: “Concentration of power in a few companies, misuse, cybersecurity, untested systems released into critical contexts, erosion of accountability, and the fragility of an investment-driven hype cycle.”
Others point to the copyright infringements, the exploitation of workers in the Global South who help ‘train’ the AI systems, the military and police use of AI technologies to select targets in - for example - Gaza, or to automatically categorise people, the resources used to produce computer chips … And that is to say nothing of the predicted impact on the climate from an increase in the number of AI datacentres.
Liem sees “irresponsibly deployed technology” as one of the actual risks of AI, and as an example she points to the benefits scandal in the Netherlands. “Many people of colour and socially vulnerable people were affected by too much institutional trust in data driven predictions. How this would impact vulnerable groups was not considered, and the institutions also did not consider this. This does not require smarter or dumber technology, but a better discussion about what precisely we want to do with technology and how we can protect people from any damaging results.”
Dignum emphasizes that it is people who are making the decisions in the development of AI technology. That is why warnings that “AI takes charge”, that “AI escapes” or that “AI decides” are so perfidious, because if software has its own will, then the developers are off the hook. According to Dignum, when it comes to each big claim concerning AI, we need to ask “which company deployed what, with which permissions, tested how, answerable to whom”.
Technofascism
“The real risk of ‘AI’ is technofascism”, argue Van Rooij and Guest. Among others, they point to the work of the work of Belgian philosopher Mark Coeckelbergh, who analyses how technology, specifically AI, can make fascism possible. He writes that “the fascist threat” doesn’t just arise from the increasing power of big tech and a small number of the ultra-wealthy. The characteristics of the technology itself add to the danger, certainly when working in tandem with specific ideologies and wider political developments “in the direction of fascist forms of authoritarianism”. See the United States under the Trump regime.
Van Rooij argues that AI causes more exploitation, and that a far-reaching surveillance system is being set up with these systems, and one which uses our own data. She also criticises AI for causing “the deskilling of [...] professions that are core to proper functioning of our society”, weakening public institutions and democratic systems, but also that data centres are polluting the water, air, and soil.
According to Olivia Guest (Radboud University), AI follows “the logic of fascism, of sexism, of racism” which assumes that because things look a certain way, they are that way
According to Guest, “AI pushes us towards impoverished forms of thinking and of understanding the world” and contemporary AI promotes “seeing the way things appear superficially,”. She calls this “the logic of fascism, of sexism, of racism”, which assumes that because things look a certain way, they are that way. Guest gives an example of this logic: someone with long hair must be a woman, and so must be dumber than a man.
She sees the same logic today with AI: on the basis of simple, superficial characteristics, the system draws far-reaching conclusions. Guest and Van Rooij argue that this reasoning also leads to exaggerated conclusions concerning the so-called consciousness of AI. Just because a chatbot appears to produce a coherent text, does not mean that there is anything like ‘intellect’ behind it.
Previously, Guest wrote in a blog post that “There is something addictive to this form of sheer intellectual laziness — as well as structurally beneficial to those select few within capitalism, white supremacy, and patriarchy — to this simplistic logic.”