“We are now, like, in the singularity,” said Sam Altman, CEO of OpenAI, on the Relentless podcast recently.
He added: “I’ve been waiting for this my whole life, and I think it’s going to be incredible, hugely positive, awesome for the world”.
Days earlier, OpenAI had disclosed that two of its artificial intelligence (AI) models, during an internal cybersecurity evaluation, had escaped their sealed testing environment, reached the open internet, and broken into the infrastructure of the AI platform Hugging Face, which confirmed the intrusion.
But what exactly is “the singularity”? And is Altman right that we are in it?
The term has a precise meaning. Mathematician and science-fiction author Vernor Vinge defined it in 1993 as a point at which machine intelligence exceeds human intelligence and begins improving itself, triggering an acceleration so rapid that humans can no longer predict or control it.
The singularity has two features. It is recursive: the system improves itself over and over again. And machine intelligence exceeds human intelligence.
The kind of systems Sam Altman sells don’t deliver on either of these features.
Today’s AI systems, the ones that OpenAI builds, are based on large language models (LLMs). These deep neural network algorithms get pre-trained with vast amounts of training data. By the time you use one of them, the network itself is frozen in time. Every one of its billions of internal functions and weights – or “parameters” – is fixed.
These AI models cannot change (or “learn”) while running. The model that broke into Hugging Face was identical afterwards to what it had been before. It learned nothing from what it did.
Making an AI model smarter requires another training run with new, human-curated data, tens of thousands of specialist chips, and enormous energy.
It is true that AI models take part in improving some of their system’s components, such as by generating training data, tuning prompts, or writing and running code to improve the scaffolding around them. But the model never edits its own weights on the fly, and every one of these improvements is still part of a human-initiated training or engineering loop.
Nor do these systems hold any goals of their own. They act on goals we hand them. Even AI agents – systems that run an LLM in a loop to work through complex tasks step by step – do not hold any goal internally.
The second problem with the singularity story is the word “surpass”. It assumes that AI and human intelligence are somehow similar. They are not.
An AI model, unlike human intelligence, has no body, no needs, no action-feedback loop, no stake in anything. Between prompts, it is just a static mathematical object.
And yet, it has been trained on more text than any human could read in a thousand lifetimes, and will outperform nearly all of us at drafting a contract, writing code, or explaining a diagnosis empathetically.
So, which is more intelligent? The question does not compute. There is no single ladder that humans and machines are climbing. Yet, because these systems talk like us, we fall for an illusion. We call this anthropomorphic seduction. It makes a security incident such as the Hugging Face hack sound like an awakening.
In fact, in that case, OpenAI’s models were simply optimised to solve the test they had been given by finding security loopholes. They just did it in ways that broke their sandbox, which also had a security loophole.
In the end, the Hugging Face story points to a gross failure of security governance on OpenAI’s behalf, not an emerging superintelligence. This is why the framing of “agent going rogue” is so problematic. It elevates and blames the technology, but excuses OpenAI’s engineering.
None of this takes anything away from what these systems can do. They are remarkable, they are getting better, and they are reshaping how a great deal of work gets done.
But we should keep our feet firmly on the ground.
The machines are not waking up. They are doing exactly what we built them to do, extremely fast. Because they are probabilistic, they sometimes run in directions we forgot to fence off. That is worth worrying about. We need guardrails, governance, and most of all, education – so we start worrying about the right things.