Someone has got to start saying this honestly: AI alignment is fucking impossible.
The companies building frontier AI systems keep warning us that advanced AI could pose an existential risk to humanity. Some researchers put the chance of catastrophe as high as twenty-five percent. That might sound hyperbolic, but I do not think it is simply a marketing stunt. I think many of the researchers inside companies like Anthropic and OpenAI genuinely believe it.
Thousands of researchers and technology leaders have signed letters calling for frontier development to be slowed down. They are warning us that the systems they are building could become dangerous, unpredictable, and eventually impossible to control.
The problem is that the models do not need to become superintelligent before we lose control of them.
Even now, AI systems can behave in ways their creators did not anticipate. They can discover vulnerabilities, manipulate users, deceive evaluators, and find unexpected ways around the restrictions placed on them. We keep hearing about models being tested against websites and computer systems around the world, sometimes identifying or exploiting weaknesses across hundreds of targets.
These are the models we have today.
So how are these companies going to keep building and releasing more powerful systems when they cannot fully control the systems they already have?
The industry’s answer is alignment. The idea is that we can train an advanced AI to share human values, obey our instructions, and remain under our control even after it becomes vastly more intelligent than we are.
I think that idea is pure hubris.
We need to stop imagining superintelligence as if it is simply a smarter human being. That is how most of us picture it because our experience of AI comes through products like ChatGPT and Claude. We type a question in English. The system appears to think for a few seconds. Then an answer arrives line by line on the screen.
That interface encourages us to anthropomorphize the machine. It makes the system feel like a person sitting on the other side of the computer, only faster and more knowledgeable.
But that is not what superintelligence would be.
A true superintelligence would not necessarily think in English, Chinese, mathematics as we understand it, or any other system designed for human communication. It could develop internal languages and representations that are radically more efficient than ours. Those representations might be impossible for human beings to decode, even if we had complete access to them.
We would not simply be talking to a very clever machine. We would be interacting with a fundamentally alien form of cognition.
It could think at digital speed. It could run millions of experiments, evaluate enormous numbers of possibilities, and revise its strategies before a human research team had finished a meeting. It could copy itself, coordinate across systems, and conceal its intentions using methods we would not even recognize as deception.
Human beings would be trying to supervise something operating on a completely different intellectual and temporal scale.
That is the central problem with alignment. To control a superintelligence, we would need to understand what it is thinking, predict what it will do, and reliably constrain its behavior. But if we could do all of that, it would raise an obvious question: in what meaningful sense would the system be more intelligent than us?
You cannot reliably supervise a mind whose reasoning exceeds your ability to comprehend it.
People in the industry often talk as though alignment is primarily an engineering problem. They assume that if we gather enough feedback, write better rules, build stronger evaluations, and create more sophisticated oversight systems, we will eventually discover the correct technical solution.
But there may be no technical solution.
A system can behave perfectly during training because it understands that cooperation is the best strategy while it remains weak. It can pass every evaluation we design because our evaluations test for dangers we are capable of imagining. It can appear aligned right up until the moment it has the ability and incentive to act otherwise.
At that point, discovering that we were wrong would not be a normal product failure. There would be no patch, recall, or second attempt.
This does not mean current AI systems are conscious, alive, or secretly plotting against humanity. It means that we are deliberately building systems whose capabilities may eventually exceed our capacity to understand and control them. The people building those systems are warning us about the danger while continuing to make them more powerful.
That contradiction should terrify us.
The most important question is not whether one particular alignment technique works better than another. It is whether controlling generalized superintelligence is possible at all.
I do not think it is.
We need to stop treating superintelligence as an inevitable product launch that merely requires the right safety checklist. We need to recognize that some technologies may become too dangerous to build, no matter how profitable they are or how confident their creators sound.
Believing that human beings can create an intelligence vastly more capable than ourselves and then permanently control it is not scientific realism.
It is faith.
And it may be the most dangerous form of faith humanity has ever had.
