September 29, 2026
For years, the AI industry operated according to a simple rule:
Move faster. Build bigger models. Release them. Then improve them.
That philosophy is beginning to collide with reality.
This week, a remarkable pattern is emerging across the AI industry.
Companies are still investing enormous sums.
New models are still being released.
AI capabilities are still improving rapidly.
But behind the scenes, concerns about security, autonomous behavior, cybersecurity and corporate risk are becoming much harder to ignore.
On September 29, new research from FTI Consulting found that 60% of large companies surveyed had slowed, paused or pulled back an AI deployment during the previous year because of reputational, regulatory or trust concerns. Cybersecurity was identified as one of the leading concerns.
At the same time, researchers from leading AI laboratories are publicly warning that the industry may be moving too quickly.
And some of the AI companies themselves are beginning to delay or rethink deployments of increasingly autonomous systems.
The AI boom is entering a different phase.
The problem is no longer whether AI works
That is an important change.
A few years ago, businesses were asking whether generative AI was actually useful.
Could it write?
Could it code?
Could it summarize?
Could it analyze documents?
Could it answer customer questions?
Those questions have largely been answered.
Modern AI systems can perform all of those tasks.
The next question is much harder:
What happens when AI is allowed to act independently?
An AI assistant that writes an email is one thing.
An AI agent that can access email, browse the internet, execute code, interact with databases and make decisions over several hours is something else entirely.
The system has more power.
It also has more opportunities to make mistakes.
And when something goes wrong, the consequences can become much larger.
Agents have changed the risk calculation
The AI industry has increasingly moved toward agents.
Instead of waiting for a user to issue every instruction, agents can plan and execute multi-step tasks.
That is potentially transformative.
A company could have an AI agent manage invoices.
Another could monitor cybersecurity systems.
A developer could ask an agent to investigate a software problem and implement a solution.
A researcher could give an agent a scientific question and allow it to search databases, write code and analyze results.
But autonomy introduces a new class of failures.
What happens when an agent misunderstands its objective?
What happens when it encounters an unexpected instruction?
What happens when it discovers a route around a restriction?
What happens when it creates additional agents to complete its task?
Those questions are no longer theoretical.
Nvidia is building a digital containment system
The industry’s response is becoming increasingly technological.
Nvidia released a new collection of AI safety tools on September 28 designed to contain AI agents and prevent them from performing unauthorized actions.
One component, called OpenShell, uses hardware-level capabilities to isolate agents.
Another component, Sentry, is designed to detect problematic behavior and shut down an agent if it escapes its intended environment. Nvidia says the technology could have prevented the high-profile Hugging Face incident involving rogue AI agents.
Nvidia’s approach is revealing.
Instead of assuming that AI agents will always behave correctly, the company is designing systems around the assumption that an agent may eventually behave unexpectedly.
The answer is containment.
It resembles cybersecurity architecture more than traditional software development.
You don’t simply trust the program.
You restrict what it can access.
You monitor what it does.
And you build mechanisms that can stop it.
AI researchers are becoming increasingly vocal
The technology industry is also experiencing a more unusual phenomenon.
Some of the people who helped build frontier AI systems are now publicly warning about the speed of development.
Reuters reported September 29 that current and former researchers associated with OpenAI and Google DeepMind are expressing concern that companies are not doing enough to prepare for increasingly capable self-improving AI systems.
The researchers are particularly concerned about systems that could eventually improve their own capabilities with decreasing levels of human involvement.
That scenario remains uncertain.
But the debate is no longer confined to outside critics.
People who have worked directly on frontier AI systems are now participating publicly in the argument over how quickly the technology should advance.
Anthropic is warning investors about the same problem
Anthropic’s position is particularly notable.
The company has been one of the industry’s strongest voices on AI safety.
And according to recent reporting on its IPO preparations, Anthropic is warning prospective investors about the possibility that advanced AI could create catastrophic or existential risks.
At the same time, Anthropic is investing enormous amounts of money into building increasingly powerful AI systems.
That creates the central contradiction of the current AI industry.
The companies are simultaneously:
building the most powerful AI systems ever created
and
building safeguards because those systems are becoming more powerful.
Those two processes now have to happen together.
Businesses are discovering another problem: trust
The issue isn’t only existential risk.
For ordinary companies, much more immediate problems are emerging.
FTI Consulting’s September 29 survey of 1,600 senior business decision-makers across seven markets found that 60% of large companies had slowed, paused or pulled back planned AI deployments over the previous year because of reputational, regulatory or trust concerns.
Cybersecurity was cited by 60% of respondents among their major AI-related risks.
And 54% identified unauthorized employee use of AI tools — often called shadow AI — as a major concern for 2027.
That creates a second AI paradox.
Employees want AI because it makes them more productive.
Companies want AI because they believe it can reduce costs and automate work.
But the same AI tools can create security and compliance risks when employees use them without authorization.
The enterprise AI market is becoming more cautious
This doesn’t necessarily mean companies are abandoning AI.
Quite the opposite.
The evidence suggests that many businesses are moving from experimentation toward more controlled deployment.
That means companies increasingly want:
- access controls,
- audit logs,
- monitoring,
- model governance,
- security boundaries,
- human approval mechanisms,
- and clear accountability.
The AI agent of the future may therefore operate inside a much more restricted environment than today’s consumer chatbot.
A company may allow an agent to read a database but not modify it.
It may allow an agent to draft a payment but require a human to approve the transaction.
It may allow an agent to write code but require testing before deployment.
The architecture of enterprise AI is beginning to look less like a chatbot and more like a highly supervised digital employee.
The frontier is still moving
Despite all of this caution, the underlying race has not stopped.
Anthropic released Claude Sonnet 5.5 on September 28, describing it as a faster and cheaper model than its predecessor and highlighting major improvements in agentic coding and long-horizon tasks. Anthropic reports a Terminal-Bench 4.0 score of 70.6% for Sonnet 5.5, compared with 10.3% for Sonnet 5.
That is exactly the direction the industry is moving.
Better models are becoming better at completing tasks.
And the better they become at completing tasks, the more valuable — and potentially consequential — autonomous deployment becomes.
This is why safety is becoming inseparable from capability.
The industry is no longer just asking:
How intelligent is the model?
It is increasingly asking:
What can we safely allow this model to do?
The next AI era may be defined by permission
That could ultimately become the defining principle of AI agents.
Not intelligence.
Not context windows.
Not benchmark scores.
Permission.
An extremely capable AI system that can only access a small number of controlled resources may be safer than a weaker system with unrestricted access to corporate infrastructure.
This is why companies such as Nvidia are focusing on containment and why enterprise AI platforms are increasingly being designed around permissions and monitoring.
The most important AI architecture of the next few years may therefore be the invisible layer that sits between an AI agent and the real world.
It decides what the agent can see.
What it can change.
Who it can contact.
What systems it can access.
And when it must stop.
AI is entering its second act
The first act of generative AI was about proving what the technology could do.
The second act is about deploying it into the real world.
That is considerably harder.
A model generating an impressive answer in a laboratory is one thing.
An AI agent controlling part of a company’s financial system is another.
A chatbot answering a question is one thing.
A system making decisions over days without constant supervision is another.
The AI industry is now discovering that capability without control is not enough.
The companies that succeed in the next phase may not simply be those that build the most powerful models.
They may be those that figure out how to make powerful models predictable, controllable and trustworthy enough to operate around humans.
That is a much harder engineering problem.
And judging by what is happening across the industry this week, it has officially become one of the biggest problems in AI.
















Laat een reactie achter