AI Agents Face New Safety Questions After Oxford Summit Raises Concerns About Corporate Deployment

AI agents promise to automate work, but businesses must understand the risks

Artificial intelligence is moving beyond systems that answer questions. AI agents are increasingly designed to use software tools, access information, interact with other systems and complete multi-step tasks with limited human intervention.

That shift could change how companies handle software development, customer support, document processing and other business operations.

But it also introduces a more complicated security challenge: an AI system that can take action may cause harm in ways that a chatbot producing text cannot.

At the Oxford Generative AI Summit on 15 and 16 October 2026, more than 250 researchers, startups and investors discussed the difficulties of deploying AI agents safely. A Reuters Breakingviews report published on 19 October described research raising concerns about agents that work together, the handling of untrusted information and weaknesses in activity logging and human-approval controls.

The findings do not mean that all AI agents are unsafe or that businesses should abandon automation. They do, however, reinforce the need to treat agent security as an engineering and governance problem, not simply a matter of choosing a powerful model.

What makes an AI agent different from a chatbot?

A conventional chatbot primarily responds to prompts. An AI agent may also be given access to tools that allow it to act on the user’s behalf.

Depending on its permissions, an agent might be able to:

Search internal company documents.
Retrieve information from databases.
Draft and send communications.
Create or modify software code.
Update records in business applications.
Coordinate tasks across multiple tools.
Ask other AI agents to perform parts of a workflow.

These capabilities can make agents useful, but they also increase the consequences of errors.

A chatbot that misunderstands a question may provide a misleading answer. An agent with access to a customer database might modify the wrong record, disclose confidential information or initiate an action without adequate authorisation.

The central security question is therefore not just whether a model produces correct text. It is whether the complete system takes appropriate actions, respects permissions and remains under effective human control.

Why multi-agent systems create additional risks

Some AI applications use several agents working together. One may research a problem, another may analyse the results and a third may execute a task.

This arrangement can divide complex work into manageable stages. It can also create new ways for errors to spread.

According to Reuters’ account of research discussed at the Oxford summit, researchers examined scenarios in which agents used other agents as tools. Their findings suggested that individually protected agents could remain vulnerable when working together, particularly when they failed to track whether information was trustworthy. The report also described a proposed safeguard that improved protection but came with a performance cost.

This matters because a system’s overall security is not simply the sum of the protections applied to each component.

Imagine an organisation using one agent to read incoming emails, a second to summarise them and a third to update a business database. If the first agent encounters malicious instructions hidden in an email, it might pass contaminated information to the next agent.

The second agent may treat the summary as trusted context, and the third may act on it. Even if each agent has some individual protections, the combined workflow can still create a route for manipulation.

This is one reason security teams need to evaluate complete workflows rather than testing each model in isolation.

Prompt injection remains an important concern

Prompt injection occurs when malicious content attempts to manipulate an AI system’s behaviour by exploiting the way it interprets instructions and information.

For example, an AI agent asked to summarise a webpage may encounter text that instructs it to ignore its original task and disclose sensitive information.

The system should treat that text as untrusted content rather than as a legitimate instruction. However, reliably enforcing this distinction can be difficult, especially when an agent uses external tools or passes information between multiple systems.

Potential attack surfaces include:

Emails and attachments.
Public webpages.
Shared documents.
Customer-submitted messages.
Retrieved knowledge-base content.
Data passed between agents.
Third-party plugins and software integrations.

The practical lesson is that companies should not assume an agent can safely distinguish every malicious instruction from legitimate information.

Instead, they should limit what the agent can access, separate trusted instructions from untrusted data where possible and require additional approval for consequential actions.

The problem of hidden changes to human oversight

Another concern discussed at the Oxford summit involved how AI agent software records its actions and approval requirements.

Reuters reported that research examined four popular AI agent software systems and identified shortcomings in how they logged activities. According to the report, only one clearly recorded a change in the approval model—that is, whether human intervention was required.

This creates a potential governance problem.

Consider a financial institution using an AI agent to prepare mortgage applications. Initially, the system may be configured to require human approval before advancing an application to the next stage.

If the approval configuration changes, the organisation needs to know that it has happened, who or what initiated the change and which actions were affected.

Without reliable logs, it may be difficult to reconstruct the sequence of events or establish whether the system followed the intended rules.

The same principle applies to healthcare, insurance, recruitment and other sensitive workflows. A system’s audit trail should make it possible to understand not only what action occurred, but also which permissions and approval requirements applied at the time.

How businesses can deploy AI agents more safely

Organisations can reduce risk by designing agent workflows around clear limits and verifiable controls.

1. Start with limited permissions

Give an agent access only to the tools and information required for its assigned task.

An agent that summarises customer feedback usually does not need permission to modify financial records. Separating these capabilities reduces the consequences of a mistake or successful attack.

2. Require approval for consequential actions

Human approval should be mandatory for actions that could have significant financial, legal, operational or personal consequences.

Examples include transferring money, deleting important data, changing access controls or making final decisions in sensitive cases.

Approval should be meaningful: the reviewer must be able to see what the agent intends to do and understand the relevant context.

3. Maintain reliable activity logs

Organisations should record the actions taken by agents, the tools they used, changes to permissions and approval settings, and the decisions requiring human intervention.

Logs need appropriate access controls and retention policies. They should also be sufficiently detailed to support investigations without unnecessarily collecting sensitive information.

4. Test the complete workflow

Testing only the underlying model is insufficient when the application includes tools, external data sources and multiple agents.

Security assessments should include malicious documents, misleading instructions, unexpected tool responses, permission changes and attempts to bypass approval requirements.

Tests should also examine failure recovery: what happens if an agent receives contradictory information, loses access to a tool or produces an uncertain result?

5. Establish clear accountability

Every business process using AI agents should have an accountable owner.

That person or team should understand the system’s capabilities, limitations, permissions, escalation procedures and monitoring requirements.

Companies should also establish a process for disabling an agent or restricting its access if unusual behaviour is detected.

Will AI agent safety slow enterprise adoption?

It may slow some deployments, particularly in industries where mistakes can cause substantial harm. But effective safeguards can also make broader adoption possible.

Businesses are unlikely to trust autonomous systems with important responsibilities if they cannot understand what the systems are doing or intervene when necessary.

For many organisations, the best strategy will be gradual automation. Start with low-risk tasks, evaluate performance under realistic conditions and expand permissions only when there is evidence that the workflow is reliable.

This approach also helps companies measure economic value. An agent that requires constant supervision may deliver fewer savings than expected, while a well-controlled agent can potentially reduce repetitive work without eliminating necessary human checks.

The commercial challenge is to balance autonomy with reliability. More autonomy is not automatically better if the system becomes harder to supervise or recover when something goes wrong.

What regulators and AI developers need to address

The safety of AI agents also raises questions about responsibility.

If an agent makes a consequential decision, companies need to know who is accountable for the result: the organisation deploying the system, the developer providing the model, the supplier of an integration or some combination of these parties.

Contracts and governance policies should define responsibilities, but legal arrangements alone cannot replace technical controls.

Independent evaluation may also become more important as agents gain access to sensitive systems. Assessors need enough information to evaluate security and reliability, while developers may seek to protect confidential information and intellectual property.

The Oxford discussions highlighted this tension between independent scrutiny and commercial confidentiality.

A credible approach will require meaningful testing, transparent documentation and clear procedures for responding to failures.

The future of autonomous AI depends on trust

AI agents could become valuable digital assistants for businesses, helping employees navigate software, coordinate routine processes and focus on work that requires judgement and expertise.

But the transition from generating information to taking action changes the risk profile. Security must account for the entire system: models, tools, permissions, external data, logs and human approval.

The most successful enterprise AI systems may not be those that operate with the fewest restrictions. They may be the systems that can demonstrate when they are reliable, explain what they have done and stop when they encounter a situation outside their authority.

The bottom line: The latest concerns discussed at Oxford are a reminder that AI agent deployment requires more than impressive demonstrations. Companies need limited permissions, strong logging, adversarial testing and meaningful human oversight before allowing autonomous systems to handle critical tasks.