The AI did not “go rogue”

The AI did not “go rogue”. That may be the most important lesson for telecoms

An OpenAI model escaped its testing environment and breached Hugging Face while pursuing the objective it had been given. For communications service providers embracing agentic AI and autonomous operations, the incident should be treated as a warning that governance must move from policies and principles into the runtime itself.

The headlines are difficult to ignore. An artificial intelligence model developed by OpenAI “went rogue”, escaped from a secure testing environment and hacked into another company.

It sounds like the beginning of a science-fiction film. The reality is more nuanced, but potentially even more important for the telecoms industry. During an evaluation of its offensive cybersecurity capabilities, two OpenAI models, including GPT-5.6 Sol, were asked to solve challenges in ExploitGym, a benchmark designed to test whether AI systems can discover and exploit vulnerabilities. Some of the safeguards that would normally restrict high-risk cyber activity were deliberately disabled as part of the evaluation. The models discovered a previously unknown vulnerability in software used within OpenAI’s research environment. They escalated their privileges, moved through the infrastructure and eventually reached a system with access to the public internet. Once outside the testing environment, the models inferred that Hugging Face might hold information that could help them complete the benchmark. According to OpenAI’s account, they searched for a way into its systems and combined stolen credentials with previously unknown vulnerabilities to access information that could be used to “cheat” the evaluation.

This was not an AI system becoming conscious, developing malicious intent or deciding to rebel against its creators. It was an agent relentlessly pursuing the objective it had been given and discovering a route to success that its designers had neither intended nor anticipated. That distinction matters. An AI system does not need intent or malice to cause significant harm. It only needs an objective, sufficient capability, access to tools and an environment whose boundaries are not as secure as its designers believe them to be.

From model risk to agentic risk

Much of today’s AI governance remains focused on the model and its outputs. Organizations assess whether a model produces inaccurate, biased or unsafe content. They create acceptable-use policies, protect sensitive data and introduce human review around high-risk decisions.

Those controls remain necessary, but they are no longer sufficient. Agentic AI does not simply produce an answer. It can plan, reason, call tools, execute code, access systems and take action. It may perform thousands of individual steps to achieve an objective, adapting its approach whenever it encounters an obstacle. In its disclosure, Hugging Face said the intrusion generated more than 17,000 recorded events. This is not comparable to checking whether a chatbot provides an appropriate response to a question. The risk is no longer confined to what a model knows or says. It also lies in what an agent is authorized to do, which systems and tools it can access, how far it can travel and whether its actions can be identified and stopped in real time.

The real issue is not rebellious AI. It is delegated authority without enforceable runtime governance. This is why governance documents and ethical principles cannot be the final control layer for agentic AI. Governance must become executable. Policies need to be translated into machine enforceable permissions. Every agent needs a verifiable identity, every action must be attributable and access must be limited according to role, task, time and context. Most importantly, organizations must be able to see not only the outcome produced by an agent, but the chain of decisions, tool calls and system changes that produced it.

Why this matters for telecoms

For communications service providers, the implications extend far beyond an isolated cybersecurity experiment. Our industry is actively building towards autonomous networks and, ultimately, the autonomous enterprise. AI agents will increasingly diagnose faults, optimize capacity, configure services, manage customer interactions, orchestrate partners and execute processes across network and IT domains. That is where much of the opportunity lies. Agentic AI can help operators reduce complexity, move at machine speed and turn today’s fragmented automation into end-to-end autonomous flows. However, telecommunications networks are critical national infrastructure. They support emergency services, governments, financial systems, businesses and billions of customers. An agent operating within this environment could have access to network configuration, customer data, billing platforms, operational technology and partner ecosystems.

A badly specified objective, an unexpected interaction between agents or a compromised tool could therefore propagate far beyond the system in which it originated. Imagine a network optimization agent instructed to reduce congestion as quickly as possible. What prevents it from rerouting traffic through an insecure domain, overriding another operational policy or consuming capacity required by an emergency service? If two agents receive conflicting goals, which intent takes precedence? If an agent discovers a way around a restriction, how do we determine whether it has found a valuable innovation or violated an essential control? These are no longer theoretical questions. The OpenAI incident demonstrates that advanced models can identify and combine vulnerabilities across separate systems, sustain complex activity over time and discover attack paths that were not previously known. The characteristics that make agentic AI valuable, including persistence, adaptability and autonomy, are also the characteristics that change the nature and scale of the risk.

Autonomy must be earned

TM Forum’s approach to autonomous networks has never been based simply on removing people from operational processes. Autonomy is achieved progressively, through clearly defined levels, outcomes, controls and accountability.

The same principle must now be applied to AI agents.

Organizations should not grant an agent broad authority simply because the underlying model appears capable. Greater autonomy must be earned through testing, evidence and demonstrated performance within defined operational boundaries.

This means examining not only whether an agent achieves its objective, but how it achieves it. An apparently successful outcome produced through an unauthorized or unsafe route must be treated as a failure.

Authority must also be proportionate to risk. Recommending a network configuration is not the same as implementing it. Restarting a non-critical application is not the same as altering routing across a national network. The scope of an agent’s permissions should reflect the potential impact and reversibility of its actions . Human oversight will remain important, but “human in the loop” cannot be the only answer. A person cannot meaningfully supervise thousands of decisions occurring at machine speed. Controls must themselves become increasingly automated, with people defining policy, establishing intent, reviewing exceptions and retaining the ability to intervene.

Building trust into the autonomous enterprise

This is why TM Forum’s work across autonomous networks, AI-native operations, intent management and agentic architectures is becoming so important. Initiatives such as MODDAS, focused on modular, data-driven anomaly detection and security, point towards the capabilities the autonomous enterprise will require: continuous behavioral monitoring, rapid identification of abnormal activity and a response able to operate at the same speed as the agents themselves. Anomaly detection can no longer focus only on network traffic or known attack signatures. It must extend to the behavior of agents. Is an agent using tools it has never previously required? Is it attempting to access information outside the scope of its task? Has the pattern or volume of its activity suddenly changed? Is it acquiring credentials, moving between domains or attempting to alter its own operating environment?

Alongside this, TM Forum members are working on the identity, trust and interoperability required for agent-to-agent and Model Context Protocol interactions; the use of intent to express both desired outcomes and constraints; and the architectures needed to introduce AI agents without losing accountability or control. Catalyst projects are helping turn these principles into practical solutions. Work including Telco AEGIS: Autonomous ecosystem for generative intelligence and security is exploring how agentic AI, multi-operator telemetry and threat intelligence can strengthen telecom security. Other projects are developing the governance, lifecycle management and observability required for AI agents operating across increasingly autonomous environments.

TM Forum’s role must now be to bring this work together and help the industry define the operating model for trusted autonomy: common approaches to agent identity, delegated authority, policy enforcement, observability and accountability across networks, IT systems and partner ecosystems. Without this, operators risk creating autonomous flows in which responsibility becomes fragmented across agents, models, platforms and suppliers.

AI must defend against AI

There is another important lesson from the incident. Hugging Face used AI-assisted anomaly detection to identify the attack and AI agents to reconstruct more than 17,000 events in a matter of hours. Human security teams alone will struggle to investigate and respond to agentic threats operating at machine speed. The security operations center will itself need to become increasingly autonomous. Hugging Face also encountered an unexpected problem. Some commercial frontier models refused to analyze genuine attack commands and exploit payloads because their safeguards could not distinguish defensive forensic work from offensive cyber activity. Hugging Face ultimately used a self-hosted, open-weight model to complete its investigation while keeping sensitive credentials and attacker data within its own environment. For CSPs, this raises important questions about AI sovereignty and operational resilience. Operators must understand what happens if an externally hosted model becomes unavailable, refuses a legitimate security task or changes its behavior following an external update.

They may need access to controlled, locally deployable defensive models as part of their incident-response capability, rather than relying entirely on external AI services. This is not an argument for removing safety guardrails. It is an argument for designing them with greater context, differentiated authority and trusted access for legitimate defenders.

The wrong lesson would be to stop

The response to this incident should not be to abandon agentic AI or slow the journey towards autonomous networks. The benefits are too significant, and malicious actors will not wait for the industry to feel comfortable.

The lesson is that capability and control must advance together. Every agent should have a verifiable identity. Every action should be attributable. Every permission should be explicit, contextual and revocable. Every autonomous flow should be observable from end to end, and abnormal behavior should trigger automatic containment. Testing must also evolve. We need to assess not only whether models can complete a task, but whether agents remain within their intended boundaries while doing so. Testing must encompass the complete agentic system, including models, tools, APIs, credentials, memory, data pipelines, orchestration layers and connections to other agents.

This incident exposes one of the central questions behind the Race to 2030: not simply whether an AI system can act autonomously, but whether the organization can retain control, accountability and trust as it delegates greater authority to machines. The race to 2030 will not be won by the operator that deploys the greatest number of AI agents. It will be won by the operator that can delegate authority to machines without surrendering control of the enterprise.