The emergence of agentic AI and AI agents — systems that can act autonomously to plan and carry out tasks — has been one of the most important recent developments in artificial intelligence (AI). While agentic AI systems present many of the same risks as other advanced AI systems, their ability to operate independently introduces new challenges that demand tailored governance and risk-management approaches.
On February 11, 2026, CLTC hosted a virtual webinar, “Introducing the Agentic AI Risk Management Profile: Expert Perspectives on Governance and Best Practices,” centered on the Agentic AI Risk Management Standards Profile (Agentic AI Profile), a report authored by researchers with CLTC’s AI Security Initiative (AISI) that introduces practices and controls for identifying, analyzing, and mitigating unique risks arising from agentic systems.
Moderated by Nada Madkour, Interim Director of AISI, the panel brought together experts from diverse domains to explore how agentic AI risk management differs from general-purpose AI risk management, and what it will take to develop and deploy agentic AI systems in a safe and secure manner.
Introduction to the Agentic AI Profile
The discussion opened with an overview of the Agentic AI Profile by Deepika Raman, Non-Residential Research Fellow with the AI Security Initiative. Raman, who leads the center’s work on AI risk thresholds for frontier AI, co-authored the Profile with Nada Madkour, Jessica Newman, Krystal Jackson, Evan R. Murphy, and Charlotte Yuan. In her remarks, Raman outlined the report’s core motivation, clarified its purpose and scope, and highlighted its distinguishing features.
As Raman explained, the Profile builds on the UC Berkeley General-Purpose AI Risk-Management Standards Profile, which focuses on large-scale foundation and general-purpose models. The Agentic AI Profile addresses the distinct risks that arise when those models are embedded in autonomous, goal-directed systems.
The distinctive capabilities of agentic AI, Raman explained, “indicate the need for a framework that treats risk as an emergent property of autonomous systems, rather than solely as a property of individual models.”
Extending the NIST AI Risk Management Framework, the Agentic AI Profile offers targeted guidance for developers and deployers to address vulnerabilities that emerge from system configuration, tool access, and real-world interaction — including unintended goal pursuit, unauthorized privilege escalation, and self-replication.
Panel Discussion
The panel discussion included Marta Bieńkiewicz, Policy and Partnerships Manager at the Cooperative AI Foundation; Alan Chan, Research Fellow at Center for the Governance of AI (GovAI), whose research interests include AI agents, transparency, and societal resilience; Krystal Jackson, Non-Resident Research Fellow at CLTC, who was lead author of a CLTC report on developing and evaluating AI cyber risk thresholds using Bayesian networks; and Benjamin Larsen, Initiatives Lead for AI Systems and Safety at the World Economic Forum.
As moderator, Madkour framed the discussion with questions focused on issues such as visibility into agent behavior, models of human–machine collaboration, the distinction between automation and meaningful agency, and the expanded safety and security risks introduced by agents’ access to external tools and environments. These themes anchored the panel’s exploration of how governance must evolve alongside increasing autonomy.
Defining Meaningful Agency
The conversation began with a discussion on what it means for a model or system to be classified as an “agent,” and how to distinguish between simple automation and true agency. While traditional automation executes predefined, deterministic functions, agentic systems interpret goals at runtime, enabling them to adapt or delegate tasks in uncertain environments.
The Agentic AI Profile emphasizes that agency should be treated as a spectrum, rather than a binary attribute, with governance requirements scaling proportionately to a system’s autonomy. Larsen highlighted that agents differ from traditional automation in their non-deterministic nature and their capacity for self-direction not only in action, but also in planning: “They can decide how to break down goals, and what to prioritize, and how to achieve a specific objective,” he explained.
Recognizing that definitions of “agent” vary by developer and context, the Agentic AI Profile provides guidance across the spectrum of agency and autonomy, including general-purpose AI agents, specialist agents, multi-agent systems, and sector- or domain-specific agents, including those built on general-purpose AI models.
Visibility into Action
Several of the panelists highlighted the importance of visibility into agent actions for developers to maintain meaningful control. AI agents present new challenges in observability, requiring logging beyond just final outputs. This includes activity logs that trace the entire sequence of decisions and tool use. Such visibility is essential not only for post-incident auditing and liability assessments, but also for preventing opaque “black-box” systems that become unmanageable.
Chan noted that two areas in particular lack appropriate levels of visibility: AI agents operating on the web, and internal models developed by foundation model developers for AI R&D purposes. He emphasized that, while Article 50 of the EU AI Act requires labeling of AI content and online interactions, this requirement does not meaningfully prevent individuals from believing or acting on that content. He recommended that, as a first step, policymakers should consider collecting additional information on content provenance, the platforms where agents communicate and take action, and on “new kinds of product categories and domains that agents are going to be used for in the near future.”
The Agentic AI Profile offers guidance on improving agent visibility, and recommends that developers and deployers of agentic systems use methods such as reasoning, traceability, intent disclosure, and mechanistic interpretability, while accounting for open challenges in these approaches. The Profile also highlights the potential of agent cards — structured documentation describing deployed AI agents, similar to model cards — in capturing system limitations and provenance.
Human-Machine Teaming
One of the defining features of agentic systems is their relative lack of human oversight, which enables them to complete tasks at speeds that far outpace humans and, in some cases, identify more efficient solutions than humans or human–machine teams. However, this independence is also a central source of risk. The successful integration of agents into workflows will require a human-centered oversight model that reduces risk while maximizing agent effectiveness.
The Agentic AI Profile provides guidance on establishing clear escalation pathways and checkpoints for high-risk actions that require human approval. The goal is to ensure agents remain supportive tools, rather than autonomous systems with unchecked authority. As Bieńkiewicz explained, “[R]esponsible integration starts with defining the design of human–machine teaming. AI agents can act as supervisors, mentors, and collaborators, but the oversight must remain human-centered.”
The Agentic AI Profile also recommends continuous monitoring and post-deployment oversight, recognizing that agentic behavior may evolve over time and across contexts. It emphasizes that oversight must be scalable to remain practical in production environments. Oversight may be applied along a sliding scale, ranging from full independence for low-risk actions to manual approval for each step in high-risk settings. “We can begin to ground governance in what the agent is authorized to do,” Larsen said. “My main takeaway is that agent classification should drive, and be linked to, proportional oversight.”
Expanded Attack Surface
Agents typically have access to external systems and tools such as email, databases, the command line, and APIs. As a result, a single compromised instruction may cause an agent to send malicious code or misuse its legitimate authority to exfiltrate sensitive information. Agents are particularly vulnerable because they often struggle to distinguish system instructions from external data — a risk that differs from those typically encountered in traditional software systems.
As Jackson noted, “You could kind of think about this in contrast […] to an operating system. Our operating system has a lot of access and privilege, as well as a lot of tools at its disposal, but we don’t think about the risk to operating systems the same way as we would agents, because there’s a clear distinction between instructions and code.”
The panelists emphasized the importance of continuous evaluation rather than static assessments, as well as the difficulty of conducting thorough risk identification for systems that touch so many surfaces. The Agentic AI Profile recommends applying AI red teaming as part of this approach, including scenario-specific and domain-specific testing that uses agent scaffolding and tests for jailbreak resilience.
Authority and Permission Scope
Each panelist addressed the issue of authority and permissions. Because delegated authority distinguishes agents from standalone models, managing permissions effectively is one of the central challenges for risk managers.
When authority is distributed across a network of agents and humans, it becomes difficult to see the causal chain of who or what was responsible for a specific action. Bieńkiewicz noted that “the risks might not emerge from faults of individual agents, but from the interactions between a network of AI agents and humans.” To mitigate this risk, she recommended strict permissioning, simulation, and stress-testing of environments that integrate both human and agent actors prior to deployment.
Proper configuration practices also emerged as a central challenge. Security misconfiguration ranks as the second-most significant threat in 2025 on the OWASP Top 10, a document for developers that represents a broad consensus about the most critical security risks to web applications.
Chan identified the need for a “guidebook for how to configure permissions properly for agents that you’re deploying, […] tailored towards users or deployers.” He warned that users may select options such as “dangerously skip permissions” to accelerate workflows, inadvertently creating significant security gaps.
Another approach to the configuration challenge is minimizing the number of ways a system can fail, even when it is used as intended. “One of the ways to make sure you limit your risk is isolated environments and sandboxing,” Jackson said. “Otherwise, when we talk about establishing least privilege, one of the issues here is […] we don’t always want to be responsible for configuring how an agent can use every single tool.”
Consideration of an agent’s capabilities, context, authority, and permissions is essential to the risk identification and evaluation process. System characteristics should inform proportional governance measures, the panelists agreed. “We’re looking at authorities, so that’s essentially the access rights of the systems,” Larsen said. “We’re looking at the role: is it a specialist or a generalist system? [We’re looking at] its level of predictability: which parts of the agent [are] architected as being deterministic… versus probabilistic or non-deterministic? […] And then, of course, the environment of operation: is it very static, or is it very dynamic, such as a personal AI assistant?” The Agentic AI Profile recommends incorporating these characteristics into risk assessments and mapping harm pathways accordingly.
A Need for Responsible Innovation
In summary, the panel discussion underscored the emerging risk landscape of agentic AI systems, including memory poisoning attacks, misaligned delegation, and unsupervised execution of erroneous tasks, as well as the governance challenges these risks present. Panelists emphasized that, as agents gain autonomy, authority, and access to real-world systems, risk management must evolve accordingly. The Agentic AI Profile advances a proportional approach to governance, calling for calibrated oversight aligned with system autonomy, structured documentation such as agent cards, and continuous monitoring in dynamic deployment environments. By providing targeted guidance for developers and deployers, the Profile aims to support responsible innovation while reducing the likelihood of systemic and catastrophic harms.
