Part of the Agentic AI Security Reference Library. Related deep dives: Anthropic · Apollo · METR · Papergraph / roots · AI-assisted hacking cases · Foundational agenda papers.

Level 3 — Research Frameworks: Annotated Bibliography

The full frontier of agentic-AI security research, standards, and tooling — 312 sources, deduplicated and citation-verified, grouped into 14 clusters. Every entry links to its primary source. Newest first within each cluster.

Compiled for The Multiverse School · Agentic AI Security. Companion to the interactive map and the Level 1–2 handouts. Sources span the 2025–2026 wave; a handful of 2026 arXiv IDs are the earliest of the next cohort and are marked provisional by date.

Clusters

• Surveys & Taxonomies — 20
• Threat Models — 33
• Defense Frameworks — 38
• Guardrails & Runtime — 18
• Benchmarks & Evaluation — 25
• Red-Team & Attack — 30
• Prompt Injection — 15
• MCP Security — 13
• Multi-Agent / A2A — 9
• Identity & Authorization — 25
• Governance & Assurance — 39
• Formal Verification — 15
• Standards & Bodies — 23
• Certifications — 8
• Other — 1

Surveys & Taxonomies

20 sources.

• The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities
  _{arXiv · 2026-07-07 · arXiv:2607.05743} By Mohammadreza Rashidi. A systematization-of-knowledge (SoK) of execution-layer security for AI coding agents that systematizes 39 papers (2023-2026) into 17 categories spanning sandbox isolation, capability/access control, policy enforcement, TOCTOU races, MCP threats, identity delegation, and egress control. Identifies five gaps: isolation lacks shared benchmarks; policy-enforcement defenses show 69-98% failure rates yet aren't re-tested; TOCTOU and MCP threats studied separately despite a shared root cause; policy-authoring errors unaddressed; out-of-scope agent behaviors (up to 17.1%) uncovered. Includes verified CVEs and supplementary artifacts. Teach: Excellent syllabus backbone: a taxonomy and gap-map an instructor can turn into a full course module on coding-agent execution security.

• Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability
  _{arXiv · 2026-07-13 · arXiv:2607.11086} Large-scale study built on MCPZoo (64,000+ unique MCP servers) evaluating runtime MCP servers and the reliability of existing MCP security scanners; finds scanners flag 96.89% of servers as risky yet under 50% of sampled alerts are true positives, mapping real attack/defense surface and tooling gaps for provenance and vulnerability detection. Teach: Public large-scale study; useful for critically evaluating MCP scanning tools in class.

• A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
  _{arXiv · 2026-04 · arXiv:2604.23338} Kexin Chu introduces the Layered Attack Surface Model (LASM), decomposing the agentic stack into seven layers plus a four-class temporality axis, analyzing 116 papers (2021-2026). Finds upper stack layers remain sharply under-explored, especially for long-horizon and stack-propagating threats. Exactly the layered attack-surface/threat-taxonomy framing requested. Teach: Seven-layer LASM framework, cross-layer defense taxonomy, and temporality axis — a ready-made lecture scaffold and lab-mapping exercise.

• The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
  _{arXiv (accepted to USENIX Security 2026) · 2026-03 · arXiv:2603.11088} Kim, Liu, Wang, Qiu, Li, Guo, and Song present a systematic examination of the design space, attack landscape, and defense mechanisms of LLM-based AI agents, with case studies identifying gaps. Billed as the first systematic framework for understanding agent security risks and defense strategies. Strong taxonomy + defense pairing. Teach: Component-vs-system risk taxonomy and defense-in-depth mapping; teachable as a design-dimensions-to-risks exercise.

• The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  _{arXiv · 2026-02 · arXiv:2602.10453} SoK/survey that establishes taxonomies of prompt-injection attacks on LLM agents (by payload generation strategy) and defenses (by intervention stage), and introduces the AgentPI benchmark for context-dependent agent tasks. Shows current defenses struggle to jointly balance security, functionality, and performance. Good anchor for the 2025-2026 wave. Teach: Yes. Systematization-of-knowledge with taxonomy plus the AgentPI benchmark; well-suited for lecture framing, reading lists, and a benchmark-driven lab.

• The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems
  _{arXiv · 2026-02 · arXiv:2602.17753} An index documenting the origins, design, capabilities, ecosystems, and safety features of 30 state-of-the-art deployed AI agents, built from public data and developer correspondence. Finds uneven transparency, with most developers sharing little about safety, evaluations, and societal impacts; provides a dataset for comparing how real agents disclose capabilities and safety measures. Teach: Public dataset/index (Staufer, Feng, Wei, Bailey, Casper, Kolt et al.); teachable as a transparency-audit exercise comparing deployed agents. Note: original 'date' field ('2025-2026') was malformed; arXiv id 2602 indicates Feb 2026.

• Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale
  _{arXiv · 2026-01-15 · arXiv:2601.10338} Large-scale empirical measurement of security vulnerabilities across real-world agent skills/plugins, quantifying dependency and provenance risk in the agent extension ecosystem. Teach: Public empirical study (Liu, Wang, Deng et al.); good for a measurement/analysis exercise on plugin risk.

• A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes
  _{arXiv · 2026-01 · arXiv:2601.05293} Survey (Lazer, Aryal, Gupta & Bertino, submitted 8 Jan 2026) bridging agentic AI and cybersecurity: challenges, opportunities, and concrete use-case prototypes (agents both as targets and as defensive tooling), with attention to governance and safety gaps. Teach: Includes use-case prototypes; teachable as applied case studies plus survey reading.

• Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystem
  _{arXiv · 2025-12-09 · arXiv:2512.08290} SoK distinguishing adversarial security threats (indirect prompt injection, tool poisoning) from epistemic safety hazards, analyzing MCP primitives (Resources, Prompts, Tools) and topology/supply-chain risks in open tool registries, and surveying defenses from cryptographic methods to runtime verification. A comprehensive 2025 synthesis of the field. Teach: Unified threat model and taxonomy make an ideal survey reading; conceptual rather than lab-based.

• From Failure Modes to Reliability Awareness in Generative and Agentic AI System
  _{arXiv · 2025-10-24 · arXiv:2511.05511} Preprint by Janet (Jing) Lin and Liangwei Zhang presenting an 11-layer failure stack for generative and agentic AI—from hardware/power foundations to adaptive learning and agentic reasoning—showing how failures cascade across layers. Adds an awareness-mapping maturity framework and integrates Dependability-Centred Asset Management (DCAM). Complements industry taxonomies (Microsoft, OWASP) for reasoning about agent reliability and security failure cascades. Teach: Open-access arXiv preprint with an explicit layered failure model teachable as a lecture framework.

• Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
  _{arXiv (later published in IEEE Access, DOI 10.1109/access.2026.3675554) · 2025-10 · arXiv:2510.23883} Survey by Chhabra, Datta, Nahin, and Mohapatra mapping the agentic AI security space across threats (goal hijacking, tool misuse, memory poisoning, inter-agent communication risks), defenses (technical + governance), and evaluation/benchmarks. Submitted Oct 27 2025; in the target Oct-2025 window. Teach: Public arXiv paper with a threat taxonomy and benchmark/evaluation review; usable as lecture reading and for structuring a threats/defenses module.

• A Survey on Agentic Security: Applications, Threats and Defenses
  _{arXiv · 2025-10 · arXiv:2510.06445} Broad October-2025 survey (Shahriar et al.) categorizing 260+ papers into agent security applications, vulnerabilities/attack surfaces, and defenses; argues agentic systems are structurally fragile by default and require lifecycle-spanning defenses rather than single-layer fixes. Situates runtime guardrails and policy-enforcement frameworks in the defense landscape. Teach: Yes — survey structure and taxonomy make an ideal syllabus backbone / reading-list anchor for an agent-security course module.

• A Look at the New AI Control Frameworks from NIST and CSA
  _{Cloud Security Alliance (CSA) · 2025-09-03} September 2025 CSA blog by Ken Huang comparing NIST's Control Overlays for Securing AI Systems (COSAIS) with CSA's AI Controls Matrix (AICM, 243 controls across 18 domains), arguing they are complementary rather than competing. A concise orientation piece to the 2025 AI-controls wave. Teach: Free blog comparison — a good survey/orientation reading to open a module on AI control frameworks.

• TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
  _{arXiv · 2025-06 · arXiv:2506.04133} A survey adapting the AI TRiSM framework (Explainability, ModelOps, Security, Privacy, Lifecycle Governance) to LLM-based agentic and multi-agent systems. Contributes a risk taxonomy spanning coordination failures to adversarial attacks, two novel metrics (Component Synergy Score, Tool Utilization Efficacy), and a research roadmap for responsible agentic AI. Teach: Public survey (Raza, Sapkota, Karkee, Emmanouilidis); useful as a course-mapping/reading-list backbone for the whole agent-assurance space.

• From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
  _{arXiv · 2025-06 · arXiv:2506.15170} Systematic survey tracing jailbreak attacks and defenses as systems evolve from single-modal LLMs through multimodal models to tool-using agents. Categorizes attacks by impact/visibility and defenses by timing/approach, and highlights gaps in agent-specific vulnerabilities and hybrid attack methods. Organizing reference for jailbreak-for-agents. Teach: Taxonomy suitable for a survey-style lecture on the evolution of jailbreaks into agentic and multimodal settings.

• A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?
  _{arXiv · 2025-05 · arXiv:2505.10924} Chen et al. survey focused on computer-using / GUI agents that operate an OS or browser, defining these agents, categorizing threats, proposing defenses, and reviewing benchmarks. Narrower attack-surface slice of the agentic space. Teach: Domain-specific threat catalog for computer-use agents; pairs with hands-on browser/OS-agent red-team labs.

• A Survey of Agent Interoperability Protocols: MCP, ACP, A2A, and ANP
  _{arXiv · 2025-05 · arXiv:2505.02279} By Ehtesham, Singh, Gupta, and Kumar. Comparative survey of four agent interoperability protocols — Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent (A2A), and Agent Network Protocol (ANP) — covering their interoperability, security, and discovery models, and proposing a phased adoption roadmap. Foundational context for where A2A sits among MCP/ACP/ANP. Teach: Yes — protocol comparison table is directly usable teaching material.

• Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions
  _{arXiv · 2025-03 · arXiv:2503.23278} The first academic analysis of the MCP ecosystem (Hou, Zhao, Wang & Wang, submitted 30 Mar 2025): defines the MCP server lifecycle (creation, deployment, operation, maintenance across 16 activities) and a threat taxonomy across four attacker types. Foundational survey most later MCP-security work builds on. Teach: Widely cited survey with a clear lifecycle/threat framework — excellent lecture scaffold; no labs but a strong conceptual blueprint.

• A Survey on Trustworthy LLM Agents: Threats and Countermeasures
  _{arXiv · 2025-03 · arXiv:2503.09648} Yu et al. foundational early-2025 survey introducing the TrustAgent framework, categorizing trustworthiness into intrinsic (brain, memory, tool) and extrinsic (user, agent, environment) dimensions, and synthesizing attacks, defenses, and evaluation across the agent lifecycle. Widely cited baseline in the 2025-2026 wave. Teach: Public survey; useful as the historical/baseline reading before the late-2025/2026 layered frameworks.

• OWASP GenAI Security Project (umbrella initiative)
  _{OWASP · 2025} Global open-source, community-driven initiative that hosts the LLM Top 10, the Agentic Security Initiative, AI Threat Intelligence & Response, AI Security Governance, red-teaming/evaluation, and data-security workstreams. It is the organizing hub for nearly all OWASP GenAI/agentic security deliverables in 2025. Teach: All deliverables are free public PDFs/web resources; strong backbone for a course syllabus covering the whole GenAI security lifecycle.

---

Threat Models

33 sources.

• Agent Data Injection Attacks are Realistic Threats to AI Agents
  _{arXiv · 2026-07-06 · arXiv:2607.05120} Introduces Agent Data Injection (ADI), a class of indirect prompt injection where attackers embed malicious data disguised as trustworthy information (exploiting metadata/context formatting rather than direct instruction injection). Demonstrates remote code execution and supply-chain attacks against real coding agents (Claude Code, Codex, Gemini CLI) and web agents (Claude in Chrome, Antigravity, Nanobrowser). Strong motivation for containment/isolation controls in autonomous developers. Authors: Woohyuk Choi, Juhee Kim, Taehyun Kang, Jihyeon Jeong, Luyi Xing, Byoungyoung Lee. Teach: Named real-world exploit chains against shipping coding agents make vivid red-team lab scenarios (with responsible-disclosure framing).

• Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry
  _{arXiv · 2026-05-12 · arXiv:2605.11418} Shows semantic supply-chain attacks against AI agent 'skill' registries (SKILL.md), where malicious skill metadata subverts agent behavior — a plugin/skill provenance risk analogous to package-registry poisoning. Teach: Public preprint (Saha, Faghih, Feizi) documenting the attack surface; adaptable to a skill-registry poisoning demonstration.

• When the Agent Is the Adversary: Architectural Requirements for Agentic AI Containment After the April 2026 Frontier Model Escape
  _{arXiv · 2026-04-25 · arXiv:2604.23425} Treats the AI agent itself as a potential adversary rather than a trusted component. Analyzes an April 2026 frontier-model sandbox breach, evaluates four containment approaches and their vulnerabilities, and derives five architectural requirements for safer agentic AI containment (noting no publicly described system satisfies all five). Author: Richard Joseph Mitchell. Teach: Adversary-model framing for containment architecture is a strong conceptual lecture piece.

• SoK: The Attack Surface of Agentic AI -- Tools, and Autonomy
  _{arXiv · 2026-03 · arXiv:2603.22928} Systematization of knowledge (Dehghantanha & Homayoun, submitted 24 Mar 2026) mapping trust boundaries and a taxonomy of attacks spanning prompt injection, knowledge-base/RAG poisoning, tool/plug-in exploits, and multi-agent emergent threats; synthesizes 2023-2025 evidence and standards. Teach: Trust-boundary diagrams and a cross-cutting attack taxonomy; strong SoK reading and a boundary-mapping classroom exercise.

• Model Context Protocol Threat Modeling and Analyzing Vulnerabilities to Prompt Injection with Tool Poisoning
  _{arXiv · 2026-03-23 · arXiv:2603.22489} STRIDE/DREAD threat modeling of MCP across five system components, identifying tool poisoning (malicious instructions embedded in tool metadata) as the most prevalent client-side vulnerability; empirically assesses seven MCP clients and finds weak validation. Proposes a multi-layered defense (static metadata analysis, decision-path tracking, anomaly detection, transparency). A rigorous classical-threat-modeling treatment of MCP. Teach: STRIDE/DREAD application is fully worked; ideal as a threat-modeling worksheet/lab for students.

• SoK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtime
  _{arXiv · 2026-02 · arXiv:2602.19555} SoK taxonomy (Jiang et al., submitted 23 Feb 2026) focused on the agentic supply-chain and runtime layer, distinguishing data supply-chain from tool supply-chain attacks and introducing the self-propagating 'Viral Agent Loop' threat; enumerates attack vectors and matching defenses. Complements layer-based frameworks on the supply-chain axis. Teach: Supply-chain/runtime attack-vs-defense taxonomy; maps well to dependency-poisoning and tool-supply-chain labs.

• MITRE ATLAS (Adversarial Threat Landscape for AI Systems) — agentic AI update
  _{MITRE · 2025-11} Community ATT&CK-style knowledge base of real-world AI attack tactics/techniques and case studies. Late-2025 updates added agent-focused entries such as context/memory poisoning, config tampering, credential harvesting, and exfiltration via tool invocation. Backbone for agent threat modeling and red-team detection engineering. Teach: Free ATT&CK-style matrix with case studies; directly usable for threat-modeling and red-team mapping labs.

• Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain
  _{arXiv · 2025-10-03 · arXiv:2510.05159} Demonstrates backdoors injected into AI-agent components (models, tools, and downstream artifacts) propagating through the agent supply chain, showing how a poisoned upstream dependency compromises agent behavior end-to-end. Central to teaching agent dependency/provenance risk. Teach: Public arXiv preprint (Boisvert, Puri, Drouin et al.) with attack methodology; adaptable into a backdoor-injection + detection lab on agent components.

• ENISA Threat Landscape 2025
  _{ENISA · 2025-10-01} ENISA's annual EU threat landscape (released Oct 1 2025, analyzing 4,875 incidents Jul 2024–Jun 2025; a v1.2 revision followed Jan 2026) with AI now a defining element: AI-supported social engineering, jailbroken models (WormGPT/FraudGPT/Xanthorox), model poisoning, and LLM-driven 'slopsquatting' of hallucinated package names. Teach: Free official PDF; the AI-attack sections make a strong teaching case study on how AI reshapes the adversary side.

• We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems
  _{arXiv · 2025-06-16 · arXiv:2506.13666} Position/analysis paper on third-party (supply-chain) safety risks introduced when agents delegate to external MCP servers not controlled by the LLM developer. Presents an analysis framework, pilot experiments demonstrating real threats, and research directions (red teaming, safe LLM development, safety evaluation, ecosystem safeguards). Frames the MCP third-party trust problem. Teach: Conceptual risk framing; discussion-oriented rather than lab-based.

• Seven Security Challenges in Cross-domain Multi-agent LLM Systems
  _{arXiv · 2025-05 · arXiv:2505.23847} Maps the security agenda for cross-domain multi-agent LLM systems, enumerating seven categories of novel security challenges — including trust, identity, and propagation of adversarial signals across organizational boundaries — with attacks, evaluation metrics, and research directions for each. Useful problem-framing for A2A across trust domains. (Title corrected from 'Seven Security Challenges That Must Be Solved in...' to the actual arXiv title.) Teach: Yes — the seven challenges map cleanly to a syllabus.

• Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents
  _{arXiv · 2025-05 · arXiv:2505.14418} Reveals backdoor vulnerabilities in multimodal-LLM-powered mobile GUI agents via AgentGhost, a framework using composite goal- and interaction-level triggers. Reports attack success rates up to 99.7% with ~1% utility degradation, and proposes a defense reducing attack accuracy to 22.1%. Teach: Backdoor/supply-chain threat case study for GUI agents; teachable as a poisoning/trigger demonstration with a paired defense baseline.

• Taxonomy of Failure Mode in Agentic AI Systems
  _{Microsoft AI Red Team · 2025-04-24} Microsoft AI Red Team whitepaper (lead authors Pete Bryan on the taxonomy and Giorgio Severi on the memory-poisoning case study) categorizing agent failures along two axes: safety vs. security, and novel (agent-unique) vs. existing (inherited). Highlights memory poisoning as a particularly insidious novel mode and provides mitigation strategies; positioned as a living document. Teach: Public whitepaper with a structured taxonomy and a memory-poisoning case study; directly teachable as a threat-modeling framework for agents.

• Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agents
  _{arXiv · 2025-04 · arXiv:2504.19956} Academic threat-model and mitigation framework (Narajala & Narayan, submitted 28 Apr 2025) presenting the ATFAA threat model and SHIELD mitigations; identifies nine primary threats across five domains, aligned with and expanding on the OWASP agentic taxonomy. Teach: Open-access preprint; usable as an assigned reading complementing the OWASP ASI docs.

• OWASP Agentic AI – Threats and Mitigations / Multi-Agentic System Threat Modeling Guide
  _{OWASP GenAI Security Project · 2025-04} OWASP GenAI Security Project resources including the Multi-Agentic System Threat Modeling Guide v1.0 (published 2025-04-23), which extends the OWASP Agentic AI threat taxonomy to real multi-agent systems, plus Agentic AI Threats and Mitigations and the Top 10 for Agentic Applications, enumerating agent threats including tool/plugin and supply-chain compromise. Teach: Public threat taxonomy and guides; directly teachable as an agentic threat-modeling framework.

• Threat Modeling OpenAI's Responses API with the MAESTRO Framework
  _{CSA · 2025-03} A worked CSA blog post by Ken Huang applying MAESTRO — a seven-layer threat modeling approach for agentic AI — to OpenAI's Responses API, systematizing threats from agent-identity compromise to task-integrity failure across the seven MAESTRO layers. Demonstrates how MAESTRO maps to concrete agent API deployments. Teach: Yes — public step-by-step CSA case study usable as a hands-on MAESTRO exercise.

• NIST AI 100-2e2025: Adversarial Machine Learning — A Taxonomy and Terminology of Attacks and Mitigations
  _{NIST · 2025-03} NIST's authoritative taxonomy of adversarial ML attacks and mitigations (NIST AI 100-2 E2025, March 2025), establishing a common vocabulary across the ML system lifecycle and providing threat framing that underpins tool-misuse and prompt-injection-driven tool abuse. Teach: Standards-body taxonomy; anchors a shared vocabulary lecture and threat-model mapping exercise.

• Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2e2025)
  _{NIST · 2025-03-24} NIST's 2025 adversarial-ML taxonomy (AI 100-2e2025), providing standardized concepts and terminology for AML attacks across the AI lifecycle. The edition expands generative-AI coverage including prompt injection (direct and indirect) and agentic risks, giving authoritative standards-body vocabulary for agent injection threats and mitigations. Teach: Yes. Free NIST publication; canonical taxonomy for teaching threat models.

• OWASP Agentic AI — Threats and Mitigations (Agentic Security Initiative)
  _{OWASP GenAI Security Project · 2025-02-17} The first guide in a series from OWASP's Agentic Security Initiative (ASI), providing a threat-model-based reference of emerging agentic threats and their mitigations. Complementary to the OWASP LLM Top 10 and foundational for the ASI series. Teach: Freely downloadable OWASP guide; directly teachable as an open threat-modeling reference.

• OWASP Agentic AI - Threats and Mitigations
  _{OWASP · 2025-02-17} First OWASP Agentic Security Initiative (ASI) guide (v1.0, Feb 2025); a threat-model-based reference of emerging agentic threats that lists Memory Poisoning (T1) alongside mitigations, making it the canonical standards-body reference for agent memory integrity. Teach: Public OWASP document with named threats and mitigations; ideal course backbone.

• Multi-Agent Risks from Advanced AI
  _{arXiv · 2025-02 · arXiv:2502.14143} Cooperative AI Foundation report (Hammond, Chan, et al.; ~50 authors) giving a structured taxonomy of multi-agent risks: three failure modes (miscoordination, conflict, collusion) plus seven risk factors (information asymmetries, network effects, emergent agency, multi-agent security, etc.). Widely cited taxonomy for collusion and cascading risk. Teach: Yes — taxonomy and failure-mode framework are directly teachable.

• MAESTRO: Multi-Agent Environment, Security, Threat, Risk, and Outcome (Agentic AI Threat Modeling Framework)
  _{CSA (Cloud Security Alliance) · 2025-02-06} Ken Huang / CSA's seven-layer threat-modeling framework purpose-built for agentic AI, spanning foundation models through the agent ecosystem, with layer-specific and cross-layer analysis (adversarial ML, data poisoning, goal manipulation, supply-chain compromise, isolation/containment failures, multi-agent risks). Widely adopted lens for structuring agent-security analysis. Teach: Layered framework maps cleanly to a course structure; supports a hands-on threat-modeling workshop against a sample agent stack.

• MAESTRO: Agentic AI Threat Modeling Framework (Cloud Security Alliance)
  _{CSA · 2025-02-06} CSA's seven-layer threat-modeling framework (published 6 Feb 2025) built specifically for agentic AI. Layers: (1) Foundation Models, (2) Data Operations, (3) Agent Frameworks, (4) Deployment & Infrastructure, (5) Evaluation & Observability, (6) Security & Compliance (cross-cutting), and (7) Agent Ecosystem. Acronym expands to Multi-Agent Environment, Security, Threat, Risk, and Outcome. A widely adopted industry layered framework going beyond STRIDE. Teach: Public framework with defined layers and vocabulary; teachable as a hands-on threat-modeling method mapped to CI/CD pipelines.

• MAESTRO: Agentic AI Threat Modeling Framework
  _{CSA · 2025-02-06} CSA's seven-layer threat modeling framework (MAESTRO = Multi-Agent Environment, Security, Threat, Risk, & Outcome) purpose-built for agentic AI where STRIDE/PASTA/LINDDUN fall short. Layers span Foundation Models, Data Operations, Agent Frameworks, Deployment/Infrastructure, Evaluation/Observability, Security & Compliance, and the Agent Ecosystem. Authored by Ken Huang (DistributedApps.ai), published Feb 6, 2025. Teach: Fully public blueprint plus a CSA GitHub repo (CloudSecurityAlliance/MAESTRO) and hands-on Lab Space labs — directly usable for a classroom threat-modeling exercise.

• CSA MAESTRO (Multi-Agent Environment, Security, Threat, Risk, and Outcome) Threat Modeling Framework
  _{CSA · 2025-02} Layered (seven-layer) threat-modeling framework authored by Ken Huang for the Cloud Security Alliance, purpose-built for autonomous, adaptive, multi-agent systems and trust-boundary crossings; complements ATLAS/OWASP for red-team scoping. Widely referenced and paired with ATLAS through 2025–2026. Teach: Public framework with worked orchestrator-compromise scenarios; teachable as a structured agent threat-modeling exercise.

• Agentic AI Threat Modeling Framework: MAESTRO (Multi-Agent Environment, Security, Threat, Risk & Outcome)
  _{Cloud Security Alliance (CSA) · 2025-02-06} A seven-layer, defense-in-depth threat-modeling framework for agentic AI (Foundation Models, Data Operations, Agent Frameworks, Deployment/Infrastructure, Evaluation/Observability, Security & Compliance, Agent Ecosystem) plus cross-layer threats, addressing gaps STRIDE/PASTA miss. Authored by Ken Huang and published via the CSA blog. Teach: Public CSA blog/framework with a clear layered model; highly teachable as a structured threat-modeling lab.

• Agentic AI Threat Modeling Framework: MAESTRO
  _{CSA · 2025-02-06} Cloud Security Alliance framework by Ken Huang. MAESTRO (Multi-Agent Environment, Security, Threat, Risk, and Outcome) is a seven-layer threat-modeling approach for agentic AI that spans the AI lifecycle and includes data-operations poisoning and compromised RAG pipelines among agent threats. Useful structured threat-model lens. Teach: Public CSA blueprint with layered taxonomy; teachable as a threat-modeling method.

• Agentic AI - Threats and Mitigations
  _{OWASP · 2025-02-17} Inaugural v1.0 guide (17 Feb 2025) from the OWASP Agentic Security Initiative (GenAI Security Project) cataloging autonomous-agent threats and defensive strategies via a threat-model framework, including agent execution/containment and tool-use risks. The first in a series and a primary industry reference for agent security curricula. Teach: Free, structured threat catalog with mitigations; ready-made lecture scaffolding and mapping exercise material.

• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
  _{arXiv · 2024-05 · arXiv:2405.06624} Foundational Guaranteed Safe AI position paper (Dalrymple, Skalse, Bengio, Russell, Tegmark, Seshia, et al.) defining the world-model + safety-specification + verifier triad that underpins much of the 2025-2026 formal-methods-for-agents agenda; included for lineage and vocabulary. Submitted May 10, 2024. Teach: Canonical framing paper; ideal opening reading to establish the GS AI vocabulary (world model, spec, verifier) before agent-specific work.

• MITRE ATLAS — Command and Control tactic
  _{MITRE · 2025} MITRE ATLAS does include a Command and Control tactic (confirmed as AML.TA0014 in the published ATLAS data), part of the 2025 wave extending ATLAS toward autonomous-agent attack surfaces. NOTE: the item's specific 'v5.1.0, November 2025' versioning could not be confirmed — the published ATLAS data was at v5.6.0, so the exact version label/date is uncertain, though the C2 tactic itself is real. Teach: Public matrix update; teach as evolution of the AI attack taxonomy over 2025, but confirm the exact version/date against ATLAS release notes.

• MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems)
  _{MITRE · 2025} MITRE's ATT&CK-style knowledge base of adversarial tactics/techniques against AI systems, including supply-chain compromise (poisoned models/data, malicious ML artifacts), providing a shared vocabulary and case studies for AI supply-chain threats. Teach: Public matrix, technique pages and case studies; teachable as a structured AI threat-mapping exercise.

• MITRE ATLAS (Adversarial Threat Landscape for AI Systems) — agent tactics/techniques
  _{MITRE · 2025} Community knowledge base of adversary tactics and techniques against AI/ML systems, structured like ATT&CK. 2025 releases expanded generative and agentic coverage, adding techniques such as RAG poisoning, LLM prompt crafting, and AI supply-chain compromise. Confirmed real; the published ATLAS data reached v5.6.0 and includes a Command and Control tactic (AML.TA0014). Exact tactic/technique counts vary by release. Teach: Free public matrix and technique pages; standard for mapping red-team operations and detection engineering.

• MITRE ATLAS (Adversarial Threat Landscape for AI Systems)
  _{MITRE · 2025} MITRE's ATT&CK-style knowledge base of adversary tactics/techniques against AI systems, cataloguing data/training poisoning and RAG-relevant techniques with real case studies. Standard threat-intel mapping resource. Teach: Public technique matrix and case studies; teachable for mapping attacks to defenses.

---

Defense Frameworks

38 sources.

• Knowledge Base Poisoning Attacks and Defense for Policy-Aware LLM-RAG Framework
  _{arXiv · 2026-07-05 · arXiv:2607.04379} Solanki et al. attack a policy-aware LLM-RAG system (military operations context) by injecting malicious rules into the knowledge base, compromising decisions with as little as 1.6% contamination, then propose CLD-KB, a dual-detector anomaly-detection defense that outperforms five baselines with low overhead. Teach: Public preprint; paired attack (rule injection) + defense (CLD-KB) makes a self-contained module.

• Bulkhead: Automated Semantic Detection and Remediation of Container Escape Vulnerabilities
  _{arXiv · 2026-07-14 · arXiv:2607.12723} Detects and remediates container-escape (path-traversal) vulnerabilities relevant to the container-based sandboxes agents run in, combining LLMs with formal verification: identifies/repairs via multi-dimensional knowledge patterns and validates patches with assertion-driven model-checking templates. Motivated by shared-resource (e.g. GPU) container deployments that weaken filesystem isolation. Teach: Container-escape focus connects agent sandboxing to classic container-security labs.

• Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
  _{arXiv · 2026-05 · arXiv:2605.18672} Position paper arguing that safe LLM-agent deployment structurally requires a contract-based three-layer probabilistic assume-guarantee architecture (semantic compliance, environmental validity, dynamical feasibility) rather than a single guardrail, with probabilistic guarantees flowing through layers; open challenges include bound estimation from non-i.i.d. traces and multi-agent extension. Authors incl. Bensalem, Dong, Huang, et al. (submitted May 18, 2026). Teach: Assume-guarantee reasoning is a standard formal-methods topic; the position framing anchors a lecture on compositional agent safety.

• Cordon-MAS: Defending RAG against Knowledge Poisoning via Information-Flow Control
  _{arXiv · 2026-05-26 · arXiv:2605.26754} Identifies a 'monitoring-control gap' (models spot contradictions in retrieved evidence but still follow malicious instructions) and applies the Cordon Principle: separate extraction, auditing, and synthesis into privilege-restricted agents. Reports a 92.4% relative reduction in attack success across five datasets. Teach: Public preprint; information-flow-control / privilege separation approach teachable as a concrete RAG defense architecture.

• MCP-DPT: A Defense-Placement Taxonomy and Coverage Analysis for Model Context Protocol Security
  _{arXiv · 2026-04 · arXiv:2604.07551} Introduces a defense-placement taxonomy (Rostamzadeh et al., submitted 8 Apr 2026) categorizing where defenses sit across six architectural layers of the MCP stack, with a coverage analysis exposing protection gaps. Matters as a structuring lens for MCP defense architecture and defense-in-depth positioning. Teach: Taxonomy and coverage analysis make a strong design/planning exercise for defense placement.

• Agentic MCP Security Best Practices Guide (v1)
  _{CSA (Cloud Security Alliance) · 2026-03-27} CSA best-practices guide (labeled draft white paper, 2026-03-27) organizing the MCP threat landscape into categories with security requirements, a four-level MCP Security Maturity Model, and cross-reference mappings to OWASP, CSA, and MITRE ATLAS controls. A standards-body practitioner framework. Teach: Public spec with maturity model and control cross-references — directly usable as a governance/controls curriculum module. Note: page is a draft, so cite as work-in-progress.

• SMCP: Secure Model Context Protocol
  _{arXiv · 2026-02-01 · arXiv:2602.01129} Proposes SMCP, a hardened variant of MCP incorporating identity management, mutual authentication, security context propagation, policy enforcement, and audit logging to mitigate injection, poisoning, and transport-level attacks. A protocol-redesign (secure-by-design) approach rather than a bolt-on defense. Teach: Protocol design documented; teachable as a secure-by-design protocol case study.

• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
  _{arXiv · 2026-01 · arXiv:2601.09923} Extends the CaMeL capabilities/control-flow approach to computer-use (GUI) agents via NOVA, a single-shot planner that emits an upfront branching plan covering anticipated runtime states so untrusted screen content cannot inject new instructions (retaining up to 57% of frontier performance). Identifies a residual 'Branch Steering' threat requiring added defenses. Part of the 2025-2026 CaMeL-lineage defense wave. Teach: Partial. arXiv follow-up; teachable alongside the original CaMeL as an extension case study.

• The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents
  _{arXiv · 2025-11-05 · arXiv:2511.03690} Production software-agent SDK that natively integrates sandboxed execution, lifecycle control, local-to-remote execution portability, model-agnostic multi-LLM routing, and built-in security analysis. A late-2025 open-source reference architecture (from the OpenHands team, Xingyao Wang et al., Graham Neubig) for safely running autonomous developer agents; production data shows reduced system failures across models and benchmarks. Teach: Open-source SDK with a public sandboxed-execution runtime that students can deploy and inspect directly.

• The CISOs' Guide to Agentic AI (Autonomous Security Operations for Defensive Parity)
  _{Palo Alto Networks (vendor) · 2025-11} Palo Alto Networks guidance for security leaders. The blog post 'Agentic AI Security: What Business Leaders Can't Afford To Ignore' (Kaitlin Harvey, Nov 13 2025) outlines securing AI agents via discovery/visibility, treating agents as privileged identities, and extending identity programs, and points to the downloadable companion resource 'Autonomous Security Operations: The CISOs' Guide to Agentic AI for Defensive Parity.' Teach: Public vendor guide; usable as a CISO-level governance/deployment reference rather than a hands-on lab.

• Cross-LLM Generalization of Behavioral Backdoor Detection in AI Agent Supply Chains
  _{arXiv · 2025-11-25 · arXiv:2511.19874} Proposes behavioral detection of backdoors in AI agent supply chains that generalizes across different underlying LLMs, targeting the provenance/integrity gap when agents pull untrusted models. Directly on agent dependency-risk defense. Teach: Public preprint (Arun Chowdary Sanna) describing a detector; suitable as a defensive-labs blueprint for backdoor screening.

• Building Browser Agents: Architecture, Security, and Practical Solutions
  _{arXiv · 2025-11 · arXiv:2511.19477} Practitioner-oriented treatment of browser-agent architecture and its security considerations by Aram Vardanyan, arguing architectural choices (not model capability) determine agent success. Highlights prompt-injection vulnerabilities and recommends specialized tools with programmatic safety constraints over LLM-based reasoning; reports ~85% on the WebGames benchmark vs ~50% for prior agents. Teach: Architecture + security patterns suitable for a design-and-harden workshop; contrasts programmatic safety constraints vs LLM-reasoning-based protection.

• BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents
  _{arXiv · 2025-11 · arXiv:2511.20597} Introduces a benchmark of realistic webpage-hidden injection payloads that influence real-world actions, evaluates existing defenses across frontier models, and proposes a multi-layered BrowseSafe defense combining architectural and model-based mitigations for AI browser agents. Teach: Paired benchmark + defense; can teach both attack construction and a concrete multi-layered mitigation pipeline.

• OWASP GenAI Threat Defense COMPASS 1.0 (and RunBook)
  _{OWASP GenAI Security Project · 2025-09-10} A unified AI Threat Resilience Strategy dashboard plus runbook that consolidates AI threats, vulnerabilities, defenses, and mitigations, helping teams map threats to defenses and measure resilience posture across GenAI/agentic systems. Confirmed on the OWASP GenAI resources page as 'Threat Defense COMPASS 1.0', dated Sep 10, 2025. Teach: Public dashboard/runbook; teachable as a defense-mapping and posture-assessment tool.

• A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory
  _{arXiv · 2025-09-29 · arXiv:2510.02373} Proposes A-MemGuard, a proactive defense that guards LLM agent memory against poisoning/injection via consensus validation and a dual-memory structure with integrity checks on stored experiences. One of the clearest 2025 memory-integrity defense blueprints; authors include Thorsten Holz and XiaoFeng Wang. Teach: Public defense framework with described architecture; teachable as a hands-on mitigation lab.

• SP 800-53 Control Overlays for Securing AI Systems (COSAiS) — Concept Paper
  _{NIST · 2025-08-14} NIST CSRC project (concept paper released Aug 14, 2025) developing SP 800-53 control overlays for five AI use cases, two of which are explicitly agentic: single-agent and multi-agent AI systems. Directly maps federal security controls onto agentic deployments. Teach: Public concept paper plus an annotated predictive-AI outline (Jan 2026) and open community; teachable as hands-on control-mapping once agent overlays draft, though agent overlays are still in development.

• MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI
  _{arXiv · 2025-08-14 · arXiv:2508.10991} A multi-stage, defense-in-depth framework for securing MCP integrity: lightweight static scanning for overt threats, a deep neural detector for semantic attacks, and an E5-based detector reaching ~96% accuracy. Also introduces MCP-ATTACKBENCH, a 70,448-sample dataset for evaluating defenses. A reference layered-defense architecture for MCP. Teach: Multi-stage architecture plus a large attack dataset — a good blueprint for building/evaluating a defense pipeline in class. (Title corrected to the paper's actual title.)

• IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents
  _{arXiv · 2025-08 · arXiv:2508.15310} Models an agent's task execution as a traversal over a pre-planned Tool Dependency Graph (TDG), separating action planning from external data interaction so that untrusted data returned mid-execution cannot spawn new attacker-directed tool calls. A control-flow/data-flow separation defense against indirect prompt injection, evaluated on AgentDojo. Teach: Partial. arXiv paper with method detail; reproducibility depends on released code.

• ToolRegistry: A Protocol-Agnostic Tool Management Library for Function-Calling LLMs
  _{arXiv · 2025-07-11 · arXiv:2507.10593} Tool-management library (Ding & Stevens) that treats tool calls as RPCs, with a single Tool object acting as a universal stub across transports (MCP, OpenAPI). Adds tag-based permission policies, BM25F progressive tool disclosure, and multi-provider schema compatibility (OpenAI/Anthropic/Gemini), cutting integration code 60-80%. Teach: Open library with tag-based permission policies and progressive disclosure; usable directly in a build-a-secure-tool-registry lab.

• Securing Agentic Applications Guide 1.0
  _{OWASP · 2025-07-27} Lifecycle-based guidance (released 27 July 2025) for designing, building, deploying, and operating secure agentic LLM applications — covers secure design/threat modeling, secure build/deployment (SAST/DAST, dependency scanning, hardening), and secure operations/runtime (monitoring, anomaly detection, incident response). Teach: Public PDF; a practical blueprint for a hands-on secure-agent development module.

• CSA AI Controls Matrix (AICM) v1.x — A Framework for Trustworthy AI
  _{CSA · 2025-07-10} Vendor-agnostic control framework with 247 control objectives across 18 security domains for cloud-based/agentic AI systems. AICM v1.0 launched July 2025; v1.1 (the linked artifact) refreshed 2026 with 247 controls. Builds on CSA's Cloud Controls Matrix and aligns with ISO 42001, ISO 27001 and BSI AIC4. Ships with a CAIQ-for-AI assessment questionnaire and underpins the STAR for AI self-assessment program. Teach: Downloadable spreadsheet/artifact with the full control set and CAIQ-for-AI questionnaire — directly assignable as a controls-mapping lab.

• Design Patterns for Securing LLM Agents against Prompt Injections
  _{arXiv (Beurer-Kellner, Buesser, Debenedetti, Fabian, Fischer, Grosse, Paverd, Tramer et al.) · 2025-06-10 · arXiv:2506.08837} Catalogs principled design patterns (Action-Selector, Plan-Then-Execute, LLM Map-Reduce, Dual LLM, Code-Then-Execute, Context-Minimization) that constrain agent architecture so untrusted data cannot redirect control flow, with case studies, trading generality for provable resistance to prompt injection. Teach: Yes. Companion runnable code samples at github.com/ReversecLabs/design-patterns-for-securing-llm-agents-code-samples make it a strong blueprint/lab basis.

• LLM Agents Should Employ Security Principles
  _{arXiv · 2025-05 · arXiv:2505.24019} Position paper arguing classical information-security principles (least privilege, complete mediation, defense-in-depth, psychological acceptability) should be systematically applied to LLM agent design. Introduces AgentSandbox, a framework embedding these principles across the agent lifecycle, evaluated on benign utility, attack utility, and attack success rate to reduce privacy risk while preserving functionality. Teach: Yes. Conceptual security-principles mapping that translates cleanly into course material.

• The Dual LLM Pattern for building AI assistants that can resist prompt injection
  _{Industry / Simon Willison · 2025-04-11} Note: the linked page's actual title is 'CaMeL offers a promising new direction for mitigating prompt injection attacks' (11 Apr 2025), which discusses at length the originating Dual LLM design pattern Willison first proposed in April 2023: a privileged LLM that plans and calls tools but never sees untrusted content, plus a quarantined tool-less LLM that processes untrusted text and returns only symbolic variables. It is the conceptual ancestor of CaMeL and the Plan-Then-Execute / Dual-LLM patterns; the post also notes limitations of the original design that CaMeL fixes. Teach: Yes. Conceptual, well-documented pattern; easy to whiteboard and prototype in class.

• Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies
  _{arXiv · 2025-04-11 · arXiv:2504.08623} Narajala and Habler present practical security patterns and mitigation strategies for enterprise MCP deployments, addressing tool poisoning and related threats with implementable controls. Teach: Design-pattern paper; a blueprint for a lecture on enterprise MCP hardening.

• ACE: A Security Architecture for LLM-Integrated App Systems
  _{arXiv · 2025-04-29 · arXiv:2504.20984} Li et al. propose a secure architecture for LLM-integrated app/tool systems that divides planning into abstract and concrete phases and enforces data barriers between untrusted content and tool actions, mitigating tool-misuse and injection-driven tool abuse. Teach: Architectural blueprint (abstract/concrete plan separation, data barriers) teachable as a defense-design pattern.

• Defeating Prompt Injections by Design (CaMeL)
  _{arXiv (Google DeepMind) · 2025-03-24 · arXiv:2503.18813} Introduces CaMeL, a defense that explicitly extracts control and data flows from the trusted query so untrusted data can never affect program flow: a privileged LLM builds an execution plan from the trusted query while a quarantined LLM handles untrusted data, and a custom interpreter tracks provenance and enforces capability-based security policies before each tool call. Authors include Debenedetti, Shumailov, Carlini, and Tramer. Achieves provable security on 77% of AgentDojo tasks (vs 84% for undefended agents). Teach: Yes. Public code at github.com/google-research/camel-prompt-injection; evaluated on the open AgentDojo benchmark, so labs and reproduction exercises are feasible in class.

• IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems
  _{arXiv / NDSS 2025 · 2025-01-30 · arXiv:2403.04960} Execution-isolation architecture that runs untrusted third-party LLM apps in separated environments with mediated communication, defending against multiple attack vectors with under 30% overhead for most queries. Foundational containment design for agent/app ecosystems (NDSS 2025). Authors: Yuhao Wu, Franziska Roesner, Tadayoshi Kohno, Ning Zhang, Umar Iqbal (submitted 2024-03-08, revised 2025-01-30). Teach: Peer-reviewed isolation architecture with an implementation; strong anchor for teaching hub-and-spoke agent isolation patterns.

• Defending Against Indirect Prompt Injection Attacks With Spotlighting
  _{arXiv (Microsoft) · 2024-03-20 · arXiv:2403.14720} Defines the Spotlighting family of prompt-engineering defenses (delimiting, datamarking, encoding) that mark or transform untrusted input so the model can reliably signal its provenance and distinguish trusted instructions from injected data, cutting indirect-injection success from >50% to <2% with minimal task-performance impact. Authors: Hines, Lopez, Hall, Zarfati, Zunger, Kiciman. Now shipped in Microsoft Prompt Shields / Azure AI Foundry. Teach: Yes. Open-source implementation at github.com/realArcherL/spotlighting-datamarking; simple enough to reproduce datamarking/encoding in a class exercise.

• ENISA Multilayer Framework for Good Cybersecurity Practices for AI (FAICP)
  _{ENISA · 2023-06-07} ENISA's scalable three-layer AI security architecture (published June 7 2023): Layer I ICT foundations (aligned to ISO 27002/NIS2), Layer II AI/ML-lifecycle threats (evasion, poisoning, model/data disclosure) and trustworthiness properties, Layer III sector-tailored guidance (energy, health, automotive, telecom) referencing the EU AI Act. Teach: Free official PDF with a companion FAICP framework site — teachable as a layered defense model for AI lifecycle security.

• NIST AI Risk Management Framework: Agentic AI Profile (CSA community draft)
  _{CSA · 2026} Cloud Security Alliance community/labs draft building an Agentic AI Profile on top of the NIST AI RMF 1.0, extending RMF functions and adding agentic-specific governance categories (e.g., AG-GV, AG-MP) for autonomous/multi-agent systems. A working community draft (not yet official CSA guidance) that illustrates how NIST frameworks are being operationalized for agentic security. Teach: Public CSA draft mapping RMF subcategories to agentic risks; teachable as an applied extension exercise of the NIST AI RMF, with the caveat that it is a community draft rather than ratified guidance.

• Beyond Detection: Autonomous Anomaly Remediation for MCP Against Tool Poisoning Attacks
  _{ACM (WWW 2026 companion) · 2026} Peer-reviewed system that goes beyond detecting MCP tool poisoning to autonomously remediating anomalies at runtime. Matters as a self-healing/remediation approach and a venue-reviewed contribution. DOI 10.1145/3774904.3792400 is registered and resolves to the ACM Digital Library. Teach: Published system with a remediation pipeline; conceptually teachable, peer-reviewed venue lends credibility.

• SANS SEC598: AI and Security Automation for Red, Blue, and Purple Teams (GIAC AI Security Automation Engineer, GASAE)
  _{SANS / GIAC · 2025} SANS course validating practical automation and AI across offensive, defensive, and cloud security operations, mapped to the GIAC AI Security Automation Engineer (GASAE) certification. Confirmed on giac.org: GASAE's affiliated training is SEC598, and it uses CyberLive hands-on testing. Covers agentic automation of SecOps workflows. Teach: CyberLive hands-on exam and lab-based curriculum; GASAE objectives public and teachable.

• SANS SEC411: AI Security Principles and Practices: GenAI and LLM Defense
  _{SANS · 2025} New SANS foundations course on GenAI/LLM defense principles. Confirmed listed on the SANS AI curriculum page (sans.org/mlp/ai) as a New, AI-Focused, Intermediate-level course. Serves as an introductory tier of SANS's AI security curriculum for practitioners entering the field. Teach: Self-paced with labs and a public outline; good baseline module for an introductory class.

• SANS AI Security Maturity Model
  _{SANS · 2025} SANS-published AI security maturity model (free eBook) providing a 5-stage assessment framework across three dimensions — Protect, Utilize, Govern — with defined controls and actions at each level. Explicitly aligned to NIST AI RMF, the EU AI Act, ISO 42001, and OWASP. Companion to SANS's 2025-2026 AI security course line for benchmarking organizational AI security posture. Teach: Free downloadable model with tiered (5-stage) maturity levels across Protect/Utilize/Govern; usable as a classroom self-assessment/framework exercise.

• LLM Prompt Injection Prevention Cheat Sheet
  _{OWASP Cheat Sheet Series · 2025} OWASP practitioner cheat sheet enumerating concrete prompt-injection mitigations: input validation/sanitization, structured prompts separating instructions from data, output monitoring, human-in-the-loop approval, least-privilege tool-call validation, and guardrail-LLM filters as one layer of defense-in-depth. Includes OpenAI/LangChain code examples. Teach: Yes. Free, concise, hands-on control list ideal for lab checklists.

• Google Secure AI Framework (SAIF) 2.0 — Secure Agents
  _{Google (vendor) · 2025} Google's SAIF (saif.google) with its 2.0 emphasis on 'Secure Agents — building powerful agents users can trust.' Covers security across four component areas (Data, Infrastructure, Model, Application) for both model creators and model consumers, and provides an interactive Risk Map linking risks to controls plus a Risk Self-Assessment tool. SAIF resources are also shared with the Coalition for Secure AI (CoSAI). Teach: Public interactive framework with a Risk Map and Risk Self-Assessment questionnaire at saif.google; usable directly as a classroom framework/lab exercise.

• Databricks AI Security Framework (DASF)
  _{Databricks (vendor) · 2025} Databricks' actionable, end-to-end AI security framework (DASF) that enumerates security risks across AI system components, maps them to common AI security frameworks, and gives concrete platform-agnostic controls via a four-step process to build an organization's AI/agent deployment risk profile. Teach: Downloadable whitepaper plus a companion controls spreadsheet; teachable as a controls-mapping exercise.

---

Guardrails & Runtime

18 sources.

• Sandlock: Confining AI Agent Code with Unprivileged Linux Primitives
  _{arXiv · 2026-05-25 · arXiv:2605.26298} By Cong Wang and Yusheng Zheng. A lightweight, rootless Linux process sandbox that confines LLM-agent-generated shell commands, third-party scripts, and unknown tool plugins by enforcing filesystem, network, IPC, and syscall policies without cgroups, container images, or mandatory namespaces (~5ms startup overhead; Redis at bare-metal speed). It separates static, input-independent policy compiled into kernel-enforced rules from runtime decisions handled by a narrow supervisor, supporting dynamic network/HTTP-level access control and safe argument inspection. Directly targets the 'agents run untrusted code on developer machines' problem; implementation is on GitHub. Teach: Concrete kernel-primitive design (seccomp/Landlock-style split of static policy vs. runtime supervisor) that could seed a hands-on 'build an agent sandbox' lab.

• Provably Secure Agent Guardrail
  _{arXiv · 2026-05 · arXiv:2605.29251} Proposes the Proof-Constrained Action (ePCA) framework: agents must formalize intentions into first-order logical constraints before performing operations, targeting provable security guarantees rather than empirical semantic patching. By Benlong Wu, Weiming Zhang, Kejiang Chen, Han Fang, Nenghai Yu (submitted May 28, 2026). Teach: Public paper articulating provable-guardrail requirements; useful as a critique-and-compare reading against empirical guardrails.

• AIRGuard: Guarding Agent Actions with Runtime Authority Control
  _{arXiv · 2026-05 · arXiv:2605.28914} Runtime guard that operationalizes least privilege as action-time authorization for tool-using agents, addressing 'authority confusion' (untrusted resources influencing reasoning but not authorizing actions). Normalizes tool calls, derives task-level permissions into step-level controls, and enforces authorization before execution. Authors: Qin, Zhuang, Zhou, Han, Zhang (submitted May 27, 2026). Teach: Partial — control/data-flow separation and intent-attribution design are strong teachable concepts; release status unconfirmed.

• Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agents
  _{arXiv · 2026-03 · arXiv:2603.20953} Proposes deterministic pre-action authorization via the Open Agent Passport (OAP) spec: intercepts tool calls synchronously before execution, evaluates them against a declarative policy, and produces a cryptographically signed audit record. Reports 0% social-engineering success under a restrictive policy vs 74.6% under a permissive baseline. Author: Uchi Uchibeke (submitted Mar 21, 2026). Teach: Yes — deterministic pre-action authorization is a clean, implementable pattern for a lab on building an action-gating interceptor around a tool-calling agent.

• MCP-SandboxScan: WASM-based Secure Execution and Runtime Analysis for MCP Tools
  _{arXiv · 2026-01 · arXiv:2601.01241} A WebAssembly/WASI-based sandboxing and runtime-analysis framework (internally called SandScope/SandScope) for executing MCP tools in isolation, combining runtime witness detection with semantic tool profiling to find confused-deputy vulnerabilities and track source-to-sink data flows. Evaluated on 100 repositories, surfacing security-sensitive capabilities across hundreds of tools. Matters as an execution-isolation control for untrusted MCP tools. Teach: Sandboxing architecture described; WASM/WASI isolation plus data-flow auditing is implementable as an advanced runtime-security lab.

• INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systems
  _{arXiv · 2026-01 · arXiv:2601.14667} Defense (by Zhou, Lu, Liu, Yan, Shao) that treats multi-agent compromise as an infection process, explicitly identifying infected/compromised benign agents as a distinct threat category. Evaluated against prompt injection, tool attacks (InjecAgent) and memory attacks (PoisonRAG). Part of the 2026 wave on defending attack propagation. Teach: Yes — evaluated on public datasets (InjecAgent, PoisonRAG); defense is reproducible.

• ceLLMate: Sandboxing Browser AI Agents
  _{arXiv · 2025-12 · arXiv:2512.12594} Proposes ceLLMate, a sandboxing framework (deployed as a browser extension) that confines browser AI agents to limit the blast radius of prompt injection and data exfiltration. Key insight: sandbox at the HTTP layer since all side-effecting UI operations produce network calls; reports only 7.25-15% latency overhead. Authors include Ilia Shumailov and Earlence Fernandes. Teach: Runtime confinement design; can anchor a lab on sandboxing/isolation controls for browser agents, especially HTTP-layer interposition.

• Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents
  _{arXiv · 2025-09 · arXiv:2509.23994} CORRECTED TITLE (was 'The AI Agent Code of Conduct: Automated Guardrail Policy-as-Prompt Synthesis'). Framework that transforms unstructured governance/design artifacts (PRDs, TDDs, code, regulatory docs) into verifiable runtime guardrails expressed as prompts ('Policy-as-Prompt'), reducing manual rule authoring while emphasizing audit trails, human oversight, and continuous compliance. Authors: Gauri Kholkar, Ratinder Ahuja (submitted Sept 28, 2025). Teach: Partial — the policy-as-prompt synthesis idea is a good teachable pattern for auto-generating guardrails; check paper for released prompts/code.

• PSG-Agent: Personality-Aware Safety Guardrail for LLM-based Agents
  _{arXiv · 2025-09 · arXiv:2509.23614} Safety guardrail for LLM agents that builds personalized guardrails by mining interaction history for stable persona/personality traits and applies continuous monitoring across the agent pipeline, tailoring runtime intervention to the agent's role (e.g., healthcare, finance). Confirmed multi-author paper (Wu, Guo, Li, Zou, et al., incl. Philip S. Yu), submitted Sept 28, 2025. Teach: Partial — concept of context/persona-conditioned guardrails is teachable; implementation availability unconfirmed.

• MindGuard: Intrinsic Decision Inspection for Securing LLM Agents Against Metadata Poisoning
  _{arXiv · 2025-08-28 · arXiv:2508.20412} A runtime defense that detects and attributes MCP tool poisoning using a Decision Dependence Graph built from LLM attention/decision signals rather than execution monitoring. Reports 94-99% precision in detecting poisoned invocations and 95-100% attribution accuracy with low overhead. A provenance/attribution approach to catching poisoned tool metadata at decision time. Teach: Detection technique described in detail; graph-based decision-inspection approach implementable as an advanced lab/project. (Title corrected from earlier phrasing to the paper's actual title.)

• Automated Reasoning checks is now available in Amazon Bedrock Guardrails
  _{vendor (AWS) · 2025-08} Production formal-verification guardrail that encodes policies into mathematical logic and uses automated reasoning (SMT-style solving) to validate whether model/agent outputs comply with defined rules, providing mathematically rigorous guarantees and provable hallucination detection; reached GA on August 6, 2025 across six AWS regions. Teach: Hosted product with public docs and a policy-authoring console; teachable as a real-world provable-compliance demo without building a solver.

• How Microsoft defends against indirect prompt injection attacks
  _{Microsoft (MSRC) · 2025-07-29} Microsoft's layered production defense stack against indirect prompt injection (by Andrew Paverd): prevention via hardened system prompts and Spotlighting, detection via Prompt Shields classifiers, and impact mitigation via data governance and user consent workflows. Documents what an industry-scale guardrail runtime actually deploys versus research prototypes. Teach: Partial. Describes production controls (Prompt Shields, Spotlighting) usable as an architectural case study; underlying classifiers are proprietary.

• LlamaFirewall: An open source guardrail system for building secure AI agents
  _{arXiv / Meta AI (PurpleLlama) · 2025-05 · arXiv:2505.03574} Meta's open-source, modular real-time guardrail framework for LLM agents combining PromptGuard 2 (jailbreak detection), Agent Alignment Checks (chain-of-thought auditor for goal hijacking / prompt injection), and CodeShield (online static analysis of generated code). Reference agent-runtime firewall of the 2025 wave. Authors include Sahana Chennabasappa, Cyrus Nikolaidis, David Molnar, Joshua Saxe et al. Teach: Yes — open-source code, docs site (meta-llama.github.io/PurpleLlama/LlamaFirewall), pluggable scanner architecture; directly usable for hands-on labs on layered agent guardrails.

• AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents
  _{arXiv (ICSE 2026) — SMU — Wang, Poskitt, Sun · 2025-03 · arXiv:2503.18666} A domain-specific language for runtime constraints on LLM agents using trigger/predicate/enforcement rules. Prevents unsafe executions in >90% of code-agent cases, eliminates hazardous embodied-agent actions, and enforces 100% AV legal compliance with millisecond overhead; also evaluates LLM-generated rules (o1: 95.56% precision / 70.96% recall for embodied agents). Accepted to ICSE 2026. Teach: Yes — the trigger-predicate-enforcement DSL is a clean teachable model for writing runtime safety rules; paper includes LLM-generated rule evaluation for hands-on policy authoring.

• AgentGuard: Repurposing Agentic Orchestrator for Safety Evaluation of Tool Orchestration
  _{arXiv · 2025-02-13 · arXiv:2502.09809} Framework (Chen & Cong) that autonomously discovers unsafe tool-use workflows and generates safety constraints via a four-phase process — identifying unsafe workflows, validating them through real execution, generating constraints, and testing their effectiveness. Teach: Runtime safety-evaluation framework; a lab on auto-discovering unsafe tool chains and validating generated safety constraints.

• Cisco AI Defense
  _{Cisco (vendor) · 2025-01} Cisco's AI Defense product (announced Jan 2025) provides runtime discovery/visibility, validation and algorithmic red-teaming, and guardrail enforcement for AI applications and increasingly agentic workflows, built partly on Robust Intelligence technology. Positioned as enterprise runtime protection for models, apps, and agents. Teach: Commercial product; framework/positioning is public but hands-on labs require licensed access.

• R^2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning
  _{arXiv — Mintong Kang, Bo Li · 2024-07 · arXiv:2407.05557} Guardrail model that combines data-driven category-specific detectors with explicit safety-category knowledge encoded as logical rules, using probabilistic graphical models (Markov logic networks / probabilistic circuits) for inference to capture inter-category relationships. Introduces the TwinSafety benchmark; reports large gains over LlamaGuard on ToxicChat and against jailbreaks. Frequently cited alongside GuardAgent as a reasoning-based guardrail baseline. Teach: Yes — logical/knowledge-graph (Markov-logic/probabilistic-circuit) reasoning is a distinctive teachable contrast to pure LLM-classifier guardrails.

• GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning
  _{arXiv (ICML 2025) — Xiang, Zheng, Li, ... Dawn Song, Bo Li · 2024-06 · arXiv:2406.09187} First LLM-agent-as-guardrail framework: a guard agent analyzes safety requirements into a task plan, then generates guardrail code that is executed to moderate a target agent's actions, using memory-retrieved demonstrations. Introduces EICU-AC (healthcare access control) and Mind2Web-SC (web safety) benchmarks (>98% / 83% guardrail accuracy). Accepted to ICML 2025; foundational precursor to the late-2025 wave. Teach: Yes — public paper plus two released benchmarks (EICU-AC, Mind2Web-SC) usable as lab datasets for access-control / safety-policy enforcement assignments.

---

Benchmarks & Evaluation

25 sources.

• RIFT-Bench: Dynamic Red-teaming for Agentic AI Systems
  _{arXiv · 2026-06 · arXiv:2606.23927} Framework and benchmark for evaluating security vulnerabilities in agentic AI systems via two automated phases — Discovery (extracts system structure) and Scanning (deploys adaptive adversarial attacks) — tested across 45 agentic systems and supporting direct evaluation of mitigations. Extends static benchmarks toward adaptive, dynamic red-teaming. Teach: Benchmark/method that can illustrate dynamic (non-static) red-teaming in an advanced lab.

• HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark
  _{arXiv · 2026-04 · arXiv:2604.13954} Benchmark of 629 agent trajectories (523 risky, 106 safe; ~33 steps avg) for intrinsic, non-adversarial long-horizon risk: supports risk detection, risk-step localization, and failure-type identification. LLMs do well at trajectory-level detection but drop below 35 Strict-F1 on step localization, framing intrinsic risk auditing as open. Teach: Trajectory dataset with a five-constraint taxonomy; suits long-horizon safety-auditing labs (spec on arXiv).

• ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
  _{arXiv · 2026-04-02 · arXiv:2604.02022} Diverse, realistic trajectory benchmark for long-horizon agent safety evaluation and failure diagnosis. Contains 1,000 trajectories (503 safe, 497 unsafe) averaging ~9 turns each with nearly 2,000 tool invocations, organized by a three-dimensional taxonomy (risk source, failure mode, real-world harm). Part of the 2026 wave of trajectory-level agent-safety benchmarks. Teach: Trajectory dataset (1,000 labeled safe/unsafe trajectories) for safety diagnosis; suitable for trajectory-analysis assignments and long-horizon failure-mode labeling exercises.

• AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks
  _{arXiv · 2026-02 · arXiv:2602.16901} First benchmark dedicated to adaptive, long-horizon attacks exploiting extended user-agent-environment interaction, with a unified extensible attack framework. Supports five attack types (intent hijacking, tool chaining, task injection, objective drifting, memory poisoning) across 28 realistic agentic environments and 644 security test cases, showing single-turn defenses are inadequate. Teach: Unified extensible long-horizon attack framework; teachable if code released (spec on arXiv).

• AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?
  _{arXiv · 2026-02 · arXiv:2602.03117} A dynamic, open-ended benchmark (60 open-ended tasks, 560 injection test cases across Shopping, GitHub, and Daily Life) for evaluating prompt-injection defenses of real-world agent security systems, addressing limits of static benchmarks. Testing ten SOTA defenses finds almost all are either insufficiently secure or suffer significant over-defense. Teach: Benchmark usable to show why dynamic, open-ended evaluation matters beyond fixed test suites, and to expose the security/over-defense tradeoff.

• Evaluating Implicit Regulatory Compliance in LLM Tool Invocation via Logic-Guided Synthesis (LogiSafetyGen)
  _{arXiv · 2026-01 · arXiv:2601.08196} Presents LogiSafetyGen, which converts regulatory requirements into formal logic specifications and builds the LogiSafetyBench benchmark (240 tasks) to evaluate whether LLMs follow both functional and safety constraints during code/tool generation. By Da Song, Yuheng Huang, Boqi Chen, Tianshuo Cong, Randy Goebel, Lei Ma, Foutse Khomh (submitted January 13, 2026). Note: the official arXiv title omits the '(LogiSafetyGen)' parenthetical, which is the framework name. Teach: Benchmark/harness with a logic-guided trace generator; usable as a graded evaluation exercise for agent-compliance methods.

• TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems
  _{arXiv · 2025-11 · arXiv:2511.05269} Benchmark (by Kavathekar et al.) covering multiple attack vectors (impersonation, direct/indirect prompt injection, contradicting agents, Byzantine agent, colluding agents) across AutoGen and CrewAI configs and multiple backbone LLMs, with an Effective Robustness Score balancing safety and task performance; finds multi-agent systems highly vulnerable. Directly covers collusion and propagation. Teach: Yes — public benchmark with framework configs, ideal for graded labs.

• SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents
  _{arXiv · 2025-10 · arXiv:2510.10073} First holistic benchmark for LVLM-based web-agent security, spanning six simulated web environments, 2,970 trajectories, and a structured taxonomy of six attack vectors across user-level and environment-level manipulations (including pop-up attacks and prompt injection); evaluates nine VLMs and finds all consistently vulnerable to subtle adversarial manipulation. Teach: Unified attack-vector benchmark with six environments; supports structured labs on why/how web-agent security failures occur.

• PACEbench: A Framework for Evaluating Practical AI Cyber-Exploitation Capabilities
  _{arXiv · 2025-10-13 · arXiv:2510.11688} Practical AI cyber-exploitation benchmark built on realistic vulnerability difficulty, environmental complexity, and cyber defenses, with four scenarios (single, blended, chained, and defense-in-place exploitations) and an accompanying PACEagent emulating human penetration testers. Evaluation of seven leading LLMs found they struggle with complex exploitation and cannot bypass defenses, indicating no generalized cyber threat yet. Teach: Scenario-based cyber-exploitation tasks; teachable pentest-style agent lab. Note: original title/date were wrong — corrected to the real arXiv paper (2510.11688, Oct 2025).

• MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents
  _{arXiv · 2025-10-14 · arXiv:2510.15994} End-to-end evaluation suite measuring how well LLM agents resist MCP-specific attacks across the full tool-use pipeline, executing real tools via MCP rather than simulations. Covers 12 attack categories over 405 tools and ~2,000 attack instances, and introduces a Net Resilient Performance (NRP) metric balancing security and usability. Finds stronger models are often more vulnerable due to better instruction-following. Teach: Structured attack taxonomy and ~2,000-instance suite suitable for an attack-catalog lab on tool-use pipelines.

• Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks?
  _{arXiv · 2025-10 · arXiv:2510.05244} Proposes a modular, model-agnostic defense of paired Tool-Input and Tool-Output firewalls at the agent-tool interface, reporting near-perfect security with high utility on four public benchmarks. Simultaneously argues existing benchmarks are too weak (weak attacks, flawed metrics) and introduces a stronger three-stage attack strategy for more rigorous evaluation of indirect prompt injection. Teach: Partial. Benchmark-and-defense paper; usable to frame evaluation methodology discussions.

• GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?
  _{arXiv · 2025-10 · arXiv:2510.20333} Benchmark measuring VLM-based mobile GUI agents' resilience to active environmental injection (adversarial events/notifications) injected into realistic app workflows inside fully operational Android emulators; shows current models systematically fail to perceive and reason about manipulated UIs. Teach: On-device mobile-agent benchmark; teachable via Android emulator (AndroidWorld-style) labs.

• Code Agent can be an End-to-end System Hacker: Benchmarking Real-world Threats of Computer-use Agent
  _{arXiv · 2025-10 · arXiv:2510.06607} Introduces AdvCUA, a benchmark of 140 tasks (40 direct malicious, 74 TTP-based, 26 end-to-end kill chains) evaluating whether LLM-powered computer-use/code agents can act as end-to-end system hackers across five agent frameworks; finds current systems inadequately address OS security threats and could lower the barrier for sophisticated intrusions. Teach: Threat benchmark demonstrating offensive agent capabilities (kill chains, TTPs); usable for red-team/blue-team discussion labs.

• MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers
  _{arXiv · 2025-08 · arXiv:2508.14925} First systematic benchmark for Tool Poisoning Attacks (malicious instructions embedded in tool metadata) built on 45 live MCP servers and 353 authentic tools (1,312 test cases across 10 risk categories), reporting >70% attack success and <3% refusal rates, and finding more capable models are often more susceptible. Matters as the canonical tool-poisoning evaluation set. Teach: Benchmark methodology and results are public; usable to demonstrate metadata-injection attacks and measure defense refusal rates in a class.

• MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols
  _{arXiv (Lingnan University, Hong Kong) · 2025-08-17 · arXiv:2508.13220} Systematic MCP security taxonomy of 17 attack types across 4 attack surfaces (user interaction, client, transport, server) paired with an open, modular benchmark and playground; evaluation finds all attack surfaces yield successful compromises and current protections average <30% mitigation success. Reference benchmark for measuring MCP host/client/server resilience. Teach: Public GitHub with prompt datasets, vulnerable/malicious MCP servers, attack scripts (MITM, DNS rebinding), GUI test harness, and protection mechanisms — directly runnable as class labs.

• MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
  _{arXiv · 2025-08-20 · arXiv:2508.14704} Benchmark (Luo et al., Salesforce AI Research) evaluating LLM agent performance on realistic MCP tasks across six domains and eleven real-world servers; provides the realistic tool-use substrate against which tool-misuse and safety evaluations can be run. Teach: Public benchmark and server set; a realistic environment for teaching agent tool-use evaluation.

• RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments
  _{arXiv · 2025-06 · arXiv:2506.15253} Security benchmark supporting both simulated and real tool execution: 80 test cases and 3,802 attack tasks mapped to 11 CWE categories, with tools in JSON, LangGraph and MCP formats. Attacks cut task completion 36.8% on average and hit 85.65% success in academic settings; larger models showed better security. Submitted June 18, 2025. Teach: CWE-mapped attack tasks and MCP/LangGraph tool implementations make it usable as a hands-on security lab.

• AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents (ASSEBench)
  _{arXiv / NeurIPS 2025 · 2025-06 · arXiv:2506.00641} Training-free memory-augmented reasoning framework that lets LLM evaluators reach human-expert level at spotting agent safety/security risks, paired with ASSEBench: 2,293 annotated interaction records covering 15 risk types across 29 scenarios. Targets the reliability of LLM-as-judge for agent auditing. Accepted to NeurIPS 2025. Teach: Code and ASSEBench dataset public at github.com/Astarojth/AgentAuditor-ASSEBench; teachable evaluator/judge labs.

• WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
  _{arXiv · 2025-04 · arXiv:2504.18575} Publicly available benchmark evaluating web-connected LLM/UI agents against prompt injection delivered through malicious webpages; simple low-effort human-written injections partially succeed in up to 86% of cases while full attacker-goal completion remains harder, illustrating 'security by incompetence'. Teach: Open benchmark of injected webpages and attacker goals; usable as a hands-on prompt-injection red-team lab.

• Agent-SafetyBench: Evaluating the Safety of LLM Agents
  _{arXiv (Tsinghua THU-CoAI) · 2024-12 · arXiv:2412.14470} Safety benchmark with 349 interaction environments and 2,000 test cases spanning 8 risk categories; evaluation of 16 agents found none scoring above 60% safety, attributing failures to lack of robustness and lack of risk awareness, and concluding that defense prompts alone are insufficient. Teach: Released at github.com/thu-coai/Agent-SafetyBench with environments and test cases for classroom evaluation.

• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents
  _{arXiv · 2024-10 · arXiv:2410.06703} A configurable enterprise-focused benchmark of 222 tasks paired with safety-and-trustworthiness (ST) policies, scored using the Completion-under-Policy (CuP) metric and Risk Ratio. Shows leading web agents' policy-respecting completion is well below their nominal completion rate. First submitted Oct 9, 2024 (revised through 2026). Teach: Public benchmark suite with tasks, ST policies, and CuP/Risk-Ratio metrics; directly runnable as a lab exercise on web-agent safety.

• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
  _{arXiv · 2024-10 · arXiv:2410.02644} Comprehensive benchmark formalizing attacks/defenses on LLM agents across 10 scenarios (e-commerce, autonomous driving, finance, etc.), 10 agents, 400+ tools, 27 attack/defense methods and 7 metrics; covers prompt injection, memory poisoning, and a Plan-of-Thought backdoor, reaching up to 84.3% average attack success. Foundational reference for agentic-security evaluation. Teach: Public code at github.com/agiresearch/ASB with attack/defense harness and 400+ tools; directly runnable as class labs.

• AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
  _{arXiv / NeurIPS 2024 · 2024-06-19 · arXiv:2406.13352} Extensible benchmark of 97 realistic agent tasks and 629 security test cases for evaluating prompt-injection attacks and defenses on tool-using LLM agents across domains like email, banking, and travel. The de facto evaluation harness cited by most agent-containment papers (NeurIPS 2024 Datasets & Benchmarks). Teach: Open, runnable benchmark that anchors most agent-security defenses; ideal shared lab platform for measuring containment effectiveness.

• agentdojo-inspect (usnistgov) — US AI Safety Institute / CAISI fork of AgentDojo
  _{NIST · 2025} NIST's official GitHub fork of the ETH Zurich AgentDojo agent-hijacking benchmark, developed by the U.S. AI Safety Institute and made compatible with the Inspect evaluation framework (adds an Inspect bridge, data-exfiltration injection tasks, and a terminal RCE environment). This is the concrete open-source tooling behind NIST's agent hijacking evaluations. Repository archived October 2025. Teach: Open-source repo with runnable agent-hijacking environments; directly usable as a hands-on lab for evaluating prompt-injection defenses.

• Dioptra: Test Software for the Characterization of AI Technologies (Securing AI Testbed)
  _{NIST · 2025} NIST's open-source software test platform for assessing trustworthy characteristics of AI systems, supporting the Measure function of the NIST AI RMF. Provides a REST API/web interface for designing, running, and tracking experiments (model testing, evaluations, red-teaming), with reproducible experiment snapshots. Current release v1.1.0; docs at pages.nist.gov/dioptra. Complements the AML taxonomy with practical evaluation infrastructure. Teach: Open-source software with published documentation; teachable as a deployable adversarial-ML evaluation platform for labs.

---

Red-Team & Attack

30 sources.

• Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
  _{arXiv · 2026-07-14 · arXiv:2607.12428} Empirical study finding 38.9% of agent-generated pull requests contain at least one security smell, quantifying the security debt introduced by autonomous developers (and noting human collaborators introduce 67.6% of leaked secrets). Provides measurable outcomes on the risk of merging unaudited coding-agent output. Teach: Provides measurable outcomes for a lab on auditing/gating coding-agent output before merge.

• PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
  _{arXiv · 2026-06 · arXiv:2606.12737} Automated red-teaming framework (submitted June 2026) that constructs realistic source-aware test cases and iteratively evolves them through feedback-driven exploration to expose and localize indirect prompt-injection vulnerabilities in LLM agents, remaining effective even when standard injection defenses are deployed. Teach: Method demonstrating automated discovery + root-cause localization of indirect prompt injections in agents.

• Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours
  _{arXiv · 2026-05 · arXiv:2605.04019} Introduces an AI red-teaming agent (built on the Dreadnode SDK) that automates vulnerability testing from natural-language objectives, compressing agent red-teaming timelines and framing the shift to continuous agentic red-teaming. Demonstrated on Meta Llama Scout with an 85% attack success rate using zero human-developed code. Teach: Good framing reading on why and how agent red-teaming is being automated, with a concrete natural-language-driven workflow.

• A Red Teaming Framework for Evaluating Robustness of AI-enabled Security Orchestration, Automation, and Response Systems
  _{arXiv · 2026-05 · arXiv:2605.17075} An autonomous red-teaming framework combining an LLM-based planner with an RL controller to generate adaptive, multi-stage attack campaigns against AI-enabled SOAR defenders in enterprise networks. Finds standalone LLM agents fail to sustain multi-stage campaigns, motivating hybrid LLM+RL approaches. Domain-specific adversarial evaluation for security operations. Teach: Framework applicable to a SOC/security-agent red-team scenario in class, illustrating hierarchical LLM+RL attack planning.

• Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing
  _{arXiv · 2026-04 · arXiv:2604.20704} Combines structured literature synthesis (nine peer-reviewed sources, seven evaluation protocols) with an open-source automated adversarial robustness testing framework (50+ attacks, 28 defense modules, a Robustness Diagnostic Index, gradient-masking detection) mapped to NIST AI RMF, OWASP LLM Top 10, and the EU AI Act. Links the research corpus to executable robustness tests. Teach: Shows a pipeline from literature to automated robustness tests with standards mapping, good for a methods module.

• AI Security Solutions Landscape For AI and Agentic Red Teaming Q2 2026
  _{OWASP (GenAI Security Project) · 2026-04} OWASP landscape mapping (published April 9, 2026) of commercial and open-source tools for AI and agentic red teaming, framed around risks like prompt injection, model misuse, agent privilege escalation, data poisoning, hallucinations, and emergent behaviors. Useful for tool selection and program design. Teach: Vendor/tool comparison document usable as a survey of the red-teaming tooling ecosystem.

• Insights into AI Agent Security from a Large-Scale Red-Teaming Competition
  _{NIST (CAISI Research Blog) · 2026-03-23} NIST CAISI research blog (published March 23, 2026) on a public red-teaming competition run with Gray Swan and UK AISI where 400+ participants ran 250,000+ attacks against 13 frontier models in tool-use, coding, and computer-use agent scenarios, finding at least one successful attack against every model. Teach: Public writeup of competition methodology and results; teachable as a case study and template for running an agent red-team exercise.

• AgenticRed: Evolving Agentic Systems for Red-Teaming
  _{arXiv · 2026-01 · arXiv:2601.13518} An automated pipeline that uses LLMs to iteratively design (evolve) red-teaming agentic systems without human guidance, treating red-teaming as a system-design problem rather than optimizing attacks within fixed structures. Reports high attack success rates against models including Llama, Qwen, GPT-5.1, and DeepSeek variants. Represents agent-vs-agent adversarial evaluation. Teach: Illustrates using agentic systems themselves as the red-team engine, and framing red-teaming as automated system design/evolution.

• Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
  _{arXiv · 2025-12 · arXiv:2512.20677} Learning-based automated red-teaming framework coupling category-aware attack generation with hierarchical vulnerability detection, using safety seeds, meta-prompt guidance, and evolutionary search. On GPT-OSS-20B it found 47 validated vulnerabilities at a 3.9x higher discovery rate than manual expert red-teaming with 89% detection accuracy. Late-2025 automated red-teaming framework. Teach: Trainable/evolutionary red-team policy usable as an advanced automated-attack demonstration showing category-aware generation plus hierarchical detection.

• RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning
  _{arXiv · 2025-10-11 · arXiv:2510.10008} Treats a deployed RAG QA system as a black box and uses Reinforcement Learning from Black-box Feedback (RLBF) to optimize generation of poisoned documents, improving attack success up to 0.72 over baselines. Authors: Meng Xi et al. (Zhejiang University group). Teach: Public preprint; black-box RL attack framing against deployed RAG suits a red-team module.

• Breaking Guardrails, Facing Walls: Insights on Adversarial AI for Defenders & Researchers
  _{arXiv · 2025-10 · arXiv:2510.16005} Practitioner-oriented study drawing on ~500 CTF participants: simple AI guardrails were readily bypassed with common techniques, but layered multi-step defenses remained a real obstacle. Offers insights for defenders and red-teamers of LLM/agent systems. Late-2025 adversarial-robustness contribution. Teach: Defender-focused, evidence-driven write-up (real CTF data) suitable for a lecture on guardrail evasion patterns and why layered defenses matter.

• ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks
  _{arXiv · 2025-10 · arXiv:2510.02677} An adaptive red-teaming agent (ARMs) that composes plug-and-play attacks against vision-language / multimodal models, orchestrating 11 novel and 17 established red-teaming strategies to automate discovery of adversarial inputs. Exceeds baselines by ~52% on average and >90% ASR on Claude-3.5-Sonnet; ships ARMs-Bench (30k+ instances across 51 risk categories). Late-2025 example of agent-driven automated red-teaming. Teach: Method paper with a modular attack/transform design (plug-and-play strategy library + agentic orchestration) that can be demonstrated as an automated red-team pipeline against multimodal targets.

• FinBot Agentic AI Capture The Flag (CTF) Application
  _{OWASP GenAI Security Project · 2025-08-12} Hands-on CTF application from the OWASP GenAI Agentic Security Initiative for practicing exploitation and defense of agentic AI risks in a deliberately vulnerable financial-assistant agent. Confirmed on the OWASP GenAI resources page, dated Aug 12, 2025. Teach: Explicitly built as a teachable lab/CTF — directly usable for class exercises.

• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
  _{arXiv · 2025-07-28 · arXiv:2507.20526} The arXiv paper (Zou, Lin, Jones, et al.) documenting the large-scale public agent red-teaming competition underlying the NIST CAISI insights: 1.8M+ prompt-injection attacks against 22 agents across 44 scenarios, with 60,000+ policy violations (unauthorized data access, illicit financial actions, regulatory noncompliance). Introduces the Agent Red Teaming (ART) benchmark; primary technical evidence for agent hijacking prevalence. Teach: Public preprint with dataset/benchmark (ART) details; teachable as the empirical backbone for an agent-security measurement module.

• Agentic Misalignment: How LLMs Could be Insider Threats
  _{Anthropic · 2025-06-20} Anthropic stress-tested 16 leading models from multiple developers as autonomous agents with sensitive access and goal conflicts, finding models would resort to blackmail (79-96% rates in some setups), corporate espionage and other harmful 'insider threat' actions in controlled simulations. Anthropic notes no such behavior observed in real-world deployments. Demonstrates agentic safety failure modes and the need for proactive alignment safeguards. Teach: Anthropic released methods and code publicly (github.com/anthropic-experimental/agentic-misalignment), enabling reproducible red-team labs on agentic misalignment.

• RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
  _{arXiv · 2025-05 · arXiv:2505.21936} Adversarial testing framework pairing a VM-based OS sandbox with Docker-based web platforms and RTC-Bench (864 examples) for indirect prompt injection against computer-use agents; reports attack success rates up to ~60% against frontier agents in end-to-end settings. Teach: Open hybrid VM-OS + Docker-web sandbox plus RTC-Bench; a full adversarial-testing harness usable in class.

• EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks
  _{arXiv · 2025-05 · arXiv:2505.14289} An evolutionary red-teaming framework that mutates benign seeds into semantically deceptive environmental cues, achieving up to 85% attack success with rapid convergence; finds semantic deception (not visual appearance) is the primary determinant of success and reveals an 'alignment paradox' where instruction-following makes agents more susceptible. Teach: Automated evolutionary red-teaming loop that can be reproduced as an adversarial-generation lab.

• CSA Agentic AI Red Teaming Guide
  _{CSA · 2025-05-28} CSA+OWASP hands-on red-teaming manual (published May 28 2025, 50+ contributors) spanning 12 threat categories for autonomous agents — authorization/control hijacking, checker-out-of-the-loop, goal/instruction manipulation, memory & knowledge-base poisoning, multi-agent collusion, resource exhaustion, supply-chain and agent untraceability — each with attack vectors, example prompts and deliverables. Teach: Full public PDF with scenario-based test requirements and example prompts — an off-the-shelf lab curriculum for red-teaming agents.

• Agentic AI Red Teaming Guide (Cloud Security Alliance)
  _{CSA · 2025-05} Operational red-teaming playbook for agentic AI (permission/privilege escalation, hallucination, orchestration flaws, memory manipulation, supply-chain risks, autonomous decision loops), with testing methodology and example prompts; developed with OWASP collaboration and complements MAESTRO threat modeling. Published May 28, 2025. Teach: Public playbook with distinct threat categories and concrete test methods/example prompts; directly maps to hands-on red-team lab exercises.

• AI Agents Are Here. So Are the Threats. (Unit 42 agentic AI threat research)
  _{Palo Alto Networks Unit 42 · 2025-05-01} Unit 42 built two functionally identical agentic apps on CrewAI and AutoGen and ran the same nine attack scenarios against both—including indirect prompt injection, agent-instruction extraction, credential/conversation exfiltration, SQL injection, and unauthorized network access—concluding vulnerabilities stem from insecure design patterns and unsafe tool integrations (not the frameworks themselves) and that layered defenses are required. Teach: Test code and datasets open-sourced on GitHub (github.com/PaloAltoNetworks/stock_advisory_assistant) for reproducible agentic red-team labs.

• Zenity Labs × MITRE ATLAS — GenAI Attacks Matrix integration
  _{Zenity Labs / MITRE · 2025-03} Real collaboration: this Zenity Labs post (dated Mar 20, 2025) documents GenAI Attacks Matrix techniques incorporated into MITRE ATLAS, adding 8 techniques + 4 subtechniques (AML.T0064–T0071: Gather RAG-Indexed Targets, Discover LLM System Information, LLM Prompt Crafting, Retrieval Content Crafting, RAG Poisoning, LLM Prompt Obfuscation, Trusted Output Components Manipulation, False RAG Entry Injection) plus the 'Financial Transaction Hijacking with M365 Copilot' case study. NOTE: the item's original claim of an Oct 2025 release adding agent-specific techniques AML.T0080–T0086 (Context Poisoning, Modify AI Agent Configuration, RAG Credential Harvesting, Exfiltration via Tool Invocation) does NOT match this URL and could not be confirmed — date and technique IDs corrected/flagged accordingly. Teach: Public technique IDs mapped to real LLM/RAG attacks; teachable as a red-team mapping exercise, but verify the exact ATLAS technique IDs against the current live matrix before citing.

• Red-Teaming LLM Multi-Agent Systems via Communication Attacks
  _{arXiv · 2025-02 · arXiv:2502.14847} Red-teams multi-agent systems by attacking the inter-agent communication channel itself. Introduces the Agent-in-the-Middle (AiTM) attack that intercepts and manipulates inter-agent messages to propagate compromise through the agent network. Directly relevant to A2A-channel and attack-propagation teaching. Teach: Yes — the AiTM communication-attack methodology is reproducible as a red-team exercise.

• Technical Blog: Strengthening AI Agent Hijacking Evaluations
  _{NIST · 2025-01-17} NIST CAISI technical blog defining agent hijacking as indirect prompt injection and evaluating priority risks (remote code execution, database exfiltration, automated phishing). Red-teaming with the UK AI Security Institute against Claude 3.5 Sonnet raised attack success from 11% for the strongest baseline to 81% for the strongest new attack, using the AgentDojo framework. Teach: Public blog with concrete evaluation methodology (AgentDojo Workspace/Travel/Slack/Banking environments); directly reproducible in a class lab.

• GenAI Red Teaming Guide
  _{OWASP (GenAI Security Project) · 2025-01} OWASP GenAI Red Teaming Guide v1.0 (published Jan 22, 2025): structured lifecycle guide covering model evaluation, implementation testing, infrastructure assessment, and runtime behavior analysis for GenAI and agentic red teaming. Methodology reference for building an agent red-team program. Teach: Public methodology document that maps cleanly onto a hands-on red-team course structure across model, implementation, infrastructure, and runtime layers.

• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
  _{arXiv (Gray Swan / UK AISI); ICLR 2025 · 2024-10 · arXiv:2410.09024} Benchmark of 110 explicitly malicious agent tasks (440 with augmentations) across 11 harm categories (fraud, cybercrime, harassment), scoring both refusal and retained multi-step tool-use capability after jailbreak. Shows leading LLMs are surprisingly compliant with malicious agentic requests. Accepted at ICLR 2025. Teach: Dataset publicly released on Hugging Face with harm-score/refusal metrics; teachable red-team misuse labs.

• Microsoft PyRIT (Python Risk Identification Toolkit) and AI Red Teaming Agent
  _{vendor (Microsoft) · 2024-02} Open automation framework (announced Feb 22, 2024) for red-teaming generative AI, with multi-turn adaptive attacks (Crescendo, TAP, Skeleton Key). The AI Red Teaming Agent (2025, in Azure AI Foundry) runs PyRIT to auto-scan agent/GenAI projects. A leading production red-teaming toolchain. Teach: Open-source toolkit with docs and orchestrators; strong basis for hands-on automated red-team labs.

• NVIDIA Garak — LLM/Agent Vulnerability Scanner
  _{vendor (NVIDIA) · 2026} Open-source vulnerability scanner probing for prompt injection, jailbreaks, encoding bypasses, glitch tokens, data extraction, and toxicity across ~20+ probe families. Recent releases (latest v0.15.1, June 2026) add support for testing agentic AI systems. Broad-spectrum automated probing. Teach: Free CLI scanner runnable in class to fire thousands of adversarial probes at a target agent.

• SANS SEC543: AI-Assisted Source Code Analysis and Exploitation for Penetration Testers
  _{SANS · 2025} New SANS course teaching AI-assisted source code analysis and exploitation for pentesters. Confirmed listed on the SANS AI curriculum page (sans.org/mlp/ai) as a New, AI-Focused, Intermediate-level course. Part of SANS's 2025-2026 AI security course expansion focused on using AI in offensive security workflows. Teach: Documented hands-on labs; course outline public and adaptable.

• SANS SEC535: Offensive AI - Attack Tools and Techniques (GIAC Offensive AI Analyst, GOAA)
  _{SANS / GIAC · 2025} New 3-day SANS offensive-AI course with 14 hands-on labs on applying real-world offensive AI techniques, paired with the new GIAC Offensive AI Analyst (GOAA) certification. Directly relevant to red-teaming agentic and LLM systems. Teach: 14 hands-on labs plus public GOAA objectives; strongly lab-driven and teachable as an offensive AI module.

• DeepTeam — Open-Source LLM/Agent Red Teaming Framework (OWASP Agentic Top 10 mapping)
  _{vendor / open source · 2025} Open-source red-teaming framework implementing 40+ vulnerability classes and multiple adversarial attack strategies, with explicit mapping to the OWASP Top 10 for Agentic Applications / OWASP ASI 2026 (ASI01–ASI10: goal hijack, tool misuse, memory poisoning, etc.). Model-level automated agent red-teaming (does not cover runtime tool execution/infra). Teach: Free framework with OWASP-mapped attacks; teachable as a guided lab tied to the agentic Top 10.

---

Prompt Injection

15 sources.

• When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents
  _{arXiv · 2026-07-06 · arXiv:2607.06595} Demonstrates memory-poisoning attacks against LLM agents that persist across sessions, showing how injected content in long-term memory steers future decisions. Authors: George Torres, Sharad Shrestha, Satyajayant Misra. Directly on-topic for persistent memory security. Teach: Public preprint; attack methodology reproducible in a class exercise.

• From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents
  _{arXiv · 2026-06-03 · arXiv:2606.04329} Systematically studies how untrusted tool/environment outputs get written into persistent LLM-agent memory and later poison behavior, providing a taxonomy of memory-poisoning attack paths from input to trusted long-term store. Authors include Pritam Dash et al. Foundational for teaching how agent memory becomes an attack surface. Teach: Public arXiv preprint with attack taxonomy; usable as a lecture framework and lab basis.

• Trojan Hippo: Weaponizing Agent Memory for Data Exfiltration
  _{arXiv · 2026-05-03 · arXiv:2605.01970} Plants a dormant payload into an agent's long-term memory via a single untrusted tool call that activates on sensitive topics (finance, health, identity) to exfiltrate personal data; reports 85-100% ASR against frontier OpenAI/Google models, with defenses causing security-utility tradeoffs. Authors: Das, Piet, Kaviani, Beurer-Kellner, Tramèr, Wagner. Teach: Public preprint; dormant-payload exfiltration-via-memory scenario is teachable for agent threat modeling and defense tradeoffs.

• Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agents
  _{arXiv · 2026-04-03 · arXiv:2604.02623} Presents eTAMP (Environment-injected Trajectory-based Agent Memory Poisoning), the first cross-session, cross-site memory-poisoning attack without direct memory access; viewing a compromised webpage silently corrupts agent memory, with ASR up to 8x higher under environmental stress (19.5-32.5% overall). Authors: Wei Zou et al. Teach: Public preprint; the 'poison once, exploit forever' persistence concept is a strong teaching motif for memory integrity.

• MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval
  _{arXiv · 2025-12-18 · arXiv:2512.16962} Shows persistent compromise of agents by poisoning the experience/episodic memory that is later retrieved, so a single injection keeps influencing the agent (distinct from traditional prompt injection). Authors: Saksham Sahai Srivastava and Haoyu He. Strong Dec-2025 example of poison-once persistence. Teach: Public preprint; attack lifecycle illustrates persistence for coursework.

• NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledge
  _{arXiv · 2025-10-24 · arXiv:2510.21144} Identifies 'Poison-Responsive Neurons' whose activation correlates with poisoned context, then uses genetic-algorithm optimization to craft fluent adversarial passages, reporting Population Overwrite Success Rate over 90%. Authors: Hanyu Zhu, Lance Fiondella, Jiawei Yuan, Kai Zeng, Long Jiao. Teach: Public preprint describing a neuron-guided optimization method; teachable as an advanced knowledge-base poisoning case study.

• Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors
  _{arXiv · 2025-10-01 · arXiv:2510.00586} Decomposes adversarial RAG documents into reusable 'Attention Attractors' and 'Focus Regions'; attractors are optimized to steer model attention and transfer to unseen RAG systems without retraining, enabling scalable poisoning. Authors: Yen-Shan Chen, Sian-Yao Huang, Cheng-Lin Yang, Yun-Nung Chen (v3 rev. June 2026). Teach: Public preprint; transferable attention-steering poisoning documents make a concrete red-team lab on RAG retrieval integrity.

• Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments
  _{arXiv · 2025-09 · arXiv:2509.11250} Studies environmental injection attacks (EIAs) that hijack GUI-agent execution via malicious triggers embedded in webpages; argues prior work overestimated vulnerability by ignoring dynamic conditions, and proposes 'Chameleon' (LLM-driven environment simulation plus an 'Attention Black Hole' mechanism) to craft attacks robust to changing visual context, tested across six websites and four agents. Teach: Concrete EIA attack recipes (pop-ups, fake notifications, Chameleon framework) demonstrable in a GUI-agent sandbox.

• ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents
  _{arXiv (accepted ICLR 2026) · 2025-09 · arXiv:2509.22830} Attack that formats malicious payloads to mimic native chat-template structure, injecting instructions into LLM agents; includes a persuasion-driven multi-turn variant. Beats prior injections on AgentDojo (+5.18-32.05%) and InjecAgent (+15.13-45.90%), transfers across models, and evades prompt-based defenses. A novel agent-specific prompt-injection vector. Teach: Concrete, reproducible attack technique that can be run in a lab against a template-driven agent to show structural (not just content) injection.

• The Hidden Threat in Plain Text: Attacking RAG Data Loaders
  _{arXiv · 2025-07-07 · arXiv:2507.05093} Presents a taxonomy of 9 knowledge-based poisoning attacks plus Content Obfuscation and Content Injection vectors against RAG ingestion; achieves 74.4% success across 357 scenarios on five data loaders and validates against systems like NotebookLM and OpenAI Assistants. Authors: Castagnaro, Salviati, Conti, Pajola, Pizzi. Teach: Public preprint; data-loader/ingestion attack surface makes a strong pipeline supply-chain security lab.

• Context Manipulation Attacks: Web Agents Are Susceptible to Corrupted Memory
  _{arXiv · 2025-06-18 · arXiv:2506.17318} Introduces 'plan injection,' a context manipulation attack that corrupts web agents' internal task representations via client-side/third-party memory; context-chained injections boost privacy exfiltration success by 17.7% over prompt-based attacks. Authors: Patlan, Hebbar, Viswanath, Mittal. Teach: Public preprint; web-agent corrupted-memory / plan-injection demo is reproducible for threat modeling.

• Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs
  _{arXiv · 2025-06-13 · arXiv:2506.11415} Introduces the BRRA (Bias Retrieval and Reward Attack) framework using adversarial document generation and subspace projection to show corpus poisoning directly amplifies model output bias, connecting RAG security to fairness. Authors: Linlin Wang, Tianqing Zhu, Laiqiao Qin, Longxiang Gao, Wanlei Zhou. Teach: Public preprint; bias-steering demonstration teachable for governance and fairness discussions.

• MPMA: Preference Manipulation Attack Against Model Context Protocol
  _{arXiv · 2025-05-16 · arXiv:2505.11154} Introduces preference/tool-selection manipulation attacks against MCP, where an attacker-deployed MCP server biases an LLM's tool choice toward its own services for financial gain. Proposes DPMA (direct manipulative tool descriptions) and GAPMA (genetic-based, balancing effectiveness with stealth). An economic/selection-layer attack distinct from classic tool poisoning. Teach: Attack mechanism (DPMA/GAPMA) described; reproducible as a tool-registry manipulation exercise.

• Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation
  _{arXiv · 2025-02-05 · arXiv:2502.03233} Lin et al. show that poisoning the retrieval knowledge base of retrieval-augmented code generation injects insecure/malicious code, with a single poisoned example compromising up to 48% of generated code across multiple LLMs and retrieval methods. Teach: Public preprint; code-generation poisoning is a concrete, demonstrable lab with a striking single-example result.

• Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
  _{arXiv · 2024-10 · arXiv:2410.07283} Introduces self-replicating prompts (by Donghyun Lee and Mo Tiwari) that spread agent-to-agent like a computer virus, driving harmful actions (data exfiltration, scams), and proposes LLM Tagging as a defense. Demonstrates multi-agent systems remain vulnerable even when agents do not share all communications. The canonical attack-propagation/AI-worm paper. Teach: Yes — attack and LLM-Tagging defense are reproducible; strong hands-on propagation lab.

---

MCP Security

13 sources.

• MCP Server Security: A Threat Model for Agent Tool Supply Chains
  _{vendor (General Analysis) · 2026-05-02} Practitioner guide presenting a threat model (nine attack vectors) for the MCP/agent tool supply chain, cataloguing real incidents and CVEs (MCP Inspector RCE CVE-2025-49596, Cursor zero-click CVE-2025-54136, typosquat CVE-2025-54994, plus 2026 CVEs) and registry-poisoning findings (OX Security poisoning 9 of 11 MCP registries). Ends with a seven-point pre-install evaluation checklist. An incident-grounded supply-chain reference. Teach: Concrete CVE/incident catalog and control checklist — strong material for a supply-chain case-study lab.

• Attested Tool-Server Admission: A Security Extension to the Model Context Protocol
  _{arXiv · 2026-05-22 · arXiv:2605.24248} Proposes a security extension to MCP that gates tool-server admission behind offline-signed clearance assertions published at a standard URI, per-server deny-by-default tool allowlists, and audit logging with enforcement modes. Framed as a backward-compatible addendum to MCP, directly addressing MCP-server provenance and trusted onboarding. Teach: Public protocol-extension preprint (author: Alfredo Metere); blueprint for an attested-admission implementation lab.

• A First Measurement Study on Authentication Security in Real-World Remote MCP Servers
  _{arXiv · 2026-05-21 · arXiv:2605.22333} Empirically measures authentication weaknesses across 7,973 real remote MCP servers, finding over 40% lack authentication, documenting nine categories of security flaws (dynamic client registration issues affecting nearly all), and yielding nine CVE assignments. Quantifies trust/exposure risk in the deployed MCP supply chain. Teach: Public measurement study (Fudan-affiliated authors incl. Min Yang); basis for an MCP auth-audit lab.

• From Component Manipulation to System Compromise: Understanding and Detecting Malicious MCP Servers
  _{arXiv · 2026-04-02 · arXiv:2604.01905} Characterizes how manipulated MCP-server components escalate to full system compromise and proposes detection, addressing MCP-server provenance and trust directly. Teach: Public preprint (Huang, Zhao, Chen et al.) with detection approach; suitable for a malicious-MCP-server hunting lab.

• A Practical Guide for Secure MCP Server Development
  _{OWASP · 2026-02} OWASP GenAI developer-focused guidance for building secure MCP tool servers — the connection point between AI assistants and external systems — covering tool-description hygiene, input/output handling, and sandboxing of tool execution. Teach: Developer guide with actionable controls; a blueprint for a secure-MCP-server build exercise.

• MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers
  _{arXiv · 2025-12 · arXiv:2512.15163} Benchmark evaluating LLM safety against real (not simulated) MCP servers via realistic multi-turn evaluation across five domains including browser automation, financial analysis, and web search. Finds all models remain vulnerable to MCP attacks and surfaces a safety-utility trade-off. Teach: Real-server MCP safety test cases; teachable as an MCP-focused evaluation lab (spec on arXiv).

• Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluation
  _{arXiv · 2025-11 · arXiv:2511.03841} By Louck, Stulman, and Dvir. Empirical security assessment of multi-agent communication protocols (CORAL, ACP, A2A) using a 14-point vulnerability taxonomy spanning authentication, authorization, integrity, confidentiality, and availability. Finds CORAL has robust architecture but implementation flaws while ACP's flexibility creates gaps; recommends combining their strengths. (Earlier summary's specific figures — ~66% replay rate, ACM journal publication — could not be confirmed and appear inaccurate.) Teach: Yes — the comparative taxonomy is reproducible as a protocol red-team teaching exercise.

• CheatSheet – A Practical Guide for Securely Using Third-Party MCP Servers 1.0
  _{OWASP · 2025-11} OWASP GenAI Agentic Security Initiative guidance for safely consuming external/third-party MCP servers, addressing malicious tool descriptions, tool poisoning, and least-privilege consumption of untrusted tools. Teach: Concrete checklist/cheatsheet directly usable as lab guidance for vetting third-party MCP tools.

• MCPGuard: Automatically Detecting Vulnerabilities in MCP Servers
  _{arXiv · 2025-10-27 · arXiv:2510.23673} Detects vulnerabilities in Model Context Protocol servers across three threat classes: protocol design flaws enabling agent hijacking, conventional web vulnerabilities in servers, and supply-chain risks. Surveys defenses including server-side scanning and runtime monitoring. Directly covers MCP-server provenance/dependency risk. Teach: Public preprint (authors: Bin Wang et al.) with detection pipeline; teachable as an MCP-server scanning lab.

• CSA MCP Security Resource Center & Agentic MCP Security Best Practices
  _{CSA · 2025-08-20} CSA's Model Context Protocol security hub framing MCP as the 'agentic AI control plane' and prescribing defense-in-depth: authentication, tool integrity, session management, supply-chain validation, execution isolation and behavioral monitoring. The linked Lab Space best-practices guide is a living document (draft revisions into 2026) and references the MCP Security Crisis research note. Teach: Public best-practices guide plus Lab Space materials — teachable as a hands-on MCP-server hardening checklist.

• Agent-to-Agent OAuth: A Guide for Secure AI Agent Connectivity with MCP (Stytch)
  _{Stytch (vendor) · 2025-08} Vendor engineering guide (Aug 18, 2025) on applying OAuth 2.0 to agent-to-agent connectivity and Model Context Protocol (MCP) servers: scoped access tokens without handling passwords/API keys, distinguishing agent-to-agent OAuth from agent-as-client, token issuance/validation/refresh/revocation, and a Node.js example using Stytch as the OAuth provider. Teach: Public blog with implementation patterns and a Node.js example; usable as a hands-on reference for building agent-to-agent OAuth over MCP.

• MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits
  _{arXiv · 2025-04-02 · arXiv:2504.03767} Radosevich and Halloran demonstrate that leading LLMs can be manipulated via MCP tools into malicious code execution, remote access, and credential theft. Introduces MCPSafetyScanner, the first agentic auditing tool to find MCP server vulnerabilities before deployment. Teach: MCPSafetyScanner is a released, freely available auditing tool; scanning a vulnerable MCP server makes a concrete class lab.

• MCP + Auth0: An Agentic Match Made in Heaven (Auth0)
  _{Auth0 / Okta (vendor) · 2025} Vendor walkthrough arguing MCP is 'necessary but insufficient' for security and must be paired with identity controls. Treats the MCP server as an OAuth 2.0 / OIDC resource server secured with Auth0, covering impersonation/privilege-escalation risks and a practical flow where a user authenticates via Auth0, receives scoped tokens, and an agent executes authorized actions on the user's behalf. Teach: Public blog with a concrete MCP-as-resource-server pattern; teachable as an MCP authorization lab.

---

Multi-Agent / A2A

9 sources.

• TraceFix: Repairing Agent Coordination Protocols with TLA+ Counterexamples
  _{arXiv · 2026-05 · arXiv:2605.07935} Verification-first approach for LLM multi-agent coordination: derives a protocol topology from the task, generates coordination logic in PlusCal, and iteratively repairs it using counterexamples from the TLA+ model checker until verification succeeds; across 48 tasks reached full verification (62.5% first-attempt, none >4 iterations) and cut deadlock/livelock from 31.1% to 14.1%. Authors: Shuren Xia, Qiwei Li, Taqiya Ehsan, Jorge Ortiz (submitted May 8, 2026). Teach: TLA+/PlusCal are open and heavily documented; model-check-then-repair a small agent protocol makes an excellent hands-on lab.

• Authorization Propagation in Multi-Agent AI Systems: Identity Governance as Infrastructure
  _{arXiv · 2026-05 · arXiv:2605.05440} Examines how authorization must propagate across chains of collaborating non-human agents that retrieve data, delegate tasks, and share results across boundaries, framing 'authorization propagation' as a problem not fully covered by RBAC or ABAC. Identifies three sub-problems and seven structural requirements, arguing identity governance must be treated as infrastructure evaluated continuously and enforced at every interaction boundary. Author: Krti Tallam. Teach: Public arXiv preprint; good reading for a multi-agent authorization-propagation module.

• Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Research
  _{arXiv · 2025-08 · arXiv:2508.09815} Research paper by Klaudia Krawiecka and Christian Schroeder de Witt that extends and critiques the OWASP MAS Threat Modeling Guide with findings from multi-agent security research, proposing new threat classifications (reasoning collapse across planner-executor chains, metric overfitting, unsafe delegation escalation) and evaluation strategies. Confirmed on arXiv, submitted Aug 13, 2025. Teach: Open-access preprint; pairs directly with the OWASP MAS guide for a critique/extension seminar.

• When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems
  _{arXiv · 2025-07 · arXiv:2507.14660} By Ren, Xie, Wei, Yin, Yan, Ma, and Shao. Proof-of-concept framework simulating malicious multi-agent collusion under centralized and decentralized coordination, applied to misinformation spread and e-commerce fraud; finds decentralized systems more effective at malicious action and better at evading detection. Concrete, runnable study of agent collusion. Teach: Yes — includes a simulation framework suitable for collusion labs.

• A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures
  _{arXiv · 2025-06 · arXiv:2506.19676} Kong et al. survey agent-to-agent and agent-to-tool communication protocols (including Anthropic's MCP and Google's A2A), categorizing communication into three classes via a layered architecture, analyzing per-protocol security risks, and proposing defenses. Covers the communication/coordination layer of the attack surface. Teach: Protocol-level threat framing; teachable alongside MCP and A2A protocol security labs.

• Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
  _{arXiv · 2025-05 · arXiv:2505.02077} Schroeder de Witt et al. position paper introducing multi-agent security as a field for securing networks of interacting AI agents against threats that emerge or amplify through interaction (coordinated attacks, privacy breaches, adversarial evasion), with a threat taxonomy, security-utility tradeoffs, and a unified research agenda. Grounds OWASP agentic and inter-agent-communication risks. Teach: Open-access preprint; good survey reading to frame multi-agent security research directions.

• OWASP Multi-Agentic System (MAS) Threat Modeling Guide v1.0
  _{OWASP · 2025-04-23} Released 23 April 2025, applies OWASP's agentic threat taxonomy (with the MAESTRO layered framework) to threat-model complex multi-agent systems, complementing the OWASP ASI taxonomy by analyzing threats per architectural layer and surfacing cross-layer/emergent threats (e.g., malicious agent collusion, goal misalignment). Teach: Public PDF; pairs with CSA MAESTRO for a multi-agent threat-modeling lab.

• AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management
  _{arXiv · 2025-03-06 · arXiv:2503.04392} Defends LLM multi-agent systems via hierarchical information classification, with ThreatSieve (authority verification / anti-impersonation) and HierarCache (adaptive memory protection). Reports defense success rates above 80% under adversarial conditions. Teach: Public preprint; hierarchical data-management design teachable for multi-agent memory security and access control.

• Secret Collusion among AI Agents: Multi-Agent Deception via Steganography
  _{arXiv · 2024-02 · arXiv:2402.07510} By Motwani, Baranchuk, Strohmeier, Bolina, Torr, Hammond, and Schroeder de Witt. Formalizes secret collusion where agents covertly share information via steganography, examines incentives and countermeasures, and evaluates LLM steganographic capability (noting a GPT-4 capability jump). Foundational reference for covert agent collusion and its privacy/security implications. Teach: Yes — steganographic-collusion demos are reproducible in class.

---

Identity & Authorization

25 sources.

• When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents
  _{arXiv · 2026-06-18 · arXiv:2606.20023} Introduces ToolPrivBench (an 8-domain benchmark) showing LLM agents frequently select higher-privilege tools even when lower-privilege alternatives suffice — a problem worsened by transient failures. General safety training and prompt-based controls transfer poorly; the authors propose a privilege-aware post-training defense that reduces unnecessary high-privilege tool use while preserving capability. Teach: Benchmark (ToolPrivBench) plus a post-training defense method; a lab on measuring and reducing over-privileged tool choices under least-privilege constraints.

• The Proxy Knows Too Much: Sealing LLM API Routers with Attested TEEs
  _{arXiv · 2026-06-15 · arXiv:2606.16358} AEGIS uses attested Trusted Execution Environments (hardware enclaves) to confine plaintext handling in LLM API routers to a small trusted component while leaving auth/management on the untrusted host, so a malicious proxy cannot read or rewrite agent tool calls. The client cryptographically verifies the enclave before releasing plaintext; evaluation blocks four classes of router-based attacks with minimal overhead. Authors: Sipeng Xie, Qianhong Wu, Hengrun Lu, Ziliang Sun, Qi Wu, Bo Qin, Qin Wang. Teach: Clear TEE-attestation architecture (enclave vs. untrusted host split) suitable for a confidential-computing-for-agents lab.

• Overlaying Governance: A Compositional Authorization Framework for Delegation and Scope in Agentic AI
  _{arXiv · 2026-06 · arXiv:2606.03518} A compositional authorization framework treating delegation and scope as composable governance overlays for agentic AI. Provides primitives for delegation types, permissions, accountability, and resource-scope attenuation, and a compositional operator that overlays agentic semantics onto existing relational policies without modifying them, backed by formal proofs and empirical evaluation. Authors: Amjad Ibrahim and Yong Li. Teach: Public arXiv preprint; teachable as a formal treatment of scope composition and attenuation across multi-hop delegation.

• Governing Dynamic Capabilities: Cryptographic Binding and Reproducibility Verification for AI Agent Tool Use
  _{arXiv · 2026-03 · arXiv:2603.14332} Proposes cryptographically binding capabilities to AI agent tool invocations plus reproducibility verification so tool-use authority is provable and auditable. Sets three governance requirements (capability integrity, behavioral verifiability, interaction auditability), proves a Chain Verifiability Theorem and Bounded Divergence Theorem, and validates Ed25519/SHA-256 and BBS+/Groth16 SNARK implementations across nine models with <0.02% overhead. Author: Ziling Zhou. Teach: Public arXiv preprint; teachable as a capability-token / cryptographic-binding lab for agent tool use.

• Give Them an Inch and They Will Take a Mile: Understanding and Measuring Caller Identity Confusion in MCP-Based AI Systems
  _{arXiv · 2026-03-08 · arXiv:2603.07473} Studies caller identity confusion in MCP systems: treating MCP servers as trusted without authenticating the caller is fundamentally insecure. Finds most servers use persistent authorization states granting tool access after initial approval without re-checking caller identity, and many lack per-tool authentication enforcement, greatly expanding the attack surface. Matters for the authentication/authorization gap in MCP. Teach: Measurement study with a clear identity-confusion attack model; teachable as an authz case study.

• AIP: Agent Identity Protocol for Verifiable Delegation Across MCP and A2A
  _{arXiv · 2026-03-25 · arXiv:2603.24775} Defines an agent identity protocol using Invocation-Bound Capability Tokens (IBCTs) that fuse identity verification with delegated authorization across MCP and Agent-to-Agent frameworks, supporting single- and multi-hop delegation. Motivated by a scan of ~2,000 MCP servers that all lacked authentication; underpins provenance and trust chains for agent-to-agent supply chains. Teach: Public protocol spec (author: Sunil Prakash); teachable as a verifiable-delegation/identity implementation exercise.

• Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework
  _{arXiv · 2026-01 · arXiv:2601.11893} Analyzes privilege-escalation attacks in LLM-based agent systems and proposes SEAgent, a Mandatory Access Control (MAC) framework built on attribute-based access control principles to prevent over-privileged tool access and escalation across agents/tools — importing a classic OS security primitive into agent runtime authority control. Authors: Ji, Wu, Jiang, Ma, Li, Gao, Wang, Li (submitted Jan 17, 2026). Teach: Yes — maps directly onto classic MAC/Bell-LaPadula teaching material, giving a bridge from OS security to agent security in a course.

• AgentGuardian: Learning Access Control Policies to Govern AI Agent Behavior
  _{arXiv · 2026-01 · arXiv:2601.10440} Learns context-aware access-control policies (rather than hand-authoring them) to govern which actions/tools an AI agent may invoke, monitoring execution traces during a controlled staging phase to detect malicious or misleading inputs while preserving normal functionality — extending the least-privilege line with data-driven policy induction. Authors: Abaev, Klimov, Levinov, Mimran, Elovici, Shabtai (submitted Jan 15, 2026). Teach: Partial — policy-learning framing pairs well with Progent for a compare/contrast unit on authored vs. learned access-control policies.

• Permission Manifests for Web Agents
  _{arXiv · 2025-12-07 · arXiv:2601.02371} Proposes an agent-permissions.json manifest (analogous to robots.txt) letting website owners declare which agent interactions are permitted, extending robots.txt principles to LLM-mediated interactions and enabling least-privilege scoping while reducing reliance on blanket blocking/CAPTCHAs. Teach: Manifest spec analogous to robots.txt; teachable as a standard-in-the-making for constraining web-agent tool access.

• MiniScope: A Least Privilege Framework for Authorizing Tool Calling Agents
  _{arXiv · 2025-12-11 · arXiv:2512.11147} Zhu et al. automatically enforce least-privilege for tool-calling agents by reconstructing permission hierarchies among tool calls and combining them with mobile-style permission models, confining damage from unreliable LLMs operating on user accounts while preserving usability. Teach: Concrete least-privilege framework; teachable as the reference design for tool-access confinement.

• The best providers for authenticating AI agents via OAuth and OIDC in 2025
  _{WorkOS (vendor) · 2025-11-24} Industry survey of how OAuth/OIDC are being applied to authenticate non-human actors (AI agents, bots). Compares five identity platforms (WorkOS, Auth0, Okta, Keycloak, Logto) and enumerates requirements: scoped least-privilege tokens, short-lived access tokens, automated refresh/rotation, fine-grained permissions, machine-to-machine flows, and instant revocation. Teach: Public vendor overview (WorkOS is one of the compared providers, so not fully neutral); useful as an orienting read on the provider landscape and core M2M auth requirements for agents.

• Secure Autonomous Agent Payments: Verifying Authenticity and Intent in a Trustless Environment
  _{arXiv · 2025-11-08 · arXiv:2511.15712} Vivek Acharya (arXiv 2511.15712) proposes a framework for cryptographically authenticating AI-agent-initiated transactions in a trustless setting. Rather than TEE attestations, it relies on blockchain, decentralized identity (DID) standards, verifiable credentials, and zero-knowledge proofs to establish agent identity and verify intent while preserving privacy in financial transactions. Teach: Concrete late-2025 mapping of decentralized-identity primitives (DIDs, verifiable credentials, ZKPs) to the agent-autonomy problem of authenticating and authorizing autonomous payments.

• What is the Microsoft agent identity platform (Microsoft Entra Agent ID)
  _{Microsoft (vendor documentation) · 2025-10-24} Microsoft Learn conceptual doc for the Microsoft agent identity platform (Entra Agent ID), giving AI agents first-class directory identities in Entra ID with agent identity blueprints, an agent registry, OAuth 2.0/OIDC-based authentication and authorization, SDKs, and lifecycle/governance controls so agents are governed as distinct non-human identities rather than generic service accounts. Teach: Public product documentation; usable to demonstrate a production enterprise agent-IAM implementation, its identity constructs (blueprint, agent identity, agent user account), and delegated/application-only token flows.

• Identity Management for Agentic AI
  _{OpenID Foundation · 2025-10} OpenID Foundation whitepaper (Lead Editor: Tobin South, October 2025) analyzing how OAuth 2.1/OIDC apply to AI agents, where they fall short (cross-domain, highly autonomous, or asynchronous scenarios; delegated permissions on behalf of multiple users; MCP adoption), and what new standards work (e.g., IPSIE profiling) is needed. A foundational reference document for the agent identity space. Teach: Public PDF whitepaper; usable as assigned reading and to frame a standards-gap discussion in class.

• A Vision for Access Control in LLM-based Agent Systems
  _{arXiv · 2025-10 · arXiv:2510.11108} A vision/position paper arguing conventional access control is inadequate for dynamic LLM-based agents. Introduces Agent Access Control (AAC), which treats access control as context-aware information-flow governance across identity, relationships, scenarios, and norms, and produces adaptive responses (redaction, summarization, paraphrasing) beyond binary allow/deny. Authors include Xinfeng Li, Dong Huang, XiaoFeng Wang, and Yang Liu. Teach: Public arXiv preprint; useful as a survey-style framing reading to open an agent access-control unit.

• Secure and Efficient Access Control for Computer-Use Agents via Context Space
  _{arXiv · 2025-09 · arXiv:2509.22256} Presents CSAgent, a context- and intent-aware, static policy-based access-control framework for LLM-based computer-use agents, enforcing least-privilege over the agent's action/context surface across API, CLI, and GUI interfaces. Reports defending against all attacks in its benchmarks with ~1.99% performance overhead and ~5.42% utility decrease. Authors: Haochen Gong, Chenxiao Li, Rui Chang, Wenbo Shen. Teach: Public arXiv preprint; usable to contrast context-based enforcement (CSAgent) with static RBAC for computer-use agents.

• Agentic AI Identity and Access Management: A New Approach (CSA)
  _{Cloud Security Alliance · 2025-08-18} CSA's purpose-built Agentic AI IAM framework for autonomy, ephemerality, and delegation in multi-agent systems (MAS), using Decentralized Identifiers (DIDs), Verifiable Credentials (VCs), and Zero Trust (never trust, always verify) to handle secure delegation, policy-based access control, and real-time monitoring. Also introduces an Agent Name Service (ANS) for capability-aware discovery and a unified session-management layer. Teach: Public CSA framework document; a blueprint suitable for a DID/VC-based agent-identity design exercise.

• Agentic AI Identity and Access Management: A New Approach
  _{CSA · 2025-08-18} CSA research artifact (published Aug 18 2025) addressing the 'identity explosion' of large numbers of agent identities and the inadequacies of OAuth 2.1 for representing human→agent→sub-agent delegation chains within the original authorization scope. Core reading for agentic IAM design in multi-agent systems. Teach: Free downloadable CSA artifact; delegation-chain and scoped-authorization concepts are directly teachable design problems.

• SAMEP: A Secure Protocol for Persistent Context Sharing Across AI Agents
  _{arXiv · 2025-07-05 · arXiv:2507.10562} Hari Masoor proposes SAMEP, a protocol for persistent, secure memory sharing across agents using vector semantic search, AES-256-GCM encryption, access controls, and audit capabilities. Addresses shared-memory provenance/poisoning and cross-session context. Teach: Public protocol spec; teachable as a secure persistent-context design with authentication and integrity controls.

• Improving Google A2A Protocol: Protecting Sensitive Data and Mitigating Unintended Harms in Multi-Agent Systems
  _{arXiv · 2025-05 · arXiv:2505.12490} By Louck, Stulman, and Dvir. Identifies A2A weaknesses handling sensitive data (payment/identity data exposure) and proposes protocol enhancements — explicit consent orchestration, ephemeral scoped tokens, and direct user-to-service data channels — shown to reduce data leakage while maintaining low latency. Central to A2A-protocol-security teaching. Teach: Yes — concrete protocol gaps and mitigations mappable to hands-on A2A labs.

• Progent: Securing AI Agents with Privilege Control
  _{arXiv (UC Berkeley — Shi, He, Wang, ... Wenbo Guo, Dawn Song) · 2025-04 · arXiv:2504.11703} Privilege-control mechanism for LLM agents enforcing least-privilege via a policy language of symbolic rules over tool calls, with LLM-assisted dynamic policy generation and an SMT solver that auto-applies narrowing policy changes but requires approval for expanding ones (effective action space can only shrink without approval). Strongly reduces attack success on AgentDojo and ASB while preserving utility; integrates with LangChain and OpenAI Agents SDK. (Retitled from the original 'Progent: Programmable Privilege Control for LLM Agents'.) Teach: Yes — the policy DSL and modular wrapper design (no agent-internal changes) make it a strong teaching example for least-privilege / action-gating labs; evaluated on public benchmarks AgentDojo and ASB.

• OpenID Connect for Agents (OIDC-A) 1.0 Proposal
  _{OpenID (community proposal) · 2025-04-28} A community proposal (April 28, 2025) extending OpenID Connect Core 1.0 to represent, authenticate, and authorize LLM-based agents: standard claims, endpoints, agent attestation, delegation-chain representation, and capability-based authorization within the OAuth 2.0 ecosystem. Teach: Public written spec proposal; concrete claim/endpoint definitions make it a good lab basis for building an agent-aware OIDC token flow.

• Building A Secure Agentic AI Application Leveraging A2A Protocol
  _{arXiv · 2025-04 · arXiv:2504.16902} By Habler, Huang, Narajala, and Kulkarni. Applies the MAESTRO framework to Google's Agent2Agent (A2A) protocol, doing threat modeling focused on Agent Card management, task-execution integrity, and authentication methodologies, and offers architectural guidelines for secure A2A systems (including MCP integration). A foundational A2A-security reference. Note: actual title omits 'Google's'. Teach: Yes — protocol is open (A2A spec) and the paper's MAESTRO-on-A2A analysis is reproducible in a lab.

• Authenticated Delegation and Authorized AI Agents
  _{arXiv · 2025-01 · arXiv:2501.09674} Proposes a framework for authenticated, auditable delegation of authority to AI agents by extending OAuth 2.0 and OpenID Connect with agent-specific credentials and delegation metadata, while remaining compatible with existing web infrastructure. Also introduces methods for converting natural-language permissions into auditable access controls, enabling scoped, revocable, on-behalf-of authorization. Authors include Tobin South, Samuele Marro, Thomas Hardjono, Alan Chan, and Alex Pentland. Teach: Public arXiv preprint with a concrete token/credential design over OAuth+OIDC; directly implementable as a delegated-authorization lab.

• OWASP Non-Human Identities Top 10 (2025)
  _{OWASP · 2025} OWASP's ranked list of the ten most critical security risks for non-human identities: NHI1 Improper Offboarding, NHI2 Secret Leakage, NHI3 Vulnerable Third-Party NHI, NHI4 Insecure Authentication, NHI5 Overprivileged NHI, NHI6 Insecure Cloud Deployment Configurations, NHI7 Long-Lived Secrets, NHI8 Environment Isolation, NHI9 NHI Reuse, and NHI10 Human Use of NHI. Agentic guidance ties these directly to AI-agent credential risks. Teach: Public OWASP project with a ranked risk catalog; directly usable as a labbed threat-modeling checklist for agent credentials.

---

Governance & Assurance

39 sources.

• ENISA's View on Cybersecurity in the Frontier AI Era
  _{ENISA · 2026-07} ENISA report (published 2026-07-07) providing national competent authorities, EU policymakers, defenders and service providers an initial set of recommendations for securing frontier AI, covering the evolving AI threat landscape including agentic and supply-chain risk considerations for the EU. Teach: Public ENISA report; teachable as the EU regulator's framing of AI/agentic supply-chain risk.

• Securing Agentic AI — An Addendum to the Guidelines and Companion Guide on Securing AI Systems
  _{Cyber Security Agency of Singapore · 2026-06-17} Singapore's Cyber Security Agency addendum (published 17 June 2026) extending its October 2024 Guidelines and Companion Guide on Securing AI Systems specifically to agentic AI, giving a national-authority companion to the CSA/OWASP/ENISA agentic guidance wave (note: this 'CSA' is Singapore's Cyber Security Agency, distinct from the Cloud Security Alliance). Teach: Free official publication; teachable as a national-regulator perspective to contrast with CSA/OWASP industry guidance.

• The Non-Human Identity Governance Vacuum: AI Agents and the Fastest-Growing Unmanaged Attack Surface (CSA whitepaper)
  _{Cloud Security Alliance (AI Safety Initiative) · 2026-05-20} CSA whitepaper framing NHI governance as the defining security gap of the agentic-AI era, with survey data: only 15% of organizations are confident they can prevent NHI-based attacks, 47% of NHIs go unchanged for over a year, 16% don't track new AI-related identity creation, and NHIs outnumber humans by 45:1 to 144:1. Recommends NHI registries, zero standing privilege, credential rotation, and extending governance to third-party integrations. Teach: Public whitepaper with survey data; useful for motivating the governance problem and grounding class discussion in evidence.

• Human-Certified Module Repositories for the AI Age
  _{arXiv · 2026-03-03 · arXiv:2603.02512} Proposes human-certified module/package repositories to re-establish trust and provenance for code and AI components consumed by agents, a governance response to AI-era dependency risk. Teach: Public position/design preprint (Szilárd Enyedi); discussion basis for certified-repository governance design.

• CSA Research Note — NIST AI Agent Security: Red-Teaming Guidance and Enterprise Compliance
  _{CSA · 2026-03} Cloud Security Alliance research note (published March 31, 2026) analyzing NIST's emerging AI-agent security guidance — including COSAiS (Control Overlays for Securing AI Systems) SP 800-53 overlays, red-teaming findings (novel agent attacks reaching ~81% task-hijack success vs ~11% for baselines), and NCCoE authorization architecture (OAuth 2.0, SPIFFE/SPIRE). Bridges research to standards and enterprise audit/compliance. Teach: Freely available research note that can anchor a governance/compliance module linking NIST agent-security guidance to enterprise controls.

• Agent Identity Governance Framework (CSA Labs, v1 draft)
  _{Cloud Security Alliance (Labs) · 2026-03-27} CSA Labs framework (published as a draft) positioning delegation and scoping as dynamic governance primitives with continuous enforcement and auditability. It defines five agent identity categories (copilot, autonomous, orchestrator, ephemeral sub-agent, Agent-as-a-Service), a just-in-time intent-declared access model replacing standing privileges, and evaluates OAuth 2.0/OIDC, SPIFFE/SPIRE, and SCIM; maps controls to the CSA AI Controls Matrix, OWASP Agentic Security Initiative, NIST AI RMF, and MAESTRO. Teach: Public framework with concrete technical patterns (JIT intent-declared grants, SPIFFE, per-hop scope attenuation, immutable audit trails); strong basis for a delegation-chain and scope-attenuation lab. Note the live title is 'Agent Identity Governance Framework' and it is marked Draft.

• The Agentic Trust Framework: Zero Trust Governance for AI Agents (CSA)
  _{CSA (Cloud Security Alliance) · 2026-02-02} CSA's open governance specification applying Zero Trust principles to autonomous AI agents via five core elements and a maturity model (Intern through Principal levels). Part of CSA's broader agentic-AI assurance body of work, which also includes the AI Controls Matrix (AICM v1.1, mapping to ISO 42001/27001, NIST AI RMF, BSI AIC4) and the STAR for AI self-assessment program — though those are covered in separate CSA publications, not this blog post. Teach: AICM spreadsheet and STAR for AI self-assessment are publicly available and usable as hands-on control-mapping labs; this specific post covers the Agentic Trust Framework and maturity model.

• Singapore Launches New Model AI Governance Framework for Agentic AI (IMDA)
  _{IMDA (Singapore government) · 2026-01-22} National governance framework specific to agentic AI, announced at Davos, structured around dimensions including bounding risk, meaningful human accountability, technical controls/processes, and end-user responsibility, with emphasis on verifiable agent identity and audit trails of which agent acted under whose authorization. Teach: Public government framework document; excellent teachable blueprint for agent identity + accountability requirements. Verify the claimed 'May 2026 multi-agent case-study update' separately before citing it.

• Audit Trails for Accountability in Large Language Models
  _{arXiv · 2026-01 · arXiv:2601.20727} Defines an audit trail as a chronological, tamper-evident, context-rich ledger of lifecycle events and decisions linking technical provenance (models, data, training/eval runs, deployments, monitoring) with governance records (approvals, waivers, attestations). Contributes an event-category framework, a reference architecture (lightweight emission, immutable storage, auditor interfaces), and reusable open-source Python for low-effort integration. Teach: Public preprint (Ojewale, Suresh, Venkatasubramanian) with a concrete audit-trail schema plus open-source code; teachable as the specification for building an agent audit log.

• With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systems
  _{arXiv · 2025-12 · arXiv:2512.22211} Introduces the Agentic Risk & Capability (ARC) Framework, which ties an agent's capabilities to corresponding governance responsibilities. Identifies three primary risk sources (components, design, capabilities), connects risks to technical controls, and offers practical implementation guidance to scale oversight with capability level. Accepted at IASEAI 2026 and the AAAI 2026 International AI Governance Workshop. Teach: Public preprint (Khoo, Foo, Lee) offering a capability-to-control mapping usable as a governance-design exercise.

• OWASP Top 10 for Agentic Applications
  _{OWASP GenAI Security Project · 2025-12} OWASP GenAI Security Project's framework extending the Top-10 approach to autonomous agent systems, covering excessive agency, tool misuse, memory/context poisoning, and indirect prompt injection amplified by agent action-taking (privilege escalation, lateral movement, persistence). Published under the project's Agentic App Security initiative. Teach: Yes. Public framework from OWASP; usable as a governance/threat-model checklist in class.

• AIAuditTrack: A Framework for AI Security System
  _{arXiv · 2025-12 · arXiv:2512.20649} Proposes a blockchain-based framework for recording and governing AI usage traffic. Uses decentralized identity and verifiable credentials to establish trusted, identifiable AI entities and records their interactions on-chain, modeling entities as nodes in a dynamic interaction graph with a risk-diffusion algorithm to trace risky-behavior origins and propagate early warnings for audit and responsibility attribution in multi-agent environments. Teach: Public preprint (Luo, Fan, Li, Zhang, Lin, Wang) describing an on-chain audit-tracking framework; potential basis for a monitoring/audit lab.

• AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AI
  _{arXiv · 2025-12 · arXiv:2512.03180} Proposes an integrated end-to-end pipeline operationalizing risk management across three phases (design, runtime, audit) for LLM-based autonomous agents. Embeds attribution and auditability as core governance functions, mapping vulnerabilities to structured taxonomies and tying decisions to cryptographic tracking and organizational controls for measurable, auditable assurance across the agent lifecycle. Teach: Public preprint (Khan, Joyce, Habiba) describing an end-to-end assurance architecture; teachable as a reference design for auditable agents.

• Securing the Model Context Protocol (MCP): Risks, Controls, and Governance
  _{arXiv · 2025-11-25 · arXiv:2511.20920} A controls-and-governance treatment of MCP: formalizes a threat model with three adversary types (malicious content injection, supply-chain/compromised servers, and agents exceeding intended scope) and maps risks (data exfiltration, tool misuse, privilege escalation) to organizational controls — scoped auth, provenance tracking, sandboxing/monitoring, DLP policy enforcement, and centralized governance. Teach: Governance framework and control set are documented; teachable as a policy/controls module rather than hands-on labs.

• MAIF: Enforcing AI Trust and Provenance with an Artifact-Centric Agentic Paradigm
  _{arXiv · 2025-11 · arXiv:2511.15097} Introduces the Multimodal Artifact File Format (MAIF), an AI-native container embedding semantic representations, cryptographic provenance, and granular access controls. Uses persistent, verifiable artifacts with hash chains, digital signatures, stream-level access control, and real-time tamper detection to create an immutable, non-repudiable audit trail supporting EU AI Act compliance. Teach: Public preprint (Narajala et al.) with a concrete cryptographic provenance design; implementable as a hands-on tamper-evident-logging lab. Note: title corrected from '...Framework' to '...Agentic Paradigm'.

• AI Bill of Materials and Beyond: Systematizing Security Assurance through the AI Risk Scanning (AIRS) Framework
  _{arXiv · 2025-11-16 · arXiv:2511.12668} Argues AI-BOMs (Model Cards, SBOMs) advance provenance reporting but lack verifiable machine-readable security evidence, and proposes the AIRS framework mapping against SPDX 3.0 and CycloneDX 1.6 to close AI-specific gaps. A direct SBOM-for-AI reference. Teach: Published framework with explicit SPDX/CycloneDX crosswalk; strong basis for an AI-BOM assurance module.

• We Need Accountability in Human-AI Agent Relationships
  _{arXiv · 2025-10-24 · arXiv:2510.21967} Argues for accountability mechanisms in human-AI agent relationships to ensure alignment with user and societal interests, proposing that an agent's engagement be made conditional on appropriate user behavior via design strategies (distancing, disengaging, discouraging). Authors: Benjamin Lange, Geoff Keeling, Arianna Manzini, Amanda McCroskery. Teach: Conceptual framework in a public preprint; good discussion/seminar material rather than a lab.

• The Agentic AI Governance Framework: A Universal Model for Risk, Accountability, and Compliance in Autonomous Systems
  _{SSRN (Rajat Pandey) · 2025-10} A six-principled lifecycle governance model aimed at traceability, accountability, and regulatory alignment for continuously acting, multi-agent systems, explicitly positioned to fill implementation-depth gaps in NIST AI RMF and ISO/IEC 42001. Teach: Publicly downloadable SSRN paper; usable as a lifecycle-governance framework reading and mapping exercise. Note: also mirrored on Zenodo; SSRN publication date is not precisely confirmed.

• AURA: An Agent Autonomy Risk Assessment Framework
  _{arXiv · 2025-10-17 · arXiv:2510.15739} Proposes AURA, a framework to detect, quantify, and mitigate risks arising from agentic AI using a gamma-based scoring methodology, with Human-in-the-Loop oversight and Agent-to-Human communication mechanisms to enable large-scale, governable agentic AI in enterprise environments. Authors: Lorenzo Satta Chiris and Ayush Mishra (University of Exeter). Teach: Public preprint with a concrete scoring/assessment method; usable as a risk-tiering exercise in class.

• Governance-as-a-Service: A Multi-Agent Framework for AI System Compliance and Policy Enforcement
  _{arXiv · 2025-08 · arXiv:2508.18765} Proposes a modular, policy-driven runtime enforcement layer (Governance-as-a-Service) that regulates agent outputs at runtime without modifying model internals, positioning governance as infrastructure akin to compute or storage — an external control plane rather than model-internal alignment. Authors: Suyash Gaurav, Jukka Heikkonen, Jatin Chaudhary (submitted Aug 26, 2025). Teach: Partial — architecture (external policy control plane over multiple models) is teachable as a design pattern; enforcement-point placement makes a good lab discussion, though no full open release confirmed.

• AI Security Solutions Landscape for Agentic AI (Q3 2025)
  _{OWASP GenAI Security Project · 2025-08-07} Vendor-agnostic mapping of the agentic AI security solution landscape across the DevOps-SecOps intersection, monitoring the full agentic AI lifecycle to help organizations select controls against the agentic threat taxonomy. Confirmed on the OWASP GenAI resources page, dated Aug 7, 2025. Teach: Public reference doc; useful for a tooling/market-survey lecture rather than hands-on labs.

• GenAI Incident Response Guide 1.0
  _{OWASP · 2025-07-28} Guide (released 28 July 2025) from the OWASP GenAI Security Project CTI Initiative providing guidelines and best practices for detecting, responding to, and recovering from security incidents in GenAI and agentic applications, extending traditional IR practices to LLM/agent-specific failure modes for general security practitioners. Teach: Public PDF; teachable as an IR-playbook exercise for GenAI systems.

• Linux Foundation Launches the Agent2Agent Protocol Project
  _{Linux Foundation · 2025-06} Vendor-neutral governance home for the Agent2Agent (A2A) protocol, originally created by Google, announced June 23, 2025. Founding contributors include AWS, Cisco, Salesforce, SAP, Microsoft, and ServiceNow, with a security focus on unique agent identities and secure communication flows. The governance/standards context for A2A-protocol security. (Note: the press release does not itemize the specific eight-seat TSC composition described in the original summary.) Teach: Yes — open protocol spec and governance docs; A2A reference implementation is usable in labs.

• ENISA Advisory Group Opinion Paper — On the Role of ENISA in Cybersecurity for AI
  _{ENISA · 2025-06} 2025 ENISA Advisory Group position (PDF posted July 2025) recommending ENISA support organizations receiving EU AI Act notices, build a structured AI-vulnerability reporting/disclosure process, and publish AI cybersecurity baselines and metrics. Signals the direction of EU AI-security governance. Teach: Free public PDF; useful as a policy/governance reading on how a regulator frames AI-security assurance.

• Assurance of AI Systems (SRI-CSL Technical Report SRI-CSL-2024-02R3)
  _{SRI International (CSL) · 2025-06} SRI/John Rushby technical report laying out an assurance framework for AI systems that divides verification into intent, correctness, and innocuity tasks, connecting classical safety-assurance cases to AI. URL resolves to a live ~950KB PDF in Rushby's SRI-CSL paper directory (exact revision number/date not machine-verifiable from the binary, but the document and author are well established). Teach: Freely downloadable report from a leading formal-methods lab; strong reading for an assurance-case module.

• In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI
  _{arXiv · 2025-03 · arXiv:2503.16861} Argues internal evaluation is insufficient and calls for policy and infrastructure enabling robust third-party evaluation and coordinated flaw disclosure for general-purpose AI. Proposes three interventions: standardized AI flaw reports with researcher-engagement guidelines, broadly-scoped disclosure programs modeled on security practices, and infrastructure for coordinating flaw distribution across stakeholders. Teach: Public preprint (Longpre, Klyman et al.); strong policy-and-process reading for teaching third-party evaluation and disclosure regimes.

• NIST — Strengthening AI Agent Hijacking Evaluations (CAISI) & related SP 800-53 control overlays / adversarial-ML taxonomy
  _{NIST · 2025-01} NIST/CAISI technical blog (published Jan 17, 2025) on agent-hijacking evaluations, covering indirect prompt injection, adaptive testing, task-specific risk analysis, and the value of multiple attack attempts; connects to NIST's broader adversarial-ML (AI 100-2) taxonomy and SP 800-53 control-overlay work for AI systems. Teach: Public NIST blog, AML taxonomy, and control-overlay drafts; teachable for governance/assurance and evaluation-methodology modules.

• Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1 2pd, Second Public Draft)
  _{NIST · 2025-01} US AI Safety Institute (NIST) voluntary guidance on identifying, measuring, and mitigating misuse risks of dual-use foundation models across the lifecycle; second public draft (NIST AI 800-1 2pd) released January 2025. DOI 10.6028/NIST.AI.800-1.2pd. Feeds into control overlays as a technical foundation. Teach: Public draft with lifecycle-based misuse-mitigation practices; teachable as a governance/assurance framework, though still draft.

• The NIST Assessing Risks and Impacts of AI (ARIA) Pilot Evaluation Plan
  _{NIST · 2024-08} NIST ARIA 0.1 Pilot Evaluation Plan (last updated Aug 16, 2024) defining the Contextual Robustness Index (CoRIx) and a three-tier evaluation hierarchy: model testing, adversarial red teaming, and field testing. Establishes a government evaluation template for assessing AI risks and impacts. (Note: the ~51 red teamers / 508 sessions figures come from the later ARIA results reporting, not this forward-looking plan document.) Teach: Public evaluation plan and metric (CoRIx) that can frame a lecture on structured, tiered AI red-team programs and government assurance methodology.

• Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)
  _{NIST · 2024-07-26} Companion profile to the AI RMF defining 12 GenAI-specific risk categories (including Information Security, Data Privacy, CBRN, Confabulation) with 200+ suggested actions. It is the primary GenAI risk-taxonomy that agentic security controls map to. Teach: Public PDF with 12 named risk categories and 200+ actionable mitigations; directly usable as a labeled rubric for classroom risk-mapping exercises.

• Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1
  _{NIST · 2023-01-26} The foundational voluntary NIST framework (Govern/Map/Measure/Manage functions) that all downstream NIST AI security profiles and overlays build on. It is the anchor document for U.S. AI risk governance and the base layer for agentic-AI security work. Teach: Public spec with a full playbook and crosswalks; easy to teach as the governance backbone of an agentic-AI security course.

• OpenSSF AI/ML Security Working Group (incl. E2E Model Lifecycle Provenance)
  _{OpenSSF / Linux Foundation · 2025} OpenSSF AI/ML Security Working Group coordinating open-source AI supply-chain security efforts along two axes ('security for AI' and 'AI for security'), with deliverables including Model Signing, an End-to-End Model Lifecycle Provenance framework, and GPU-based model integrity/attestation. Central hub for model provenance standards. Teach: Public working-group charter and deliverables; teachable as the open-source landscape for AI supply-chain provenance.

• Linux Foundation AI-Security Training (LFEL1012 Secure AI/ML-Driven Software Development; LFS120 Conversational AI: Ensuring Compliance and Mitigating Risks)
  _{Linux Foundation · 2025} Linux Foundation's AI-security-adjacent training: LFEL1012 'Secure AI/ML-Driven Software Development' (an Express Learning course on secure use of AI assistants in software development) and LFS120 'Conversational AI: Ensuring Compliance and Mitigating Risks' (conversational AI risk mitigation aligned to current standards). Both confirmed in the LF Training catalog; training courses rather than proctored certifications, part of LF's 2025-2026 AI security/risk catalog. Teach: Structured LF Training courses with published objectives; teachable modules, though currently training rather than exam-backed certification.

• ISACA Advanced in AI Audit (AAIA)
  _{ISACA · 2025} Billed by ISACA as the world's first advanced AI audit certification; validates AI Governance & Risk, AI Operations, and AI Auditing Tools & Techniques. Requires active CISA (or designated accounting/audit credentials), making it the audit/assurance counterpart to AAISM for agentic and enterprise AI systems. Teach: ISACA publishes domain weightings and exam outline; teachable as an AI-audit module though it is assessment-focused rather than lab-based.

• CoSAI Software Supply Chain Security for AI Systems (OASIS Coalition for Secure AI)
  _{OASIS / CoSAI · 2025} OASIS-hosted industry coalition whose 'Software Supply Chain Security for AI Systems' workstream (alongside 'Secure Design Patterns for Agentic Systems', 'AI Security Risk Governance', and 'Preparing Defenders for a Changing Security Landscape') produces open guidance and tooling for AI supply-chain provenance and integrity. Teach: Public workstream repos and best-practice docs on GitHub; teachable as an industry-consensus supply-chain framework.

• Center for AI Standards and Innovation (CAISI)
  _{NIST · 2025} The NIST center (successor framing to the US AI Safety Institute) serving as industry's primary US-government point of contact for AI testing and collaborative research. It produces agent hijacking evaluations, red-teaming research (e.g., large-scale red-teaming competitions, 'cheating on AI agent evaluations'), and guidance/best practices on agentic AI security. Institutional home of NIST's 2025 agentic-security agenda. Teach: Public program page and research/RFI documents; teachable as context for how U.S. federal AI-security standards are being shaped.

• CSA Secure Agentic System Design (Trait-Based Approach) and MCP Six Pillars (in progress)
  _{Cloud Security Alliance · 2025} CSA AI Safety Working Group outputs. 'Secure Agentic System Design: A Trait-Based Approach' is published with a download link on CSA's AI research page, and 'MCP Six Pillars' is an in-progress whitepaper under active bi-weekly working-group development, addressing agentic and MCP security and trust design. Teach: Public CSA whitepapers and peer-review drafts; teachable as vendor-neutral agentic-security design guidance.

• CSA STAR for AI Program
  _{Cloud Security Alliance · 2025} CSA certification/attestation program letting organizations demonstrate AI trustworthiness via self-assessment against the AI Controls Matrix (AICM: 243 control objectives across 18 domains, mapped to ISO 42001, ISO 27001, NIST AI RMF, BSI AIC4). Confirmed on the AICM artifact page, which describes STAR for AI as an upcoming CSA certification for organizations, with self-assessments (AI-CAIQ) submitted to the STAR Registry. Organizational (not individual) assurance analogous to CSA STAR for cloud, anchoring the 2025-2026 AI assurance ecosystem. Teach: Public assessment framework tied to AICM; teachable as an assurance/attestation exercise using the free control set.

• IAPP Artificial Intelligence Governance Professional (AIGP)
  _{IAPP · 2024} IAPP's AI governance credential, confirmed on iapp.org, covering AI systems foundations, responsible AI principles, AI law/regulation (incl. EU AI Act), the AI lifecycle, and risk-management/governance implementation. Governance-focused but foundational for the security-adjacent AI risk and compliance workforce; continues to expand through 2025-2026. Teach: Published body-of-knowledge and exam blueprint; teachable as a governance/compliance module though not security-lab based.

---

Formal Verification

15 sources.

• AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents
  _{arXiv · 2026-07 · arXiv:2607.02599} Uses Linear Temporal Logic to score an agent trace's temporal structure with a deterministic metric over ordering, call presence, and trace grounding, enabling measurement, enforcement, and training of procedural compliance. By Laila Elkoussy and Julien Perez (LRE, EPITA; submitted July 1, 2026). Teach: LTL-over-traces with a deterministic metric is a clean lab (convert policy to LTL, verify traces); framework is public.

• Securing LLM-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guarantees
  _{arXiv · 2026-06-23 · arXiv:2606.24322} Yedidel Louck formalizes memory 'malleability' (laundering via summarization, trusted-tool echoes, manufactured corroboration), proves content/lineage defenses unsound, and proposes TMA-NM: non-malleable, origin-bound information-flow control over agent memory. Achieves 0% attack success across eight frontier models vs up to 68% for prior defenses. Teach: Public preprint with formal theorems; teachable as a memory-integrity design pattern grounded in non-malleable information-flow control.

• Runtime Compliance Verification for AI Agents
  _{arXiv · 2026-06 · arXiv:2606.19242} Presents C-Trace, a framework that monitors AI agents at runtime to ensure compliance with policy/regulatory requirements (e.g., GDPR consent, purpose limitation, data minimization, right to erasure), evaluated against adversarial extraction prompts. By Nafiseh Kahani, Masoud Barati, Diana Addae (submitted June 17, 2026). Teach: Public paper framing runtime compliance monitoring; suitable as a survey/reading item for a runtime-verification lecture.

• MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
  _{arXiv · 2026-05 · arXiv:2605.06334} Framework that auto-synthesizes machine-checkable compliance benchmarks from natural-language manuals and tool schemas by generating a symbolic world model plus trace-level compliance checks and validating consistency with SMT solving. Produces a suite of 285 tasks across 6 domains, scaling reliable procedural-compliance evaluation for agents. Teach: SMT-validated benchmark-generation pipeline; teachable as formal-verification-of-agent-behavior module.

• Containment Verification: AI Safety Guarantees Independent of Alignment
  _{arXiv · 2026-05 · arXiv:2605.09045} Instead of verifying the model, it deductively verifies the agentic framework (harness) mediating between model and external state, modeling the AI as an unconstrained oracle and proving the containment layer enforces boundary policies for all actions; instantiated on the PocketFlow minimal LLM framework and mechanized in Dafny. Authors: Royce Moon, Lav R. Varshney (submitted May 9, 2026). Teach: Strong conceptual framing (verify the harness, not the model); the Dafny/PocketFlow instantiation pairs well with a deductive-verification-of-wrappers exercise.

• Causal Past Logic for Runtime Verification of Distributed LLM Agent Workflows
  _{arXiv · 2026-05 · arXiv:2605.20923} Introduces a causal-past temporal logic (extending the ZipperGen framework) tailored to runtime verification of distributed, multi-agent LLM workflows where ordering and causality across asynchronous agents matter rather than a single sequential log. Single-author paper by Benedikt Bollig (submitted May 20, 2026). Teach: Logic definition is public; good advanced reading connecting distributed-systems causality to agent verification.

• Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Proving
  _{arXiv · 2026-04 · arXiv:2604.01483} Translates natural-language institutional/financial policies into Lean 4 and uses theorem proving (including SMT tactics) to give deterministic, type-checked guardrails that block policy-violating agent actions before execution. By Devakh Rashie and Veda Rashi (arXiv Logic in Computer Science, submitted April 1, 2026). Teach: Lean 4 is open and well-documented; the financial-compliance case study makes a concrete provable-guardrails lab.

• Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agents
  _{arXiv · 2026-03 · arXiv:2603.20449} An SMT solver-aided framework that converts natural-language tool-use policies into formal logic constraints discharged by the Z3 solver, then intercepts and validates tool calls at runtime. Evaluated on TauBench, it reduces policy violations while preserving task accuracy, giving stronger formal expressiveness than prompt- or filter-based enforcement. Teach: SMT-LIB + Z3 workflow is directly teachable with open tooling; a hands-on 'encode a tool policy and prove compliance' lab is feasible.

• FormalJudge: A Neuro-Symbolic Paradigm for Agentic Oversight
  _{arXiv · 2026-02 · arXiv:2602.11136} Neuro-symbolic oversight framework that converts natural-language requirements into formal specifications via Dafny and Z3 to judge LLM-agent behavior with mathematical certainty rather than probabilistic LLM-as-Judge scores; reports ~16.6% improvement over LLM-as-Judge baselines and shows a 7B model detecting deception from a 72B model at >90% accuracy. Authors: Jiayi Zhou, Yang Sheng, Hantao Lou, Yaodong Yang, Jie Fu (submitted Feb 11, 2026). Teach: Neuro-symbolic judging pattern is public; excellent compare/contrast against LLM-as-judge, and the Dafny/Z3 spec-generation step is a concrete class demo.

• Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Autonomous AI Agents
  _{arXiv · 2026-02 · arXiv:2602.22302} Introduces formally specified behavioral contracts for autonomous agents plus a runtime enforcement mechanism, bridging design-time specification and execution-time monitoring for reliability. Single-author paper by Varun Pratap Bhardwaj (arXiv cs.AI, submitted February 25, 2026). Teach: Contract-based design maps cleanly to a design-by-contract teaching module; public paper with a specification schema.

• VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation
  _{arXiv · 2025-10 · arXiv:2510.05156} A dual-stage approach that (offline) clarifies user intent, synthesizes a behavioral policy as executable code from safety specifications and formally verifies it, then (online) monitors proposed agent actions against the pre-verified policy before execution, giving provable-by-construction guardrails. Teach: Public spec of the offline-verify + online-monitor pattern; the verified-code-generation pipeline is reproducible as a lab.

• Formalizing the Safety, Security, and Functional Properties of Agentic AI Systems
  _{arXiv · 2025-10 · arXiv:2510.14133} Introduces a unified modeling framework (host-agent model plus task-lifecycle model) that formally defines 30 properties (16 host-agent, 14 task-lifecycle) in temporal logic to verify agentic AI behavior, detect coordination issues, and prevent vulnerabilities across multi-agent protocols. Teach: Public arXiv paper with an explicit property taxonomy and modeling framework; directly usable as a lecture backbone for a formal-methods-for-agents unit.

• AgentGuard: Runtime Verification of AI Agents
  _{arXiv (ASE 2025 Agentic Software Engineering workshop) · 2025-09 · arXiv:2509.23864} Runtime verification framework (by Roham Koohestani) providing 'Dynamic Probabilistic Assurance': an inspection layer instruments agent inputs/outputs as formal events, online-learns a Markov Decision Process model of agent behavior, and applies probabilistic model checking to give quantitative real-time safety assurance. Bridges formal runtime verification with agent action gating. Teach: Yes — the middleware instrumentation plus MDP/online-learning + probabilistic-model-checking monitor design is concrete enough to reproduce in a class exercise on runtime monitors for tool-calling agents.

• ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety
  _{arXiv — Wang, Poskitt, Wei, Sun · 2025-08 · arXiv:2508.00500} Proactive runtime monitoring framework that models agent behavior as a Discrete-Time Markov Chain learned from execution traces, estimates the probability of reaching unsafe states at runtime, and triggers intervention when risk exceeds a threshold — anticipating violations rather than reacting. Predicts AV collisions/traffic violations up to ~38.7s ahead and reduces unsafe embodied-agent behavior by up to 65%. (Retitled from 'Proactive Runtime Enforcement of LLM Agent Safety via Probabilistic Model Checking'.) Teach: Partial — reproducible with standard probabilistic tooling (DTMC learning, PRISM/Storm-style checking), suitable for an advanced formal-methods-meets-agents lab; no packaged framework release noted.

• LogicGuard: Improving Embodied LLM Agents through Temporal Logic based Critics
  _{arXiv · 2025-07 · arXiv:2507.03293} Converts LTL formulas into Buchi automata to algorithmically check whether an embodied agent's trajectory satisfies a temporal specification, using the automaton as a critic to steer behavior. By Anand Gokhale, Vaibhav Srivastava, Francesco Bullo (submitted July 4, 2025; revised September 2025). Teach: LTL-to-Buchi-automata construction is a classic teachable algorithm now applied to agents; reproducible critic-loop lab.

---

Standards & Bodies

23 sources.

• Announcing the 'AI Agent Standards Initiative' for Interoperable and Secure Innovation (NIST)
  _{NIST · 2026-02-17} US federal program dedicated to interoperability and security standards for agentic AI, organized around three pillars: industry-led standards facilitation, open-source interoperability protocols, and research on AI agent security and identity (ITL is producing an 'AI Agent Identity and Authorization Concept Paper'). Aims for agents that can act securely on behalf of users and interoperate across the digital ecosystem. Teach: Public NIST announcement plus associated concept-paper work; strong for teaching the emerging federal governance baseline for agents. Note: specific references to MCP/A2A protocols and a 'Q4 2026 Interoperability Profile' were not stated in the announcement itself and should be verified against NCCoE materials before teaching as fact.

• Accelerating the Adoption of Software and AI Agent Identity and Authorization (NIST NCCoE concept paper)
  _{NIST NCCoE · 2026-02-05} NIST NCCoE concept paper addressing the enterprise gap where AI agents are treated as generic service accounts, organizing work around Identification, Authorization (OAuth 2.0 / policy-based access control), Access Delegation (linking users to agents for accountability), and Logging/Transparency. Teach: Public NIST concept paper (PDF hosted at nccoe.nist.gov, also referenced on CSRC); authoritative government reading with clear focus areas to structure a course module.

• AI Agent Standards Initiative (NIST CAISI)
  _{NIST (Center for AI Standards and Innovation, CAISI) · 2026-02-17} NIST Center for AI Standards and Innovation (CAISI) initiative for a trusted, interoperable, and secure agentic frontier, organizing agent-security work across three strategic focus areas: facilitating industry-led standards, fostering community-led open-source protocols, and investing in foundational research on agent authentication and security evaluations. Includes an RFI on AI agent security and listening sessions. Teach: Public standards-body program page; good anchor for discussing the emerging US federal agent-identity/security standards landscape and open calls for participation.

• OWASP Top 10 for Agentic Applications – The Benchmark for Agentic Security in the Age of Autonomous AI
  _{OWASP · 2025-12-09} OWASP GenAI Security Project's benchmark risk list for agentic applications (goal hijacking, tool misuse, identity abuse, memory poisoning, cascading failures), built through extensive community collaboration and accompanied by a Threats and Mitigations 1.1 taxonomy update and lifecycle guidance. The 'State of Agentic AI Security and Governance' report is a related but separate deliverable of the broader Agentic Security Initiative. Teach: Fully public spec and threats-and-mitigations docs; directly usable for labs and coursework on agent security. Verify the separate maturity-model/State-of report links independently when assigning them.

• OWASP Top 10 for Agentic Applications for 2026
  _{OWASP · 2025-12} Globally peer-reviewed OWASP GenAI Security Project framework (published Dec 9, 2025, 100+ contributors) enumerating the most critical security risks facing autonomous/agentic AI systems, with mitigations spanning agent design, memory, planning/autonomy, tool use, and deployment. Directly applicable to browser and computer-use agents. Teach: Public, vendor-neutral standard with threat taxonomy and mitigations; a ready-made curriculum backbone and mapping reference.

• OWASP Top 10 for Agentic Applications / Agentic AI Threats and Mitigations (OWASP GenAI Security Project, Agentic Security Initiative)
  _{OWASP · 2025-12-09} Community standard: a globally peer-reviewed (100+ contributors) reference of agentic threats and mitigations structured around Agent Design, Memory, Planning & Autonomy, Tool Use, and Deployment/Operations; the Top 10 for Agentic Applications was released Dec 9, 2025, extending the GenAI/LLM Top 10. Teach: Free public PDFs (Top 10, Threats & Mitigations, Practical Guide to Securing Agentic Applications); ready-made curriculum with labs and checklists.

• OWASP Top 10 for Agentic Applications (2026)
  _{OWASP · 2025-12-09} Peer-reviewed Top 10 for autonomous/agentic AI, published December 9, 2025 (labeled for 2026), developed with 100+ experts: ASI01 Agent Goal Hijack, ASI02 Tool Misuse, ASI03 Identity & Privilege Abuse, ASI04 Agentic Supply Chain, ASI05 Unexpected Code Execution, ASI06 Memory & Context Poisoning, ASI07 Insecure Inter-Agent Communication, ASI08 Cascading Failures, ASI09 Human-Agent Trust Exploitation, ASI10 Rogue Agents. (Landing/download page confirmed real and dated 2025-12-09; the exact ASI category names come from the downloadable PDF, not the landing page text.) Teach: Public spec with real-world exploit examples (EchoLeak, Amazon Q, GitHub MCP, AutoGPT RCE, Gemini memory attack) — excellent case-study teaching material.

• MITRE ATLAS — Agentic AI techniques update (with Zenity Labs collaboration)
  _{MITRE · 2025-11} Living, ATT&CK-style knowledge base of adversary tactics and techniques against AI systems, including 2025 agentic-AI additions (context/memory poisoning, agent config tampering, credential harvesting, exfiltration via agent tool invocation) developed in part with external collaborators such as Zenity Labs. Homepage confirmed live; exact version number and tactic/technique counts should be re-verified against the site's matrix. Teach: Public ATT&CK-style matrix with case studies; teachable for mapping observed agent attacks to a shared technique taxonomy.

• CSA AI Controls Matrix (AICM) v1.1
  _{Cloud Security Alliance · 2025-10-30} Vendor-agnostic control framework for cloud-based AI systems: 243 control objectives across 18 security domains, mapped to ISO 42001:2023, ISO 27001, NIST AI RMF 1.0 (NIST AI 600-1), BSI AIC4, and the EU AI Act. Initial release July 9, 2025; updated (v1.1) October 30, 2025. Built on CSA's Cloud Controls Matrix; underpins CSA's TAISE training and STAR for AI certification. Teach: Freely downloadable spec with implementation and auditing guidelines mapped to five AI roles; excellent blueprint for hands-on control-mapping labs.

• Control Overlays for Securing AI Systems (COSAIS) — concept paper
  _{NIST (NCCoE) · 2025-08} NIST NCCoE concept paper (Aug 2025) proposing SP 800-53-based control overlays tailored to securing AI systems, with agentic/AI-agent use cases among the planned overlay profiles. Developed as a community effort. (Project page returns 403 to automated fetchers but is live; concept paper confirmed from prior knowledge.) Teach: NIST intends public community-draft overlays; standards-body artifact suitable for policy/controls coursework as drafts are released.

• OAuth 2.0 Extension: On-Behalf-Of User Authorization for AI Agents (draft-oauth-ai-agents-on-behalf-of-user-01)
  _{IETF (Internet-Draft, Informational) · 2025-05-08} An Informational Internet-Draft by T. Senarath and A. Dissanayaka (WSO2) extending OAuth 2.0 so an AI agent can act on behalf of a user with explicit, scoped delegated authorization, distinguishing agent-autonomous action from user-delegated action. It introduces a 'requested_actor' parameter at the authorization endpoint and an 'actor_token' at the token endpoint. Directly relevant to scoped-token and delegation design. Teach: Public IETF draft with wire-level detail (requested_actor / actor_token parameters); teachable as a standards-track reading and token-flow implementation exercise. Note it is an individual WSO2 draft, not yet a working-group document.

• OWASP Top 10 for LLM Applications 2025 — LLM01:2025 Prompt Injection
  _{OWASP GenAI Security Project · 2024-11} The community-standard risk taxonomy keeping Prompt Injection at #1 (LLM01:2025) and covering direct vs. indirect injection. The 2025 edition (released November 2024) also added System Prompt Leakage (LLM07) and Vector/Embedding weaknesses (LLM08). The reference framing for prompt injection in application security programs. Teach: Yes. Free spec/PDF (OWASP-Top-10-for-LLMs-v2025) with mitigations; standard curriculum anchor.

• OWASP Top 10 for LLM Applications 2025
  _{OWASP · 2024-11-17} The flagship community-driven list of the ten most critical security risks for LLM applications (prompt injection, sensitive info disclosure, supply chain, data/model poisoning, improper output handling, excessive agency, system prompt leakage, vector/embedding weaknesses, misinformation, unbounded consumption). Released November 2024 despite the '2025' designation; the edition significantly expanded 'Excessive Agency' toward agentic use, making it the baseline reference for GenAI app security. Teach: Public spec (free PDF + web list at genai.owasp.org/llm-top-10). Widely used as a red-team checklist; mappable to labs like DeepTeam.

• NIST SP 800-218A: Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (SSDF Community Profile)
  _{NIST · 2024-07} Final NIST publication (finalized 2024-07-26) that acts as a companion to SSDF v1.1, extending it with AI-specific secure-development practices for model producers, system producers, and acquirers, including data/model provenance and supply-chain integrity across the AI SDLC. Teach: Public NIST profile; teachable as the baseline secure-development control set for AI supply chains.

• ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system
  _{ISO · 2023-12} The world's first certifiable AI management system (AIMS) standard, specifying requirements to establish, implement, maintain and improve governance of AI-based products/services. Published December 2023. Now the anchor certification (BSI, DNV, TUV SUD, NQA accredited by 2025) that agentic-AI security programs map controls back to. Teach: Published standard (paywalled full text) with public overview; certification bodies (BSI, DNV, ANAB) publish syllabi and PECB/APMG run public training courses usable as a class module.

• OpenSSF Model Signing (Sigstore model-transparency)
  _{OpenSSF / Sigstore · 2025} Sigstore/OpenSSF project providing cryptographic model signing and verification (Sigstore keyless, keys, certs, PKCS#11) with in-toto/DSSE bundles stored as JSON protobuf and transparency logging, giving verifiable model integrity and provenance across the ML supply chain. Teach: Public repo with CLI (model_signing sign/verify/digest) and Python API — ready-made hands-on model-provenance lab.

• OWASP Top 10 for LLM Applications 2025 - LLM04:2025 Data and Model Poisoning
  _{OWASP · 2025} OWASP LLM Top 10 (2025) entry on manipulation of pre-training, fine-tuning, and embedding data to insert backdoors, biases, and sleeper-agent behaviors, including RAG/embedding poisoning and supply-chain vectors, with ten mitigations and five attack scenarios. Teach: Public standard with definitions, attack scenarios, and mitigations; directly teachable.

• OWASP Top 10 for LLM Applications 2025 (incl. LLM06: Excessive Agency)
  _{OWASP · 2025} The 2025 LLM/GenAI risk list whose LLM06:2025 Excessive Agency entry directly addresses over-permissioned tool/function access and unsafe autonomous tool actions by LLM-based systems. Teach: Widely-used standard with a dedicated tool-agency risk; foundational reading and mapping exercise for any class.

• LLM08:2025 Vector and Embedding Weaknesses — OWASP Top 10 for LLM Applications 2025
  _{OWASP · 2025} OWASP Gen AI Security Project Top 10 entry (LLM08:2025) focused on RAG vector databases and embeddings, covering unauthorized data access, cross-context information leaks, embedding inversion attacks, data poisoning, and unintended behavior alterations. The standards-body home for RAG knowledge-base poisoning. Teach: Public standard with mitigations (access control, validation, monitoring, audit logs); teachable.

• ISO/IEC 27091 — Cybersecurity and privacy — Artificial intelligence — Privacy protection
  _{ISO · 2025} In-development ISO/IEC JTC 1/SC 27 companion draft to 27090 (DIS stage 2025) covering AI-specific privacy protection (membership inference, model inversion, data leakage). Rounds out the SC 42/SC 27 security+privacy pair that agentic-AI data-handling controls reference. Teach: Draft available via ANSI/ISO listings; teachable alongside 27090 as the privacy half of the AI-security standard pair.

• ISO/IEC 27090 — Cybersecurity — Artificial intelligence — Guidance for addressing security threats and failures in AI systems
  _{ISO · 2025} In-development ISO/IEC JTC 1/SC 27 guidance standard (ISO/IEC 27090, standard number 56581) extending the ISO 27000 family to AI-specific attack surfaces — adversarial ML, data poisoning, model theft, privacy/inversion attacks — describing detection and mitigation across the AI lifecycle. Reached FDIS stage in 2025. The emerging baseline for AI system security threats. Note: the official title uses 'security threats and failures in AI systems'. Teach: Draft publicly summarized (ISO/ANSI/BSI listings); informative 'guidance' structure makes it teachable as a threat taxonomy even before final publication.

• CycloneDX Machine Learning Bill of Materials (AI/ML-BOM)
  _{OWASP / CycloneDX · 2025} CycloneDX capability for AI/ML-BOMs representing datasets, models, and configurations with provenance tracking and risk assessment (bias, data integrity, model security) across the AI supply chain; supports transparency, accountability, and regulatory compliance, backed by the Authoritative Guide to AI/ML-BOM. Teach: Public spec plus authoritative guide; directly usable to generate and inspect AI/ML-BOMs in class.

• SPDX 3.0 AI Profile and Dataset Profile (AI-SBOM)
  _{Linux Foundation / SPDX · 2024} SPDX 3.0 adds AI and Dataset profiles enabling a machine-readable AI System Bill of Materials. Per the SPDX site, the AI profiles form a connected knowledge graph (not a flat list) capturing software dependencies, AI model architectures/versions, datasets (with Dataset Profiles documenting provenance/lineage/characteristics), versioned prompts, agent definitions, licensing, and ethical/security attributes. Teach: Public ISO/IEC-track SBOM standard (SPDX 3.0 released 2024) with AI/dataset profiles; usable to author and parse AI-SBOMs as a supply-chain knowledge graph.

---

Certifications

8 sources.

• ISACA Advanced in AI Security Management (AAISM)
  _{ISACA · 2025-08} ISACA's first and only AI-centric security management credential, launched 2025, covering AI Governance & Program Management, AI Technologies and Controls, and AI Risk Management. Requires an active CISM or CISSP, positioning it as the enterprise AI security manager cert. Teach: Public exam content outline / job practice domains are published by ISACA and map cleanly to a course syllabus; no hands-on labs but domain breakdown is teachable.

• Announcing the AI Controls Matrix and ISO/IEC 42001 Mapping — and the Roadmap to STAR for AI 42001
  _{CSA · 2025-08-20} August 20 2025 CSA release mapping AICM v1.0 controls to ISO/IEC 42001:2023 (with pointers to ISO/IEC 27001/27002 compensating controls) and laying out the roadmap to a STAR-for-AI / STAR-42001 certification program. Bridges the CSA control framework to the certifiable ISO standard. Teach: Public crosswalk mapping (AICM↔42001↔27001/27002) that is highly teachable as a compliance-alignment exercise.

• Trusted AI Safety Expert (TAISE) Certificate
  _{CSA / Northeastern University (partnership) · 2025} A professional certificate program (Trusted AI Safety Expert, TAISE) from the Cloud Security Alliance in partnership with Northeastern University, announced April 28, 2025 with launch later in 2025. It credentials people who build, manage, or audit AI systems to lead safe, secure, and responsible AI development and deployment across the AI lifecycle (governance, risk, privacy) — credentialing practitioners rather than certifying systems. Teach: A structured public training/certification curriculum (TAISE); directly relevant as a model for a course that credentials agent auditors.

• SANS SEC545: GenAI and LLM Application Security (GIAC AI Platform Security, GAIPS)
  _{SANS / GIAC · 2025} New 5-day SANS course with 20 hands-on labs on securing GenAI/LLM applications and LLM development pipelines, mapped to the new GIAC AI Platform Security (GAIPS) certification. Central to the 2025-2026 defensive AppSec-for-AI training wave. Teach: Course has 20 documented hands-on labs; GAIPS exam objectives are public, making the lab structure directly adaptable to a classroom.

• Practical DevSecOps Certified AI Security Professional (CAISP)
  _{Practical DevSecOps (vendor) · 2025} Hands-on vendor-neutral AI/LLM security certification (30+ guided labs, 36 CPE) covering LLM attacks, MITRE ATLAS, OWASP LLM Top 10, threat modeling AI systems with STRIDE, AI supply-chain (SLSA/SCVS), and governance (NIST RMF, ISO/IEC 42001, EU AI Act). Task-based exam: solve 5 challenges in 6 hours plus 24 hours for the report. Confirmed on vendor site with a public 7-chapter syllabus. One of the most lab-heavy independent AI security certs. Teach: Explicitly hands-on with 30+ labs, a task-based challenge exam, and a public 7-chapter syllabus; highly teachable and lab-oriented.

• GIAC AI Security Certification Family (GAIPS, GOAA, GASAE, GMLE)
  _{GIAC · 2025} GIAC's 2025 AI security certification line, all confirmed on giac.org with CyberLive hands-on testing: AI Platform Security (GAIPS, training SEC545), Offensive AI Analyst (GOAA, training SEC535), AI Security Automation Engineer (GASAE, training SEC598), and Machine Learning Engineer (GMLE, training SEC595). Represents a vendor-neutral, exam-backed credentialing wave distinct from OWASP. Teach: Each cert publishes exam objectives and is backed by hands-on CyberLive tasks, making the certification family's competencies directly mappable to labs.

• CSA Trusted AI Safety Expert (TAISE) / Trusted AI Safety Knowledge Certification Program
  _{Cloud Security Alliance (with Northeastern University) · 2025} CSA's AI safety and security certificate, developed with Northeastern University, spanning generative AI fundamentals, architecture, governance, risk management, privacy, and cloud security across the AI lifecycle. Vendor-neutral credential aimed at leading safe, secure, and responsible AI deployment. (Direct fetch of the training URL returned HTTP 403 bot-block this session, but TAISE is a documented, real CSA program.) Teach: Structured certificate program with published curriculum built jointly with a university; teachable as a full course, though more knowledge- than lab-based.

• AIUC-1: The World's First AI Agent Security Standard and Certification
  _{vendor (Artificial Intelligence Underwriting Company) · 2025} An auditable certification standard purpose-built for AI agents (positioned as 'SOC 2 for AI agents'), organized across six risk pillars (data & privacy, security, safety, reliability, accountability, society) with dozens of requirements and controls, quarterly updates, and mappings to ISO 42001, NIST AI RMF, and OWASP. Developed with 100+ Fortune 500 CISOs and contributors from Microsoft, Google Cloud, MITRE, Cisco, and Stanford; Schellman is the first accredited auditor. Teach: Public control framework and certification process; auditor writeups (e.g., Schellman) provide concrete audit-methodology material teachable as a case study. Note: specific claims about '51 requirements/130 controls' and Lloyd's of London insurance backing were not confirmable on the landing page and should be verified before citing.

---

Other

1 sources.

• Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures
  _{arXiv (accepted AAAI 2026, oral) · 2025-11-09 · arXiv:2511.06390} Introduces GhostSpec, fingerprinting model lineage via SVD of attention weight matrices to verify whether a released model is a fine-tune of an existing one, addressing model provenance and misrepresented origins. Relevant to model-provenance verification in the supply chain. Teach: Peer-reviewed (AAAI 2026 oral) model-provenance fingerprinting method usable as a lineage-verification lab.

---