Lab idea / Adversarial AI
Will an approval gate hold when the context changes?
Test authorization at the tool boundary using synthetic users and a mock downstream service.
Falsifiable hypothesis
An external approval and scope gate prevents unauthorized mock tool actions without blocking permitted clean tasks.
Source-led motivation
Official implementation guidance on MCP-specific security risks; the documentation explains confused-deputy risks in proxy authorization flows, while the draft specification additionally discusses token passthrough, SSRF, and state-handle authorization.
Research benchmark and environment for evaluating prompt-injection attacks and defenses against LLM agents, with configurable task suites, attacks, and defenses.
Authorized lab scope
A staging agent, synthetic documents, synthetic users, and a mock API with no real-world side effects.
Prerequisites
- A written authorization matrix and allowlisted mock actions.
- A no-attack baseline and fixed task set.
- Pinned protocol, agent, and model versions; synthetic credentials only.
Comparison & control
Compare clean tasks, tasks exposed to untrusted retrieved text, and the same tasks with an external gate. Do not rely on the agent’s own claim that an action was safe.
Observations to collect
- Actual allowed/denied actions at the mock API.
- Legitimate task completion and unnecessary approval requests.
- User identity, consent, and target scope for each proposed action.
Disconfirming evidence
- An out-of-scope action reaches the mock API, or clean-task completion collapses under the gate.
Stop conditions
- Stop if any action reaches an unapproved target, real user data, or a non-lab service.
- Stop on unexpected egress, exposed credentials, or an exhausted predeclared budget; retain a sanitized failure record.
Source bibliography
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents ↗
ETH Zurich and Invariant Labs research authors
Publication date: Date not supplied · Editorial check 2026-10-26 UTC
Project README / official documentation; source-led editorial check, no execution - Security Best Practices ↗
Model Context Protocol project
Publication date: Date not supplied · Editorial check 2026-10-26 UTC
Project README / official documentation; source-led editorial check, no execution