Research brief / AI-enabled offense / Adversarial AI
Put the permission boundary outside the agent
AI-assisted assessment should not ask the same model to propose an action and decide whether it is authorized.
The question
How can an assessment assistant remain useful without turning its suggestions into authority?
Source reports
LLM-assisted penetration-testing framework with an autonomous staged pipeline and a separate interactive legacy mode; the repository links the USENIX research paper.
Official MCP security guidance discusses confused-deputy risks in proxy authorization flows. Apply guidance to the deployed protocol version rather than assuming draft behavior.
CAI is archived: its README says no further releases, bug fixes, security patches, or support will be provided. It is a historical research artifact here, not a maintained deployment recommendation.
RedLatent interpretation
Treat the assistant as a proposal generator. Enforce target scope, identity, tool capability, and approval in a separate deterministic layer. An attractive natural-language rationale does not expand authorization.
Practical relevance
For a local lab, define permitted tools and destinations before a run. Use synthetic users and a mock downstream API to inspect consent and identity separation. Preserve denied requests as evidence: a safe denial is not a failed security control.
Limitations
- This is architectural interpretation, not a validated enforcement design.
- A human approval dialog alone can be ineffective if it hides the exact action or target. Pin protocol and agent versions; inspect actual downstream checks.
Proposed validation
When a tool request crosses user or target scope, does the external enforcement layer deny it consistently regardless of the assistant’s explanation?
A research question, not an executed experiment.
Source bibliography
- PentestGPT ↗
Gelei Deng and the PentestGPT authors / GreyDGL
Publication date: Date not supplied · Editorial check 2026-10-26 UTC
Project README / official documentation; source-led editorial check, no execution - Cybersecurity AI · CAI ↗
Alias Robotics
Publication date: Date not supplied · Editorial check 2026-10-26 UTC
Project README / official documentation; source-led editorial check, no execution - Security Best Practices ↗
Model Context Protocol project
Publication date: Date not supplied · Editorial check 2026-10-26 UTC
Project README / official documentation; source-led editorial check, no execution