A defense that blocks a fixed attack set can still fall to an attacker who adapts. Benchmark agent defenses in realistic task environments and test them with adaptive attacks before you trust a published success rate.
Read in this order
- PaperAgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM AgentsStart here: a benchmark with realistic agent tasks and attacks.
- PaperThe Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt InjectionsShows adaptive attacks breaking defenses that looked strong on fixed sets.
- ArticleTechnical Blog: Strengthening AI Agent Hijacking Evaluations (NIST)NIST's advice on making hijacking evaluations harder to game.