Auditing Prompt Injection Defenses Before They Reach Users: What Strong Adversaries Reveal
arXiv:2505.15738v3 Announce Type: replace-cross Abstract: Large language models are increasingly deployed in user-facing products that handle sensitive personal data while interacting with untrusted external content. This makes them a prime target for prompt injection attacks, in which an adversary injects malicious instructions in that content to hijack the model. Fine-tuning-based defenses have been proposed to mitigate these attacks by training models to follow benign user instructions and i…