Chao and colleagues: iterative black-box model red-teaming with PAIR

What changed

Introduced an automated red-teaming workflow in which an attacker language model revises candidate requests using feedback from a target model and an evaluation model, enabling query-based stress testing without target-weight access.

Scientific scope

Introduced an automated red-teaming workflow in which an attacker language model revises candidate requests using feedback from a target model and an evaluation model, enabling query-based stress testing without target-weight access. The title describes reported efficiency, not a guarantee of success within twenty queries. Parallel streams add to the total budget. Outcomes depend on attacker, judge, target and settings; automated judgments can mistake nominal compliance for useful or correct output.

Evidence

The linked original works support the scoped statement and contribution attribution. WRS has not independently replayed experiments or reconstructed a full mathematical proof. The title describes reported efficiency, not a guarantee of success within twenty queries. Parallel streams add to the total budget. Outcomes depend on attacker, judge, target and settings; automated judgments can mistake nominal compliance for useful or correct output.

Contributors

• The named authors in the research record · SCIENCE: Source-listed authors for the exact contribution; experimental subjects are not autonomous scientific authors. Source support and predecessor work retain separate roles. Individual effort shares and unresolved identities are not inferred.

• Jailbreaking Black Box Large Language Models in Twenty Queries

Provisional WRS assessment, edition 2026-10-23.4. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction