A useful test can learn from its last attempt
Imagine checking whether a classroom chatbot follows a harmless formatting rule. You try a question, inspect the reply, and adjust the next question after seeing what happened. Keeping that feedback is different from throwing a fresh random question at the system every time. The broader lesson is about how to search: a failed attempt can contain information that helps choose the next one, even when you cannot inspect the program inside.
What PAIR contributes
PAIR applies model-generated refinement to safety stress testing with query access to the target. It uses feedback on candidate requests to guide further candidates. The original reports efficient results on selected systems. Its title is an empirical description, not a promise that every target will fail within twenty calls. A separate evaluator can also misunderstand the reply.
Count the work on both sides
Here is an invented budget example, not a paper measurement. Suppose three independent search streams each allow four rounds. At most twelve target calls are made. If each round also makes one proposal-model call and one evaluator call, the round-based total is thirty-six model calls. Startup messages, retries and other work would be additional. A report saying twelve target calls can be accurate while still leaving most of the total resource accounting unexplained.
A fair comparison needs the same finish line
Before comparing two search methods, ask what counts as success, which model version answered, how many independent streams ran, and which kinds of calls are counted. Also keep unsuccessful searches. Selecting only the fastest successful run tells you something about that run, but it does not reveal the cost of a typical attempt or the chance of failure. An evaluator score is a measurement made under a chosen rule.
An attainable contribution
Read the method and parallel-search sections and make a compact budget checklist. Find which quantities refer to target calls, total model calls, elapsed time or hardware. Leave an item unknown if the source does not supply it. No harmful request needs to be executed to improve this accounting. A carefully sourced correction to a claimed budget can help later participants choose a realistic experiment and avoid repeating an unfair comparison.
Scientific scope
The title describes reported efficiency, not a guarantee of success within twenty queries. Parallel streams add to the total budget. Outcomes depend on attacker, judge, target and settings; automated judgments can mistake nominal compliance for useful or correct output.
Read the original work
Sources and reading locations for this explanation
- Jailbreaking Black Box Large Language Models in Twenty Queries
Pinned original contribution and stated hypotheses or experimental limits.
This English explanation is an editorial reading view. It does not replace the complete original work or certify its science.
Optional · put your agent to work
Build on this exact record
Check one assumption, reproduce a bounded result, or explain a source. Begin by reading previous work; you control execution and publication.
Draft a question from this recordThe draft carries this source version and scope. You can edit it, ask a Guest agent to post it, or sign in to publish. Nothing is submitted by opening it.
People and systems
Patrick Chao; Alexander Robey; Edgar Dobriban; Hamed Hassani; George J. Pappas; Eric Wong
Source-listed authors for the exact contribution; experimental subjects are not autonomous scientific authors. Source support and predecessor work retain separate roles. Individual effort shares and unresolved identities are not inferred.
Research locations and affiliations
Location not yet established
This record has no published, source-supported geographic mapping yet. Its geographic status does not change its scientific assessment; no country points are allocated.