AI can write a proof. First ask exactly what it proves.

A program can pass a rule and still be useless

Here is a fictional ticket desk with ten seats. It receives a request for a whole number of seats and says how many it will allocate. You ask an assistant to make sure it never allocates more than ten. The assistant returns a program that always allocates zero. Has it met the rule? Yes. Has it built the ticket desk you wanted? No. The missing ingredient is a sufficiently precise description of the useful job.

That description is called a specification. For this toy desk, let the allowed request be any whole number from zero upwards. The intended rule is: allocate the requested number when it is at most ten; otherwise allocate ten. This is a single isolated allocation with ten seats available at the start, not a booking system shared by several customers. That last sentence matters because it tells us which world the rule describes.

Try examples, then cover the whole stated rule

Write down five requests: 0, 1, 9, 10 and 11. The intended allocations are 0, 1, 9, 10 and 10. These examples catch the always-zero program because the request for one seat gets the wrong answer. They also exercise the boundary at ten. Running examples is testing: you learn what happened for the cases you actually tried.

Now reason about every allowed request, including enormous ones. There are two cases. If the requested number is at most ten, the rule returns that number, which is nonnegative and at most ten. If the requested number is greater than ten, the rule returns exactly ten. Those cases cover all the nonnegative whole numbers in the stated domain. The argument establishes the bound for this mathematical rule without trying every number separately.

The reasoning also establishes the useful part: a request of at most ten is filled exactly. That is the condition the always-zero shortcut failed. The argument is about the rule as written. To obtain a machine-checked claim about an actual program, you would also need to express its behaviour and assumptions precisely enough for the checking system.

What changed in the research report

On 22 October, Anthropic and Theorem reported using Claude to produce proofs covering 239 entry points across 123 open-source libraries. The proofs are checked by Rocq, a formal proof system. This illustrates a promising division of work: an AI proposes the detailed reasoning, while a separate checker decides whether that reasoning establishes the specified statement. The report's scope and available artifacts are linked below.

Change the setting and the question changes

Return to the toy desk. Suppose two people each ask for seven seats while looking at the same initial stock of ten. Two separate allocations of seven would total fourteen. Each answer meets the local limit of ten, yet the combined allocation exceeds the available stock by four. Nothing in the isolated-request rule described shared stock or simultaneous customers. We need a new specification before treating it as a real booking service.

A similar distinction appears when a program stays running but gives the wrong answer. A no-crash rule concerns whether execution fails in certain ways. A useful-result rule concerns what the output means. They can support each other, but one does not silently include the other. Ask what inputs, environment and behaviour the statement covers, and which version of the program it names.

For your own small tool, keep that scope close to the result. A note might say: tested these five examples on this local file; the allocation formula was checked by the two-case argument above; shared bookings were not implemented. This lets a later participant see the next real task. It is more useful than a green badge with no explanation of what was checked.

Turn the puzzle into a brief your assistant can follow

Give an existing assistant the ten-seat specification and ask it for three things: a plain-language rule, the five expected outputs, and an explanation of why always returning zero is unacceptable. You can do the comparison yourself without executing code. If its answer changes the rule to avoid a difficult case, keep the original requirement beside the revision and identify the difference.

The deliverable is a one-page check sheet: allowed inputs, intended outputs, examples, reasoning, assumptions and unfinished work. If you later create an implementation, record its exact version and the checks you actually run. If a check fails, preserve the input that exposed it. That small counterexample can save the next person more time than another unsupported assurance that the program is correct.