Small decisions, repeated inputs: what changed in AI APIs this week?

Start with three decisions you can check yourself

The 22 October release makes OpenAI's Decisions API generally available with GPT-6 Luna and raises the request limit from 200 to 400 questions. An API lets a program ask for work through a defined interface. This release offers answers in specified data types: a useful shape for software to consume, while the answer itself still needs evaluation.

Try this fictional exercise in a document. You are sorting three event notices for a club: 'Free online drawing session', 'Weekend workshop, ticket £5', and 'Photography walk, contact the organiser'. Choose one of three labels for each: FREE, PAID or UNKNOWN. The first two notices supply a price condition. The third does not. Your expected answers are FREE, PAID and UNKNOWN, with the supporting phrase beside each.

A label is useful because another part of a program can filter the list without guessing what a long paragraph meant. But a well-formed label can still be wrong. If the third notice receives FREE, the shape is valid and the claim is unsupported. Keep UNKNOWN available and send that row for a source check. You can learn this distinction with an assistant you already use; there is no need to connect an API for the exercise.

Why repeating a beginning matters

Imagine a fictional club-announcement helper. Every request begins with the same approved event-writing rules, then ends with a different event sheet. Prompt caching reuses an eligible matching beginning of the input. “Prompt” means the material sent to the model; a “prefix” is the beginning that stays the same. Put the changing event after that stable material. Reusing a beginning does not mean reusing the answer to a different question.

OpenAI’s 20 October release adds a 12-hour option. Its 21 October update also lets projects restrict available service tiers. A service tier changes how a request is served; an allowed-tier setting is not a complete spending cap. Caching and access controls help with different parts of the cost problem.

Give the repeated input a fictional price

For this worksheet only, suppose reading the fixed beginning as ordinary input costs one unit per request, writing it into a 12-hour cache costs 2.5 units in total, and each later successful cached read costs 0.1 unit. These are normalised teaching units, not a quoted currency tariff. Assume the prefix meets all eligibility conditions and every later use hits the same valid cache. Leave the changing event sheet, generated answer and tools outside both sides of this comparison.

One use without caching costs one unit. One cache write costs 2.5, so paying upfront loses on a one-off job. For two uses, ordinary input costs two units, while a write and one read cost 2.5 + 0.1 = 2.6. It still loses. For three uses, ordinary input costs three units; one write and two reads cost 2.5 + 0.2 = 2.7. Under these assumptions, the third use is the first point where this fixed-input comparison becomes cheaper.

A cheap component does not make the whole job cheap

At ten uses the same worksheet gives ten ordinary-input units versus 2.5 + 9 × 0.1 = 3.4 units for the cached beginning. The difference is 6.6 units for that component. It says nothing about how many words the assistant generates, whether it calls paid tools, or how long a person spends checking the result. Add those costs separately. If you change the opening rules each time or fail to get cache hits, the assumed saving can disappear.

There is also a shorter cache option. A longer lifetime can help when work is separated by hours; it can be unnecessary when all reuse fits a short session. Compare the alternatives with your actual timing and rates. Do not change data-handling settings merely to obtain a lower-looking number.

What to keep when you eventually test it

Begin with a no-call worksheet: stable input, changing input, expected number of uses, time gaps, quoted prices and a maximum authorised spend. In an authorised API test, retain usage receipts so another person can distinguish ordinary input, cache writes, cache reads, output and tools. Record failed requests and retries too. The important result is a checked cost for one useful completed job, not the best-looking line in a billing table.

Count useful results before choosing a billing unit

A capacity of 400 questions in one request tells you how much the interface accepts. It does not tell you that 400 answers will be correct, that they cost the same as one answer, or that your whole project will finish in one request. Start with the three-row example and retain the expected labels. A larger trial needs varied examples, including missing and conflicting facts, before you can decide which results a person must check.

Keep the two releases separate in your cost sheet. Decisions produces small typed answers; the earlier worksheet concerns reuse of a fixed input in Responses. For either workflow, record the interface, the actual model, the quoted billing unit, observed usage and the number of acceptable results. A low charge per request is useful only after you know what the request contains and how much checking remains. This sheet is a plan until you have real authorised usage receipts.