Understand AI · Chapter 1 of 13
A stronger model is not always the better choice
Choose for the job you need finished, including the mistakes you will have to repair.
About 5 minutes · school-level background
Edition 2026-10-28.1 · Checked 2026-10-28
What you will understand
Design a small, honest model comparison and explain why a cheap answer can produce an expensive task.
In this lesson
Build the idea
Start with the finished job
Imagine that you help a student club publish its weekly events. Someone sends a date in a message, another person sends a poster, and the organiser changes the room at the last minute. You want an assistant to make one accurate event listing. A model that writes a beautiful paragraph but invents the room has failed. A shorter answer that says the room is still unconfirmed may be exactly what you need. Before choosing a model, describe what a finished job looks like.
A language model generates responses from the information available to it. The surrounding product may add search, file reading, calculation or other tools. That distinction matters when you compare two assistants. One may be using a fresh organiser page while another only sees your pasted text. If the first answer is better, the difference might come from the supplied evidence rather than the model alone. Record the inputs and available tools alongside the model name.
Same task and inputs
Try each assistant
Check sources and missing facts
Count corrections and cost
A simplified reading aid. Follow the article for sources, conditions and limits.
What the source reports
What changed in the product landscape
OpenAI’s living guide, checked on 14 October, presents different models for demanding, balanced and faster lower-cost work. Anthropic’s Haiku 5.5 announcement similarly makes a case for narrower, high-volume tasks. These are provider descriptions. Neither source establishes which assistant will handle your club’s changing event information best. The practical response is to compare complete outputs on a small set of relevant cases.
Original report: OpenAI: current model guide · Anthropic: Claude Haiku 5.5
Worked teaching example
Build a test before admiring the answer
Here is a fictional six-case test. In the first two cases, every event detail agrees. In the next two, the start time is missing. In the last two, an old poster and a newer organiser message disagree. Write the expected behaviour first: extract the agreed facts, label missing information, and flag the conflict with both source dates. Do not decide what counts as success after seeing which model sounds most convincing. That would let the answer change the question.
Suppose Assistant A handles all six cases correctly. Assistant B handles five but silently chooses a room in one conflicting case. Saying that B scored five out of six is accurate for this tiny test. Saying it is reliable for all future events is not. You deliberately included a small collection of situations, not a representative sample of everything that can go wrong. Keep the failed input. It is more useful for improving the process than a rounded percentage on its own.
Now change one thing: give both assistants the same instruction to stop and report a conflict when two current sources disagree. If B now succeeds, you have learned that the instruction mattered on that case. You have not proved that the model changed or that all conflicts are solved. Try a new conflict that you did not use while tuning the instruction. This fresh case helps you tell the difference between fixing a general weakness and teaching the system one answer.
Worked teaching example
Count repair work as part of the cost
Consider another invented comparison over one hundred listings. A fast option costs one penny per listing and produces ten mistakes needing review. A slower option costs five pence per listing and produces two such mistakes. Suppose repairing each mistake takes five minutes, and you value review time at twelve pounds an hour. Five minutes is one twelfth of an hour, so each repair costs one pound in time. The fast option costs one pound in model use plus ten pounds in repair: eleven pounds.
The slower option costs five pounds in model use plus two pounds in repair: seven pounds. In this example it is cheaper overall, despite costing five times more per attempt. Now suppose the fast option makes only one mistake. Its total becomes two pounds, and the choice reverses. The lesson is to measure the work that surrounds the answer. These numbers illustrate the calculation; they are not observations about any named product.
Build the idea
Route the easy work carefully
You might send clear listings to the fast option and conflicts to the stronger one. This is routing: choosing a path according to the input. The routing decision can itself be wrong. If the first system fails to notice that the poster is old, it may classify a hard case as easy. Include those missed conflicts in your test. A two-model setup is not automatically safer or cheaper just because its diagram has more boxes.
Start with a rule you can inspect. For example, if required fields are missing or two supplied values disagree, return the case for review. For a small project, a spreadsheet check may do this better than another model call. You can add more flexible reasoning when a real input demands it. Keep one copy of the original sources so that every path is judged against the same facts.
Use the idea
Your comparison sheet
Use columns for the input, expected behaviour, model and product, available tools, instruction version, observed result, time and repair needed. Separate factual failures from style preferences. A friendly tone may help your audience, but it should not compensate for a wrong date. If you change the model version, the input format or the task, rerun the cases that depend on that change. A result from last month is a useful record, not a permanent certificate.
Your next step is small: prepare six fictional event cases and ask one assistant to handle them. You can learn something before buying a second tool. Check the answers yourself, then write down the most useful failure. That failure tells you whether the next improvement should be clearer evidence, better instructions, a simple rule, more capable reasoning or a narrower promise to the user. Choosing a model becomes a decision about finishing work.
Words you can now use
- Evaluation
- A planned check of outputs against criteria chosen for the task.
- Routing
- Sending an input to a particular model or process according to a rule.
- Test case
- An input paired with the behaviour you expect.
Check your understanding
The fast model is five times cheaper per answer. What information could still make it more expensive to use?
Show the explanation
Its mistake rate, the time and cost of repairs, retries, tool costs and the consequences of an undetected error. Compare completed acceptable tasks on the same inputs, not token price alone.
Read the original sources
- OpenAI: current model guideLiving vendor documentation · Publication date not stated · Checked 2026-10-28
- Anthropic: Claude Haiku 5.5Vendor announcement · Published 2026-10-07 · Checked 2026-10-28
Try it with your own assistant
Select and copy this bounded exercise. A normal chat assistant is enough to start.
Make six invented task inputs: two easy, two incomplete and two conflicting. Write the expected behaviour before comparing assistants. Record the model, date, instructions, failures and human review time. Use no private customer material.
Keep the task brief, supplied inputs, checked output and unresolved questions together. Using this exercise does not submit anything to WRS.