Named author’s task tests · paid training offered on the site
Terra fixed small patches, but missed a character and a conversation detail
Dustin’s AI Lab · 26 October 2026
Product and setting: GPT-6.1 Terra/Sol and Claude comparisons in a custom tool harness; not an ordinary Chat trial
What they did: Synthetic review and screen tasks, with small repeated runs; three short bug patches were each replayed twice. A test with an accessible reference fix was discarded and rerun in isolation.
What they report: The author reports successful small repairs, but Terra missed a simplified/traditional Chinese character and misread a continuing conversation. Higher effort did not repair those screen-reading failures. Sonnet and Opus handled those two cases.
Money: The report includes timings; a complete consumer-plan bill, allowance comparison and human repair cost are not established.
How far this carries: Small author-designed sample. The author also sells training; commercial independence is not established. Built-in Codex computer use and large patches were not tested. Do not infer a universal model order or consumer free-tier performance.
Read the original report ↗ · Original Chinese version of the same report ↗
Commercial wrapper’s own API task test
More thinking cost more; it did not improve these short answers
MultiAI · Launch-period test, published 26 October 2026; checked 28 October
Product and setting: GPT-6.1 Terra through OpenAI’s API, not a ChatGPT subscription
What they did: Eight short tasks with scripted right answers, each run three times at Off, Medium and Max. Examples include meeting actions, a multi-currency invoice and a date function.
What they report: The publisher reports 19/24 correct at Off and 24/24 at both Medium and Max. Its short-task sample supports trying a checked lower-effort setup before paying for more thinking; it does not establish equivalence on long projects.
Money: Projected list-rate cost per 1,000 answers: US$1.63 at Medium and US$7.21 at Max, using its measured tokens. These are projections, not a purchased 1,000-answer bill, subscription charge or complete labour cost.
How far this carries: MultiAI sells access to models. These are its measurements, not an independent WRS benchmark. The article describes the checks, but this guide does not certify the inputs, evaluator or account. App allowances and free chat were not tested.
Original Chinese newsroom hands-on inspection
Shopping moved into the chat window; check the final basket
Li Yutong and Chen Ting, National Business Daily · Published 23 October 2026; shopping observations dated 22 October
Product and setting: Mainland Qianwen and Doubao consumer apps; model and account plans unspecified
What they did: Reporters inspected shopping flows, live-shop links and the same shop’s kitchen-paper offer through two channels.
What they report: They observed in-app selection and payment in Qianwen, and live-shopping links in Doubao. A paper offer cost CNY17.43 through Doubao after coupons versus CNY27.90 through the other channel on that date.
Money: The quoted amounts are dated goods prices, not AI subscription fees. Current prices and savings for your account are unknown.
How far this carries: Selected accounts, offers and dates; no representative price or model-quality comparison. Coupon eligibility and checkout can change. The report’s company statements and expert forecasts are separate from these observations.
Editorial hands-on comparison · historical version
A party plan, a rewrite and a realistic evening routine
Amanda Caswell, Tom’s Guide · 24 September 2026
Product and setting: ChatGPT-6 and Claude Opus 5.5; account tiers and settings not stated.
What they did: Five shared prompts, with screenshots: party planning, rewriting, a holiday choice, public spending and household organisation.
What they report: The reviewer preferred Claude in four tasks, but ChatGPT for a manageable evening routine. This is a useful reminder to judge whether you can use the answer, not just its length.
Money: No task-level bill or free-quota measurement; a paid model result does not establish free-tier access.
How far this carries: One reviewer, selected prompts, no repeated trials or blinded scoring. Affiliate-funded publisher. This ChatGPT-6 result is not a test of GPT-6.1 Sol or of every current ChatGPT mode. We do not reuse the article’s unverified travel-cost arithmetic.
Public user self-report · billing not verified
Three email edits after the subscription limit
Reddit user Epsilonze · 22 October 2026
Product and setting: Claude Pro user reporting Opus 5.5 on High, after reaching a weekly limit.
What they did: The user describes buying £5 of extra usage, then requesting and revising a short job-application follow-up email.
What they report: They report the credit disappearing during three short email requests. The post illustrates the surprise of moving from included usage to paid usage.
Money: £5 is the user’s claimed top-up, not a measured price for this task. Review extra-usage settings before continuing past a plan limit.
How far this carries: No invoice, full transcript, settings export or token trace. WRS cannot verify the charge or what else was processed. It is not a £5-per-email tariff; do not extrapolate nine emails to £15. Comments about caching and plan equivalence are not verified here.
Contrasting public user self-reports
The same new Gemini, very different allowance stories
Reddit users Nug__Nug, Dingleberry_Blumpkin and Beautiful_Passion_17 · October 2026 discussion; exact individual posting dates not established
Product and setting: Gemini app, Gemini 4 Argon. Two commenters identify Pro; the study-help commenter does not establish a plan.
What they did: Participants describe ordinary questions, a Google Sheets/skills task, and asking for explanations of thermodynamics course material.
What they report: One Pro user reports 52% of a five-hour allowance used by seven queries. Another reports 12% after three skills-and-ledger builds. A student describes useful explanations with much lighter apparent usage.
Money: Reported percentages are account observations, not money or transferable tasks per plan. Check your own allowance before and after your representative task.
How far this carries: No reproducible prompts, account telemetry or verified purchases. These conflicting accounts cannot establish average capacity, a free allowance, or a diagnosed billing bug. Study help is not evidence that the generated explanations were correct.
Named reviewer’s hands-on tasks
Kimi K3.1 Pro: a working visualisation can still need repairs
象先志, published by TMTPost · 25 October 2026
Product and setting: Kimi K3.1 Pro, normal Pro speed rather than Highspeed; reviewer account tier not established.
What they did: Seven coding tasks in fresh conversations, allowing self-checks before first delivery; examples include animation, a collision demo and an audio visualiser.
What they report: The reviewer reports that changing the audio input changed the displayed spectrum. A bicycle-and-bird scene missed the requested physical arrangement. They found normal Pro subjectively faster than K3.
Money: No bill for the reviewer’s seven tasks. API output tariffs and other people’s plan percentages cannot price an app project.
How far this carries: Not a controlled speed measurement or an independent physics audit. Tests and output illustrations are the reviewer’s, not WRS runs. The article’s reported official benchmarks and other users’ quota claims are not used as measured results here.
Named user’s app and coding-tool experience
A good-looking Kimi build is not the end of the task
まさお @未経験からプロまでAI活用 · 24 October 2026
Product and setting: Kimi K3.1 Pro: Kimi Code on the author’s Max plan, plus separate app experiments.
What they did: A 25-question custom benchmark; only 15 questions were scored within the available allowance. The author also tried a character-to-3D app task.
What they report: They liked landing pages and an editor’s undo/redo, but found a marble run failed under repeated use. Maximum effort interrupted benchmark work; they preferred starting at high effort.
Money: The author reports a Max subscription, without an itemised task bill. Their stopped benchmark does not establish how many free projects another account can finish.
How far this carries: Personal benchmark and judgement, not representative usage. The Code allowance observation is not a free-chat quota. Account naming may reflect a legacy plan. App and Code results are kept separate; no claim that every app offers the same effort control.
Named reviewer’s task comparison · tool surface
Sol, Opus and Grok tried the same small website brief
Thomas Wiegold · 5 October 2026
Product and setting: GPT-6.1 Sol, Claude Opus 5.5 and Grok 4.7; this is a coding workflow, not a free consumer-chat test.
What they did: One shared design brief and two Go algorithm tasks. The article links the design outputs for inspection.
What they report: The reviewer preferred Opus’s website, called Sol’s clean but underdeveloped, and placed Grok between them. This supports inspecting the delivered page against the brief rather than treating a model label as a result.
Money: The author says they pay for Claude Max and ChatGPT’s $20 plan; no itemised cost per tested build.
How far this carries: One author, subjective design preference, limited tasks. No inference about ordinary chat quality or all projects. API-price comparisons in the article do not measure subscription costs.
App Store user complaint · historical, version unknown
When a game guide keeps mixing up characters
China App Store reviewer 77和笨鸟 · 4 September shown; year and app version not established
Product and setting: Doubao consumer app; model, plan and search settings not supplied.
What they did: The reviewer says they asked about character attributes in the game 代号鸢 and repeatedly corrected confused character names.
What they report: They report that the corrections did not reliably repair later answers. A practical lesson is to provide the official character page and ask for a source-linked table rather than keep asking the same unsourced question.
Money: No payment or allowance observation. Paying is not established as a fix for this reported factual problem.
How far this carries: No transcript, preserved prompts or independently checked outputs. The displayed date cannot be attached to the newest model. A selected complaint is neither a failure rate nor proof that the current app has the same fault.
Named user's experience
Living with three paid assistants
Learn With Darin · Updated September 2026; individual session dates not supplied
Product and setting: Consumer subscriptions; several model versions, not a frozen comparison
What they did: The author reports paying for all three and using them daily for over a year. No matched outputs or task counts accompany the preferences.
What they report: Darin prefers Claude for prose, ChatGPT for structured writing and breadth, and Gemini for editing inside Google Docs. Those are personal preferences, not measured win rates.
Money: Paid consumer use is reported; no invoices or per-task costs are published.
How far this carries: The author declares no affiliation with the three providers. The page predates October releases; its model and feature list is not our current product specification.
Third-party task test
Good-looking code that would not start
Josphat Mutai / ComputingForGeeks · Tested 13 October 2026
Product and setting: Gemini 4 Argon via OpenRouter; GPT-6.1 Sol and Opus 5.5 via their APIs
What they did: Three infrastructure tasks, repeated three times. The author ran the generated server configuration, rather than only checking its syntax.
What they report: Gemini's initial server configurations failed to start; Sol and Opus passed that check. Gemini repaired all three retested failures after receiving the error, but two repairs changed a requested image or port.
Money: Reported nine-call API charges: Sol $0.062, Opus $0.254, Argon default $0.045 or high $0.169. These exclude the person's time and infrastructure, and are not monthly app prices.
How far this carries: One publisher, tiny task set, different routes and effort settings; no inference about the consumer apps or overall reliability. A code excerpt and runtime error are inspectable, not an independently replicated full benchmark.
Named developer's working example
An existing game gains four-player support
Simon Willison · 24 October 2026
Product and setting: Claude Opus 5.5; subscription and execution settings not specified
What they did: The developer publishes his starting prompt and links the resulting game release and replay.
What they report: Willison reports extending Battledrones from two to four players, including a new map and multiplayer changes. The linked release gives readers an output to inspect.
Money: Task price, token usage and total human time are not disclosed.
How far this carries: An experienced developer working in an existing project. No comparison group, full correction count or independent security assessment; WRS has not run the game. This report does not establish a beginner's expected success rate.
Third-party small-sample test
Chinese short tasks passed; a money parser did not
TOKRACE · Text tests 18 October; visual follow-ups 20–21 October 2026
Product and setting: DeepSeek V4.1 Flash 1017, direct API; thinking on/off
What they did: Fifteen Chinese text tasks and three JavaScript tasks in each mode. The publisher describes parameters and retains failures; supplementary visual request records are downloadable.
What they report: Both modes passed 15/15 text checks and 2/3 code checks. Money parsing failed through exhausted output budget or incorrect handling of leading zeros. Visual follow-ups also retained connection failures and retries.
Money: Do not transfer the short-test cost to a full app. This guide has no verified invoice for these runs.
How far this carries: Publisher-reported results, not a WRS replication. Small tasks, mutable model alias and differing visual retry budgets prevent a general ranking. Connection failures are not all model-quality failures. Commercial independence is not established.
Read the original report ↗ · Supplementary request and failure records ↗
Public user self-report
A low bill with an important catch: cached input
Reddit user illumihani · Read 25 October 2026; exact posting date not established
Product and setting: V4.1 Flash 1017 used on a software project, not the free chat app
What they did: A public post includes usage screenshots and describes a week's work. The account, invoice and output quality have not been independently verified.
What they report: The user reports about six billion tokens and $46 over a week, mostly cache hits. They liked bounded tasks and test iteration, but reported invented details and skipped instructions; a security review was still pending.
Money: $46 is the user's reported API spend, not a verified all-in project cost or a budget promise.
How far this carries: Cached input means previously supplied text was reused at a different rate. This is one selected anecdote, not a normal bill or proof that the software is safe. Other tools and human work may add costs.
Public user self-report
A paid plan can still interrupt a session
Reddit user Silly-Catch-4518, in a community discussion · Thread labelled 9 October 2026; comment date not independently established
Product and setting: Claude Pro; exact model and usage history not supplied
What they did: One comment in a volunteer-run problem-report thread describes being stopped during project planning.
What they report: The user reports hitting a usage limit despite paying for Pro. This is a concrete purchase concern to test against your own workload.
Money: The current official Pro monthly list price is $20 before applicable tax; paying does not remove all usage limits.
How far this carries: No account logs or reproduction. A complaints thread selects for problems and cannot tell us how common they are, what caused this case, or whether it persists today.
Independent research; historical context
Measure finished work, not the feeling of speed
METR · Study used February–June 2025 tools; follow-up 24 February 2026
Product and setting: Experienced open-source developers, mainly Cursor with Claude 3.5/3.7 in the original study
What they did: The original randomized study involved 16 developers and 246 tasks. METR later explained why selection and time-measurement problems limited its newer experiment.
What they report: The original tasks took 19% longer with AI, despite participants feeling faster. The follow-up says newer tools likely help more, but its data cannot reliably establish the size of that change.
Money: Subscription price alone does not measure time saved. Track checking and repair time as part of your own trial.
How far this carries: This is a lesson about measuring productivity, not evidence that today's four products slow you down. It does not rank them or describe beginners' tasks.
Publisher hands-on comparison
Grok: a natural message, an incomplete shopping list
Amanda Caswell · Tom’s Guide · 24 October 2026
Product and setting: Consumer chats; exact model versions and paid plans not stated.
What they did: Five shared everyday prompts, including a short family message and a constrained grocery plan; selected screenshots.
What they report: The reviewer preferred Grok’s message tone but found assumed ingredients and unrealistic quantities in its meal plan.
Money: Task charges and subscription plans not disclosed.
How far this carries: Small subjective sample; no repeat runs or independent scoring. The publisher discloses affiliate links. Its overall winner is not a general product verdict.
Original Chinese newsroom hands-on test · historical
Shopping help: useful links still need a human quantity check
Chen Ting and Li Yutong · National Business Daily (每日经济新闻) · 18 June 2026
Product and setting: Mainland Qianwen and Doubao apps plus shopping integrations, not international Qwen Chat.
What they did: Reporters tried a travel-outfit brief and a two-person household shopping list.
What they report: Qianwen supplied product links but still needed manual selection. Doubao’s price comparisons were useful; its household quantities were excessive.
Money: App-plan charges and time per task were not disclosed; shopping totals are not subscription prices.
How far this carries: Historical features, no repeated benchmark or disclosed model versions. Store coverage and promotions affect results; these observations do not establish current best prices.
Publisher hands-on report · historical model
Kimi K3: an attractive prototype needed repair and more allowance
Photon Planet (光子星球), credited original report reproduced by Sina · 18 July 2026
Product and setting: K3 work/coding tasks; report explicitly identifies Kimi Code for its pricing-analysis task. Not a current consumer-chat comparison.
What they did: Seven custom tasks included an interactive 3D portfolio, requirements checks and debugging, with prompts and screenshots.
What they report: The portfolio first opened blank, then ran after repair; quota pressure prevented further mobile fixes.
Money: The portfolio reportedly took nearly an hour. Exact task bill and tokens are not supplied; historical plan claims are not current checkout prices.
How far this carries: One publisher’s task set, unknown repetitions and commercial relationship. Code/Work results do not establish K3.1 Pro app performance.
Individual public app review · unverified self-report
A Doubao user reports losing instructions across turns
App Store reviewer ren ny (displayed as ren ny) · App Store displays 5 July; year and exact app version are not shown.
Product and setting: Reviewer describes Doubao mobile and web use; plan not stated.
What they did: A written account of drafting and question-answering; no preserved prompts, outputs or measured timing.
What they report: Reports peak-time delays, forgotten formatting constraints and repeated corrections.
Money: Payment status and task costs are unknown.
How far this carries: Selected complaint, not a representative survey, verified purchase or measured failure rate. App updates may change this experience.
Public user self-report
One Kimi user ran into a Highspeed allowance
xiazhi88 on LINUX DO · 23 October 2026
Product and setting: Kimi K3.1 Pro, including Highspeed; the writer mentions a CNY699 K3 plan but does not clearly establish the plan for the later K3.1 test.
What they did: The writer describes continuous use and then trying Highspeed. Exact prompts, task outputs, total tokens and a receipt are not provided.
What they report: The user reports that a five-hour Highspeed allowance lasted less than two hours of their continuous use. They also describe earlier K3 weekly limits running out.
Money: The writer mentions a CNY699 plan for earlier K3 use; the K3.1 task bill and current plan are not established. Check the actual checkout and usage meter.
How far this carries: One self-report, not a controlled test, verified bill or representative account. After this post, Kimi said it adjusted settings; IT Home reproduced the 26 October announcement. The effect has not been independently checked here. This report does not establish free capacity or current behaviour.
Read the original report ↗ · IT Home: Kimi statement and subsequent adjustment, 26 October ↗
Use the evidence to make a decision
Start with a report that resembles your task, then inspect what it leaves unknown. A personal writing preference helps you choose what to try. A repeated code test helps you choose what to check. Neither tells you what every user will experience.
Use the same small input in two available products. Keep the outputs, count factual corrections, check the final artifact and note any interruption. Record the plan, date and actual charge. This can reveal a useful fit even when both products get most of the task right.
We do not convert selected complaints into failure rates or sell a recommendation to a sponsor. New versions can change these trade-offs; historical cases keep their original dates.
Try the shared example · Share a public or redacted usage note · Read the evidence collection as JSON