Methodology
What counts. A run counts as working if it starts in the browser and moves, even if it looks more like a pillar than a giraffe. The quality of the walk was not rated. For the other runs, the card states the reason.
Tokens. “New input” and “output” were actually computed; “cached” is the conversation history re-read with every request. Reasoning tokens count as output.
Thinking. What pi and OpenCode sent to the model about thinking was measured: each run was started again with the same harness version and the configuration of the test day against a server that only records the requests. In 75 of the 84 local runs no thinking setting was sent, so the model's default applied. 71 of these runs still contain thinking text; only the four of the two Mistral models do not. OpenCode never sent a level. pi always shows a level internally but only passed it on where the model configuration said so. The character count is the thinking text the model emitted during the run. For the cloud runs, neither harness sent a thinking setting to OpenRouter; the card shows the reasoning tokens OpenRouter billed instead.
OpenRouter cost (theoretical). For models that ran locally: what the same tokens would have cost for the same model via OpenRouter, at list prices as of November 13, 2026: new input at the input price, cached at the cache price (or the input price if none is listed), output at the output price. There the models usually run at higher precision than the quantizations tested here. OpenRouter does not offer abliterated variants and many small models.
OpenRouter cost (real). For the cloud runs: what OpenRouter actually charged. All requests went through a small local proxy that asked OpenRouter for the cheapest provider of each model and recorded the cost OpenRouter reported for every request. Where a run needed a second try, both tries are included. All cloud runs shown together cost $267.69.
Output limit. pi and OpenCode allowed up to 32,768 output tokens per response. Exceptions with a higher limit, each with pi: Claude Opus 5.5 (128,000), Gemini 4 Argon and MiniMax-M3 (131,072 each). The card names the limit.
Models. The release date is when the original model was created on Hugging Face. Checksums are the SHA256 values Hugging Face lists for the GGUF files. All local runs used llama.cpp on a Mac Studio M5 Ultra with 256 GB. The Mac Studio draws 259 watts while computing. From that and the coding time, each local card shows the energy used; electricity cost assumes 37.3 euro cents per kilowatt hour, the 2026 average household price in Germany according to BDEW. This is an upper bound, since the machine draws less in pauses between requests, for example while the harness writes files. They date from November 4 to 13, 2026, the cloud runs from November 8 to 13. Where several quantisations of a model are in the test, the quantisation is part of the name; for Qwen3.8-27B its origin follows: Unsloth re-uploaded its dynamic quants as version 3.1 on 10 and 11 November, so UD-Q4_K_M is in the test in both versions. “ISTA” marks the GSQ-RCO quants from ISTA-DASLab. For cloud models, the release date is when the model appeared on OpenRouter. Two cloud models could not be tested: Crouton Alpha is not offered in our region, and the only provider of Muse Spark 1.3 Contributor may use requests for training, which our OpenRouter account excludes. The regular Muse Spark 1.3 is in the test. If a run was repeated after a server change, only the new attempt counts.
The giraffes are exactly as the models wrote them; some load Three.js from a CDN. The stills were rendered in software.
Rating. Visitors rate the giraffes in the giraffe duel: two running giraffes appear side by side without model names, you can rotate and zoom them, pick the better one and submit. Only then does the page show the models. All votes form an Elo ranking. Against manipulation, Cloudflare Turnstile checks at the start that a human is voting; every vote needs a ticket signed by the server for exactly this pair, and each IP address counts once per pair per day.
Memory. The memory need is the size of the model files plus 10 % for context, but at least 1.5 GB, rounded up to whole GB. “Fits in X GB” compares against it. On a Mac, the GPU may only use part of the unified memory by default, so leave some headroom.
Speed. Output tokens divided by coding time. It includes prompt processing and the time the tools ran, so it is lower than the pure generation speed.
