Qwen3.8-27B: how much reasoning effort should we pay for?

Quick summary

Qwen3.8-27B at medium scores 8.18 against 7.96 for Qwen3.6-27B and answers 32% faster, so the generational gain comes with no trade-offs to weigh. Effort itself behaves less neatly: low is both slower and worse than medium, because thinking less at each step costs extra agentic rounds. Below we cover the per-setting numbers, the agentic round counts that explain them, our noise estimate, and why per-request latency is the wrong unit for agentic work.

Key findings

What do we get from one generation?

The two highlighted rows are one generation apart at the same parameter count. At medium, Qwen3.8 scores 0.22 above Qwen3.6 and answers 32% faster, quicker on 45 questions in the set. There is no trade-off to weigh here, and it gets there by needing a median of six fewer agentic rounds. (For Qwen3.6 models we used thinking mode on.)

Does more effort always cost more time?

Quality climbs at every step, but time does not follow. Low is both slower and worse than medium, which looks like a measurement error until we count the tool calls.

Rabbit hole: Why is the cheapest setting so slow?

Medium finishes in fewer agentic rounds than low on two thirds of the set, a median of six calls fewer and 10.5 seconds faster.

Thinking less at each step means worse decisions, and one worse decision costs another round trip.

It does not generalize upward. From medium to xhigh, three quarters of questions take more rounds, so xhigh buys quality by working more rather than smarter.

Where does the dial spend its tokens?

On thinking, not on output. Answer length barely moves while median latency swings by 88%, and time-to-first-token sits near one second everywhere.

Caveats

We ran each configuration once. Re-running one of them with a decode-level change that should not alter answers moved substance by 0.18, which is our best estimate of run-to-run noise. The 0.43 gap between medium and xhigh clears that, but not by a wide margin.

We are also close to the benchmark's ceiling. At xhigh, 58% of questions already score 9 or better, so there is little room left to register an improvement, and the real gap between generations is probably wider than we measured.

Latency was measured with max_num_seqs=2, so these are lightly loaded serial numbers, useful for comparing configurations but not for capacity planning.

What does this mean for developers?

Thinking budget used to be something we observed after the fact, not something we set. Now it is a serving parameter, and we think that changes how we provision: pick the effort per route rather than per deployment. Research and long-horizon work wants xhigh, latency-sensitive paths want medium, which was the fastest setting here as well and the second best overall in our test. Low is worth measuring against our own traffic first, since on this workload it did not pay for itself.

Per-request latency is also the wrong unit for agentic work. What we pay is rounds multiplied by time per round, and effort moves both, sometimes in opposite directions. The most important thing is to budget the whole trajectory instead of the single turn.

Methodology

We used our answer-quality benchmark, and ran it against the Kodesage codebase, on the same deployment, which uses a single A100 80GB GPU under vLLM 0.25.1. Answers were judged by Claude Opus 4.6, three judgments per metric averaged.

Only the model and reasoning_effort changed between runs.

Full run results

FAQ

What is Kodesage?

Kodesage is an AI-powered platform for enterprises running mission-critical legacy systems, where knowledge is typically trapped in retiring experts or proprietary vendor systems. It extracts business and technical knowledge from code, documentation, databases and issue ticketing systems to create an always-current knowledge layer accessible across engineering, support, QA and domain experts. Kodesage is deployed on-premise for industries where compliance and security are non-negotiable.

What is the agentic answer-quality set?

Real questions about an indexed codebase and its documentation, each paired with a human-written reference answer. Agentic means the model searches and reads through tool calls to build an answer, rather than being handed context up front.

What does "one fixed kodesage version" mean?

Retrieval, prompts and the agent loop were pinned to a single build for every run, so the differences we measured came from the model rather than from product changes underneath it.

How are accuracy and completeness defined?

Accuracy scores whether what the answer says is true, checked claim by claim against the reference, with fabrications penalized heavily and omissions ignored. Completeness scores the reverse: how many of the reference's key points the answer covers, with extra material neither helping nor hurting.