57% more Qwen3.8 throughput. Same Radeon. Regular vLLM.

One Radeon AI PRO R9700. The same Qwen3.8-27B MXFP4 checkpoint. The same 5 GiB cache allocation. Paiton delivers 57% more aggregate throughput than our matched Radiance + DFlash2 baseline at eight concurrent requests and reduces median time to first token from 6.59 seconds to 195 milliseconds.

Throughput and latency headlines use the full 188-request comparison at eight concurrent requests. Both accelerated engines use the same target and DFlash2 snapshots. GPU artwork is illustrative.

This is a substantial step forward for serving a 27B model on one workstation GPU. Not just faster generation when one person is waiting. More requests progressing together, almost three times the estimated cache-token capacity, and dramatically less waiting under load.

Paiton reaches 314.5 aggregate output tokens per second, against 200.3 tok/s for the matched accelerated baseline. Weighted serial decode improves by 22%, and prefill is faster at every tested prompt depth. The execution runs through a plugin on the official vLLM 0.28 ROCm runtime. The installed vLLM library stays unchanged.¹

The important combination is performance and deployment: a much stronger local serving result without moving to a separate inference-engine distribution.

See it in action

Watch Qwen3.8 generate responses on one Radeon AI PRO R9700, with Paiton + DFlash2 running through regular vLLM. The recording shows streamed output alongside live generation figures and GPU activity.

70-second screen recording with one active request at a time. The live figures describe the requests shown, not the eight-request aggregate benchmark below.

Open the recording (MP4, 10.2 MB). Use the player's fullscreen control for a closer look.

A strong baseline. A bigger result.

The investigation began with the impressive work around vLLM-Radiance. Seeing what the team achieved on AMD hardware gave us ideas and encouraged us to push our own R9700 implementation further. Its public documentation describes support for the same AMD Quark MXFP4 model and DFlash2 acceleration; its published performance results use two R9700s.²

Radiance inspired the investigation; Paiton's native implementation is our own.

We wanted to answer a different question: how much could we deliver from this model on one card, while keeping the regular vLLM deployment path?

We ran Radiance + DFlash2 and Paiton on vLLM + DFlash2 ourselves on one R9700, using the same checkpoint, target and draft snapshots, FP8 KV cache, 5 GiB cache allocation and 8,192-token context ceiling. Both accelerated profiles used seven speculative tokens, greedy sampling and disabled prefix caching.¹

Our percentage gains compare those matched single-GPU runs, not a one-card result against somebody else's two-card screenshot.

57% more throughput, with a growing lead under load

The headline comes from the full 188-request BetterBench preset, not the shorter, 128-output-token-cap comparison. We retained the preset's original longer output budgets, ten measured passes per task category, 24 requests at each tested concurrency level and repeated prefill measurements. Both engines completed the full request set.¹³

Full 188-request workload, runs 495 / 494. Aggregate throughput includes prefill and queueing. Higher is better.

Paiton leads at every tested concurrency level. The full task-category results also show a lead across code, reasoning, prose, JSON, file editing, summarization, math and chat. The improvement survives the longer workload instead of depending on a single favorable prompt.¹

These are aggregate serving rates across the active workload. 314.5 tok/s does not mean each of eight users receives 314.5 tok/s.

From a 6.59-second wait to a 195-millisecond first token

The throughput gain is large. The change in responsiveness under load is larger.

At four concurrent requests, median time to first token falls from 1,627 ms to 180 ms. At eight, it falls from 6,586 ms to 195 ms: 97% lower. These timings include queueing, so they describe the wait a client experiences before a response begins.¹

Full 188-request workload. Lower is better. Radiance retains a 10–11 ms TTFT advantage at one and two concurrent requests; Paiton still leads throughput at both levels.

For a shared coding assistant or local agent endpoint, throughput alone is not enough. An endpoint that produces more tokens but leaves requests waiting to start can still feel slow. Here, the higher concurrency result comes with a substantially shorter wait for that first token.

The same 5 GiB holds almost three times the token capacity

With exactly 5 GiB reserved on each engine, the runtimes report 25,746 cache-token slots for Radiance and 74,430 for Paiton. That is 2.89× estimated token capacity inside the same allocation.¹

Runtime-reported shared-cache capacity estimates, not physical VRAM capacity. The per-request context ceiling remains 8,192 tokens.

The run logs capture up to three active requests for Radiance versus eight for Paiton in this matched configuration. Those are observations from these runs, not universal concurrency limits for either engine.¹

The extra capacity helps explain the stronger concurrent experience: more requests can make progress inside the same budget. The capacity figures and active-request observations align with the throughput and latency result, although they do not isolate cache handling as the sole cause of the improvement.

More usable serving capacity, not more VRAM or a larger advertised context window.

Faster decode. Faster prefill. Across the workload.

The gains are not confined to admitting more requests. In the full workload, weighted serial decode rises from 86.0 to 104.9 output tok/s: 22% higher. This measures generation after the first token and is separate from the aggregate complete-workload throughput above.¹

Weighted serial decode, full workload. Same target and draft snapshots on the same R9700.

Prefill improves at every tested prompt depth as well. At approximately 1,556 / 3,024 / 5,226 input tokens, Radiance measures 2,828 / 3,003 / 2,926 input tok/s. Paiton reaches 3,182 / 3,521 / 3,367 input tok/s.¹

Full-workload prefill results: 12.5%, 17.2% and 15.1% higher, calculated from the displayed summary values. Actual prompt tokens divided by HTTP time to first token, not isolated kernel throughput.

Together, these results show improvement at several points a user notices: getting the prompt processed, starting the answer under load and generating the rest of it.

Against stock vLLM, the whole-engine gap is striking

We also ran a separate 54-request matrix with generation capped at 128 tokens, comparing stock vLLM O2, Radiance + DFlash2 and Paiton on vLLM + DFlash2. The stock profile used the best O2 settings we tested.¹

Separate 54-request matrix, runs 403 / 493 / 492. Values from the published benchmark summary. This table is not the source of the 57% headline.¹

At eight concurrent requests, aggregate throughput rises from 33.7 tok/s on stock to 328.5 tok/s with Paiton, nearly tenfold. Weighted serial decode in this shorter comparison rises from 4.5 to 113.5 tok/s.¹

The scope matters: stock uses checkpoint-native W4A4 emulation, while the accelerated configurations use W4A8 execution plus DFlash2. These are complete engine configurations on the same weights, not identical activation arithmetic or a claim that replacing one kernel delivers the entire gain. Nor do these figures describe every stock-vLLM model or quantization.

Prefill in the separate capped matrix, with actual tokenized prompt lengths. Keep these values separate from the full-workload prefill series above.

That is why the main headline uses the harder comparison: Paiton against an already accelerated Radiance + DFlash2 baseline, confirmed on the longer workload.

Regular vLLM. The installed library stays unchanged.

The deployment result is deliberate. Paiton integrates through vLLM's extension mechanisms and supplies native HIP runtime artifacts for the optimized execution. vLLM's plugin system is designed to support extensions without modifying its codebase.¹⁴

Paiton's native HIP kernels and integrated DFlash2 support provide the optimized execution on the official vLLM ROCm runtime. The release supplies the runtime artifacts needed for the tested deployment, including the DFlash2 integration, so users can deploy the supported profile without a separate DFlash package. Our Paiton compiler remains proprietary.¹⁵

We checked the ordinary vllm.entrypoints.openai.api_server entry point independently of the benchmark harness. It passed streaming chat, eight concurrent requests, a request at the configured context boundary and a fresh request afterward. All 2,893 installed vLLM files matched the official base image. Validation applies to the pinned tested runtime and supported profile, not every vLLM feature or future version.¹

Standard vLLM still carries its normal framework dependencies; Paiton's native libraries load independently of those frameworks. This is not a claim that the entire serving stack has become framework-free.

Completion and API checks are useful reliability checks. They are not an independent model-accuracy evaluation or a guarantee of identical generated text across execution profiles.

More output from the same active hour

The full-workload concurrency-eight rates translate into a useful capacity illustration. Sustaining 200.3 tok/s would produce about 721,000 output tokens per active hour. Sustaining 314.5 tok/s would produce about 1.13 million, roughly 411,000 additional output tokens from the same hour.

Equivalently, one million output tokens would take 1.387 active hours at the baseline rate or 0.883 hours at Paiton's rate: 36.3% less active time.

Arithmetic illustration using the full-workload concurrency-eight rates. Assumes those rates are sustained; this is not an hour-long measurement, energy measurement or financial-cost claim.

The hardware does not change. How much useful work it can deliver does.

Tested configuration

These are results for the tested model, runtime and workload. They do not establish a guarantee for other GPUs, larger contexts, different drafters or untested concurrency levels.¹

Run it on your R9700

The Paiton model guide is the starting point for the deployment instructions and benchmark evidence. The Hugging Face companion repository identifies the model release.¹⁶

Runtime package: Get the runtime package and release notes.⁷

Container image for this release:

ghcr.io/eliovp/paiton-vllm-plugin:qwen38-mxfp4-dflash2-rdna4-v1.0.0

Use the model guide's pinned configuration to reproduce the result. The 5 GiB cache allocation, target and draft snapshots, speculative settings and context limit are part of the comparison, not incidental defaults.

Same hardware. A much stronger serving result.

57% more aggregate throughput. 97% lower median time to first token at eight concurrent requests. 2.89× estimated cache-token capacity. Plus faster serial decode and prefill, all on one Radeon AI PRO R9700 through regular vLLM.

For local developers, that means a stronger shared endpoint from a single workstation GPU. For teams running AMD inference at scale, it is another demonstration of why execution efficiency matters alongside hardware capacity. The R9700 numbers are not a prediction of gains on other AMD platforms.

Running an AMD inference workload that should be delivering more? Talk to us about Paiton. Bring the model, workload and current baseline.

Credit where it belongs

Thank you to the Radiance team for their excellent work on AMD inference and for providing the inspiration and a strong comparison point for this investigation.

We also thank vLLM, StillDeadcode/libr4d, the Qwen and DFlash2 teams, AMD's Quark checkpoint team and BetterBench for the foundations, models and measurement tools that support this work.

Sources and benchmark references