THE SOURCE IN CONTEXT
The report divides all generated tokens by the complete request wall clock, including first-token delay. Every current table request is forced to 512 tokens with ignore_eos. Earlier naturally stopped outputs and 64-token correctness probes have different denominators and remain in the historical report rather than being merged into this cohort.
What does served rate include?
The report divides all generated tokens by the complete request wall clock, including first-token delay. Every current table request is forced to 512 tokens with ignore_eos. Earlier naturally stopped outputs and 64-token correctness probes have different denominators and remain in the historical report rather than being merged into this cohort.
Source: WeZZard · pinned document
Why were earlier results replaced?
The author remeasured the campaign on 5 September after discovering a slow-clock GPU and unequal prefix-cache policy. The corrected runs disable prefix caching for all three models, exercise complete workload shapes during warmup and retain per-node JIT caches. Earlier measurements remain visible as history; they should not be treated as the corrected cohort.
Source: WeZZard · pinned document
What can the long-context rows establish?
At one user in the long agentic workload, the reported medians are 22.5, 21.0 and 17.7 served tokens/s, with different between-campaign spreads. The author labels this row not separated by the data. Different draft models and KV pools remain attached to each model, and the high-concurrency rows do not even use the same number of users across models.
Source: WeZZard · pinned document
Why are failures and checkpoint identity useful?
The README reports a Qwen FP8-KV configuration that failed on chunked input and GLM backend attempts that did not complete a forward pass. Its DeepSeek current rows use the text 0731 checkpoint, not Vision-Exp and not V4.1 Flash 1017. These failures and identities explain why a recipe label cannot substitute for the actual six-field serving configuration.
Source: WeZZard · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| DeepSeek V4 Flash 0731, long agentic | served rate | 22.5 · spread 1.04× tokens/smedian of three campaigns | Original row ↗ |
| Qwen3.8 Flash-Next, long agentic | served rate | 21.0 · spread 1.01× tokens/smedian of three campaigns | Original row ↗ |
| GLM5.3 Flash, long agentic | served rate | 17.7 · spread 1.32× tokens/smedian of three campaigns | Original row ↗ |
DeepSeek V4 Flash 0731, long agentic · served rate · conditions
| system | 2 x DGX Spark; 121 GiB unified memory each; 200 GbE |
|---|---|
| node_count | 2 |
| parallelism | Not specified |
| kv_dtype | Not specified |
| context_limit_tokens | Not specified |
| experiment_date | 2026-09-05 |
| output_tokens | 512 |
| ignore_eos | True |
| prefix_cache | False |
| warmup | all workload/user shapes, full generation |
| repetitions | 3 |
| concurrency | 1 |
| measurement_denominator | whole request wall clock including TTFT |
| workload | agentic 33k context |
| model | DeepSeek-V4-Flash-0731 |
| kv | fp8_ds_mla |
| kv_pool_tokens | 517484 |
| draft | DSpark k5 probabilistic |
{'spread_ratio': 1.04}
Qwen3.8 Flash-Next, long agentic · served rate · conditions
| system | 2 x DGX Spark; 121 GiB unified memory each; 200 GbE |
|---|---|
| node_count | 2 |
| parallelism | Not specified |
| kv_dtype | Not specified |
| context_limit_tokens | Not specified |
| experiment_date | 2026-09-05 |
| output_tokens | 512 |
| ignore_eos | True |
| prefix_cache | False |
| warmup | all workload/user shapes, full generation |
| repetitions | 3 |
| concurrency | 1 |
| measurement_denominator | whole request wall clock including TTFT |
| workload | agentic 33k context |
| model | Qwen3.8-Flash-Next NVFP4 |
| kv | BF16 |
| kv_pool_tokens | 161728 |
| draft | NEXTN |
{'spread_ratio': 1.01}
GLM5.3 Flash, long agentic · served rate · conditions
| system | 2 x DGX Spark; 121 GiB unified memory each; 200 GbE |
|---|---|
| node_count | 2 |
| parallelism | Not specified |
| kv_dtype | Not specified |
| context_limit_tokens | Not specified |
| experiment_date | 2026-09-05 |
| output_tokens | 512 |
| ignore_eos | True |
| prefix_cache | False |
| warmup | all workload/user shapes, full generation |
| repetitions | 3 |
| concurrency | 1 |
| measurement_denominator | whole request wall clock including TTFT |
| workload | agentic 33k context |
| model | GLM-5.3-Flash |
| kv | BF16 |
| kv_pool_tokens | 162688 |
| draft | DFlash2 |
{'spread_ratio': 1.32}
AI-assisted source reading and original editorial synthesis; no independent model execution.