THE SOURCE IN CONTEXT
The k76 v2.0.0 concurrency series uses the earlier 16c9bd54788d12390838a65ce4a4ecda97fa5f1d GDN-MSE checkpoint, while v2.2.0 cold-prefix measurements use 03f4a0570496bbe741189f4a2a7d611679e85406. Both employ two directly connected Sparks, vLLM plus b12x and mixed NVFP4/MXFP8 paths. Release image digests and abbreviated engine commits do not establish an exact executed image digest.
Why keep releases separate?
The k76 v2.0.0 concurrency series uses the earlier 16c9bd54788d12390838a65ce4a4ecda97fa5f1d GDN-MSE checkpoint, while v2.2.0 cold-prefix measurements use 03f4a0570496bbe741189f4a2a7d611679e85406. Both employ two directly connected Sparks, vLLM plus b12x and mixed NVFP4/MXFP8 paths. Release image digests and abbreviated engine commits do not establish an exact executed image digest.
Source: ursuciprian · pinned document
What does the k76 concurrency series measure?
The selected cells use llama-benchy 0.4.1.dev4+g0d4de4271, 2048 input, 512 requested output tokens, temperature 1.0, top-p 0.95, top-k 20 and thinking on. Prefix caching is enabled. Means and SD cover three runs in one boot, not independent boots. Aggregate output includes prompt-processing wall time; individual decode begins after the first token.
Source: ursuciprian · pinned document ursuciprian · pinned document
What does the later cold-prefix arm measure?
At C1 and temperature zero, v2.2.0 requests one output token after 16384, 65536 or 131072 input tokens. Four boots each contain three cold prompts. The SD describes variation between boot means. These observations measure first-token latency and prefill, not sustained generation, broad long-document accuracy or multi-user full-window serving.
Source: ursuciprian · pinned document ursuciprian · pinned document
Which conclusions are outside these data?
The two arms cannot be joined into a single unchanged deployment curve. Current request slots and KV capacity are configuration limits, and cold prefix does not mean cold filesystem pages or cold compilation state. Complete executed digests and some runtime identities remain null. The publisher is already represented by a separate one-Spark document in the catalog; this new document is not a new independent researcher.
Source: ursuciprian · pinned document ursuciprian · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| TP2 v2.0.0 C1 2048 per_request_decode_rate | per request decode rate | 88.697 ± 2.562 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C4 2048 per_request_decode_rate | per request decode rate | 48.631 ± 1.533 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C8 2048 per_request_decode_rate | per request decode rate | 33.236 ± 0.977 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C16 2048 per_request_decode_rate | per request decode rate | 22.142 ± 0.362 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C1 2048 aggregate_output_rate | aggregate output rate | 88.697 ± 2.562 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C4 2048 aggregate_output_rate | aggregate output rate | 156.645 ± 4.558 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C8 2048 aggregate_output_rate | aggregate output rate | 172.213 ± 16.348 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.0.0 C16 2048 aggregate_output_rate | aggregate output rate | 231.294 ± 4.91 SD tokens/smean ± sd of 3 runs, one boot | Original row ↗ |
| TP2 v2.2.0 C1 16384 ttft | ttft | 5.487 ± 0.045 SD smean of 4 boots x 3 cold prompts, sd between boots | Original row ↗ |
| TP2 v2.2.0 C1 16384 prefill_rate | prefill rate | 2986.0 ± 24.6 SD tokens/smean of 4 boots x 3 cold prompts, sd between boots | Original row ↗ |
| TP2 v2.2.0 C1 65536 ttft | ttft | 24.041 ± 0.057 SD smean of 4 boots x 3 cold prompts, sd between boots | Original row ↗ |
| TP2 v2.2.0 C1 65536 prefill_rate | prefill rate | 2726.1 ± 6.4 SD tokens/smean of 4 boots x 3 cold prompts, sd between boots | Original row ↗ |
| TP2 v2.2.0 C1 131072 ttft | ttft | 54.037 ± 0.256 SD smean of 4 boots x 3 cold prompts, sd between boots | Original row ↗ |
| TP2 v2.2.0 C1 131072 prefill_rate | prefill rate | 2425.6 ± 11.5 SD tokens/smean of 4 boots x 3 cold prompts, sd between boots | Original row ↗ |
TP2 v2.0.0 C1 2048 per_request_decode_rate · per request decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_req |
| source_campaign | k76-capability-matrix |
| concurrency | 1 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 2.562, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C4 2048 per_request_decode_rate · per request decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_req |
| source_campaign | k76-capability-matrix |
| concurrency | 4 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 1.533, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C8 2048 per_request_decode_rate · per request decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_req |
| source_campaign | k76-capability-matrix |
| concurrency | 8 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 0.977, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C16 2048 per_request_decode_rate · per request decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_req |
| source_campaign | k76-capability-matrix |
| concurrency | 16 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 0.362, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C1 2048 aggregate_output_rate · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_total |
| source_campaign | k76-capability-matrix |
| concurrency | 1 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 2.562, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C4 2048 aggregate_output_rate · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_total |
| source_campaign | k76-capability-matrix |
| concurrency | 4 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 4.558, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C8 2048 aggregate_output_rate · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_total |
| source_campaign | k76-capability-matrix |
| concurrency | 8 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 16.348, 'kind': 'SD across runs in one boot'}
TP2 v2.0.0 C16 2048 aggregate_output_rate · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 16c9bd54788d12390838a65ce4a4ecda97fa5f1d |
| recipe_release | v2.0.0 |
| experiment_date | 2026-10-10 |
| harness | llama-benchy 0.4.1.dev4+g0d4de4271 |
| source_run | 2x:results/k76-capability-matrix-20261010-1059/2x/d0.json |
| source_metric | tg_total |
| source_campaign | k76-capability-matrix |
| concurrency | 16 |
| input_tokens | 2048 |
| output_tokens | 512 |
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| thinking | True |
| cache_state | prefix caching enabled; fresh 2048-token task, not a cold-cache campaign |
| repetitions | mean ± sd of 3 runs, one boot |
| aggregate_denominator | all output tokens / wall time including prompt processing |
| per_request_denominator | one output stream after first token |
| release_image_digest | sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821 |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 4.91, 'kind': 'SD across runs in one boot'}
TP2 v2.2.0 C1 16384 ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 03f4a0570496bbe741189f4a2a7d611679e85406 |
| recipe_release | v2.2.0 |
| experiment_date | 2026-10-15 |
| harness | cold.py (k101) |
| source_run | 1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/ |
| source_metric | ttft_s |
| source_campaign | promotion A/B |
| concurrency | 1 |
| input_tokens | 16384 |
| output_tokens | 1 |
| temperature | 0 |
| top_p | Not specified |
| top_k | Not specified |
| thinking | Not specified |
| cache_state | cold prefix with no hit, not a cold OS page-cache claim |
| repetitions | mean of 4 boots x 3 cold prompts, sd between boots |
| aggregate_denominator | Not specified |
| per_request_denominator | Not specified |
| release_image_digest | Not specified |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 0.045, 'kind': 'SD between boot means; four boots x three cold prompts'}
TP2 v2.2.0 C1 16384 prefill_rate · prefill rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 03f4a0570496bbe741189f4a2a7d611679e85406 |
| recipe_release | v2.2.0 |
| experiment_date | 2026-10-15 |
| harness | cold.py (k101) |
| source_run | 1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/ |
| source_metric | pp |
| source_campaign | promotion A/B |
| concurrency | 1 |
| input_tokens | 16384 |
| output_tokens | 1 |
| temperature | 0 |
| top_p | Not specified |
| top_k | Not specified |
| thinking | Not specified |
| cache_state | cold prefix with no hit, not a cold OS page-cache claim |
| repetitions | mean of 4 boots x 3 cold prompts, sd between boots |
| aggregate_denominator | Not specified |
| per_request_denominator | Not specified |
| release_image_digest | Not specified |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 24.6, 'kind': 'SD between boot means; four boots x three cold prompts'}
TP2 v2.2.0 C1 65536 ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 03f4a0570496bbe741189f4a2a7d611679e85406 |
| recipe_release | v2.2.0 |
| experiment_date | 2026-10-15 |
| harness | cold.py (k101) |
| source_run | 1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/ |
| source_metric | ttft_s |
| source_campaign | promotion A/B |
| concurrency | 1 |
| input_tokens | 65536 |
| output_tokens | 1 |
| temperature | 0 |
| top_p | Not specified |
| top_k | Not specified |
| thinking | Not specified |
| cache_state | cold prefix with no hit, not a cold OS page-cache claim |
| repetitions | mean of 4 boots x 3 cold prompts, sd between boots |
| aggregate_denominator | Not specified |
| per_request_denominator | Not specified |
| release_image_digest | Not specified |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 0.057, 'kind': 'SD between boot means; four boots x three cold prompts'}
TP2 v2.2.0 C1 65536 prefill_rate · prefill rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 03f4a0570496bbe741189f4a2a7d611679e85406 |
| recipe_release | v2.2.0 |
| experiment_date | 2026-10-15 |
| harness | cold.py (k101) |
| source_run | 1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/ |
| source_metric | pp |
| source_campaign | promotion A/B |
| concurrency | 1 |
| input_tokens | 65536 |
| output_tokens | 1 |
| temperature | 0 |
| top_p | Not specified |
| top_k | Not specified |
| thinking | Not specified |
| cache_state | cold prefix with no hit, not a cold OS page-cache claim |
| repetitions | mean of 4 boots x 3 cold prompts, sd between boots |
| aggregate_denominator | Not specified |
| per_request_denominator | Not specified |
| release_image_digest | Not specified |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 6.4, 'kind': 'SD between boot means; four boots x three cold prompts'}
TP2 v2.2.0 C1 131072 ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 03f4a0570496bbe741189f4a2a7d611679e85406 |
| recipe_release | v2.2.0 |
| experiment_date | 2026-10-15 |
| harness | cold.py (k101) |
| source_run | 1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/ |
| source_metric | ttft_s |
| source_campaign | promotion A/B |
| concurrency | 1 |
| input_tokens | 131072 |
| output_tokens | 1 |
| temperature | 0 |
| top_p | Not specified |
| top_k | Not specified |
| thinking | Not specified |
| cache_state | cold prefix with no hit, not a cold OS page-cache claim |
| repetitions | mean of 4 boots x 3 cold prompts, sd between boots |
| aggregate_denominator | Not specified |
| per_request_denominator | Not specified |
| release_image_digest | Not specified |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 0.256, 'kind': 'SD between boot means; four boots x three cold prompts'}
TP2 v2.2.0 C1 131072 prefill_rate · prefill rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE |
| engine | vLLM + b12x |
| engine_version | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| system | two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled |
| node_count | 2 |
| parallelism | TP2 |
| weight_precision | NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies |
| extra_prefill_revision | 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd |
| kv_dtype | fp8 |
| draft_tokens | 4 |
| context_limit_tokens | 262144 |
| configured_max_requests | 16 |
| actual_output_tokens_per_request | Not specified |
| GPU_clock_cap_MHz | Not specified |
| model_revision | 03f4a0570496bbe741189f4a2a7d611679e85406 |
| recipe_release | v2.2.0 |
| experiment_date | 2026-10-15 |
| harness | cold.py (k101) |
| source_run | 1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/ |
| source_metric | pp |
| source_campaign | promotion A/B |
| concurrency | 1 |
| input_tokens | 131072 |
| output_tokens | 1 |
| temperature | 0 |
| top_p | Not specified |
| top_k | Not specified |
| thinking | Not specified |
| cache_state | cold prefix with no hit, not a cold OS page-cache claim |
| repetitions | mean of 4 boots x 3 cold prompts, sd between boots |
| aggregate_denominator | Not specified |
| per_request_denominator | Not specified |
| release_image_digest | Not specified |
| release_digest_is_proven_executed_digest | False |
{'standard_deviation': 11.5, 'kind': 'SD between boot means; four boots x three cold prompts'}
Original source-based interpretation; local Spark Qwen draft with Codex factual review, no site model benchmark.