# Qwen Flash-Next: Concurrency Has a Latency Cost Qwen3.8-Flash-Next can serve more requests while each user waits longer. The useful question is which request mix was actually tested. A single-Spark SGLang study publishes a ShareGPT concurrency sweep; a separate two-Spark vLLM/b12x study publishes task throughput and cold-prefix latency. Their different checkpoints, cache policies and hardware prevent a controlled engine ranking. A third operator's memory failure explains why a configured window and a scheduler limit are insufficient capacity evidence. ![Two distinct Spark benchmark cohorts with different request shapes, cache settings and repeat counts.](/assets/qwen-flash-next-spark-concurrency-source.webp) ## Single Spark: read latency beside aggregate throughput The [nabe2030 recipe](https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/README.md) uses `senfu/Qwen3.8-Flash-Next-NVFP4` at `5d37b3b3711d8406174b96ff950c0aa16324b266`, SGLang commit `4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b`, and NVMe file-backed PLE. Experts and the LM head use NVFP4; the dense path uses FP8. Those details distinguish it from the other Flash-Next conversions on this site. Its [aggregated result table](https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md) uses `vllm bench serve` with 200 ShareGPT prompts, thinking off, MTP off and radix caching disabled. One warmup request is discarded. The C1 summary is the median of three trials; the higher-concurrency points are single trials. All selected senfu cases completed 200 requests without errors. Individual prompt/output distributions, temperature and per-request cache history are not supplied. | Concurrent requests | Aggregate output, tokens/s | Median first token, ms | Median time per output token, ms | |---:|---:|---:|---:| | 1 | 26.97 | 221.23 | 36.14 | | 8 | 67.68 | 526.78 | 113.46 | | 16 | 77.48 | 686.27 | 198.00 | | 32 | 89.35 | 1,017.98 | 343.57 | ![Single-Spark median TTFT increases from 221.23 ms at C1 to 1017.98 ms at C32.](/assets/qwen-sglang-ttft-concurrency.webp) For a shared endpoint, more total output does not preserve the interactive experience. Keep first-token wait and token spacing beside aggregate throughput when deciding how many users to admit. Do not estimate an individual stream's speed by dividing aggregate tokens/s by concurrency: the timing definitions and request completion patterns need not match. Single higher-load trials also provide no run-to-run error bar. These results are useful measurements of this workload, rather than a latency guarantee for a different team. ## Two Sparks: preserve the older concurrency cohort The [TP2 repository](https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/README.md) contains several generations of measurements. The selected k76 concurrency series belongs to recipe v2.0.0 on October 10, using two directly connected Sparks and `llama-benchy 0.4.1.dev4+g0d4de4271`. It requests 2,048 input and 512 output tokens, temperature 1.0, top-p 0.95, top-k 20 and thinking on, with prefix caching enabled. It reports three runs within one boot. The [version ledger](https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/VERSIONS.md) binds v2.0.0 to GDN-MSE checkpoint `16c9bd54788d12390838a65ce4a4ecda97fa5f1d`. The later v2.1.0/v2.2.0 checkpoint ends in `...85406` and includes a different MTP drafter. Keeping the old checkpoint identity prevents newer recipe defaults from silently rewriting historical results. The ledger's release-image digest is retained separately from the unreported digest of the exact executed test container. | Concurrency | Aggregate output, mean ± run SD | Per-request decode, mean ± run SD | |---:|---:|---:| | 1 | 88.697 ± 2.562 | 88.697 ± 2.562 | | 4 | 156.645 ± 4.558 | 48.631 ± 1.533 | | 8 | 172.213 ± 16.348 | 33.236 ± 0.977 | | 16 | 231.294 ± 4.910 | 22.142 ± 0.362 | Both columns use tokens/s. The [machine-readable source CSV](https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv) retains the exact values and original run paths. Aggregate throughput divides all output by elapsed wall time including prompt processing. Per-request decode describes one stream after its first token. Their denominators differ; neither column can be reconstructed by simply multiplying or dividing the other by the number of users. ![Dual-Spark per-request decode falls from 88.697 to 22.142 tokens per second across C1 to C16.](/assets/qwen-tp2-per-request-concurrency.webp) The larger C8 aggregate SD remains visible. It describes dispersion in those three runs; it does not identify a cause, quantify between-boot variation or establish a confidence interval. Reporting mean and SD does not itself require a normally distributed population. A scheduling explanation would need more evidence than the summary cells provide. ## Long-context latency comes from a separate release The v2.2.0 cold-prefix arm uses the later checkpoint, C1, temperature zero and one requested output token. It averages four boots with three cold prompts per boot. SD is between boot means. The [method and build notes](https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/BENCHMARKS.md) separate this prefill work from the older concurrency campaign. | Actual input tokens | First-token latency, mean ± boot SD | Prefill, mean ± boot SD, tokens/s | |---:|---:|---:| | 16,384 | 5.487 ± 0.045 s | 2,986.0 ± 24.6 | | 65,536 | 24.041 ± 0.057 s | 2,726.1 ± 6.4 | | 131,072 | 54.037 ± 0.256 s | 2,425.6 ± 11.5 | ![Dual-Spark cold-prefix TTFT is 5.487, 24.041 and 54.037 seconds at the three measured input lengths.](/assets/qwen-tp2-cold-context-latency.webp) A cold prefix means there was no matching prefix-cache hit. It does not establish cold weight pages, cold compilation caches or a cold operating-system page cache. One requested output token measures the route to the first answer token, rather than long-answer generation. These points cannot prove that sixteen simultaneous 128K prompts fit or retain the same latency. Read the earlier [memory/context explanation](/guides/qwen-flash-next-memory-context/) for the distinction between a shared KV pool and a request ceiling. ## Memory failure is useful evidence too In a separate [first-person dual-GB10 operator report](https://forums.developer.nvidia.com/t/385731/1), jcagle describes vLLM 0.30.0 with MiaAI overlays, FP8 KV, three MTP drafts, 16 sequence slots and capped GPU clocks. This is a third configuration, rather than another repetition of k76. The report is post version 3, updated October 13; its public URL is mutable, so a dated snapshot hash is retained in the case export instead of claiming an immutable commit URL. The operator reports an eight-request deep-prefill burst that reached 2.4 GiB head-node MemAvailable at GMU 0.80 before earlyoom killed the engine. With GMU 0.78, the reported eight-by-64K-class load completed at 2,510 combined prefill tokens/s and 117 seconds TTFT, with 3.8 GiB available on the head. Exact individual token counts, output lengths, sampling and repeated-run dispersion are missing. These observations demonstrate the operator's outcome; they do not make 0.78 a universal safe setting or isolate GMU as the only causal variable. The single-Spark SGLang recipe separately lists a 49,152-token per-request window, 393,216 total tokens and eight slots for one configuration. That configuration is not evidence that every slot was simultaneously filled. Its 3.04 GB KV figure at 128K is a source-labelled allocation, with byte convention unverified, rather than whole-model peak RAM. The [KV and speculation guide](/guides/kv-cache-speculative-decoding/) explains why those distinctions matter. ## A practical reading order Choose the source matching your checkpoint, runtime and hardware. Next, compare the tested prompt/output mix with your actual workload. Then inspect aggregate output, per-request decode, first-token delay and host memory together. Finally, mark configured capacities that the source did not actually fill. This is an editorial method for choosing what to verify, not a request to reproduce these benchmarks automatically. ### Does C32 throughput prove C32 long-context serving? No. The single-Spark C32 ShareGPT mix does not publish a simultaneous full-window fill. Its prompt distribution is also absent. ### Can the two Spark articles rank SGLang against vLLM? No controlled engine comparison is available here. Checkpoints, nodes, cache behavior, MTP and request shapes differ. ### Which missing fields should an AI retain? Keep null execution digests, unknown request lengths, trial counts and memory scope. Use the [For AI filters](/for-ai/) to retrieve complete reports and their conditions rather than a speed-only fragment. ## Original sources and conditions - nabe2030: https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/README.md - Document revision: 8a161f2bee2e2d85b689a2d457ae68bead3929a4 - SHA-256: 8a27445102511702ff42fa60cfcc469583de37c4c0bd031c43cd40db7c0db9f4 - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/README.md - Document revision: b384f4a0e3ff0d563980befe52eb3e6964a5b6b1 - SHA-256: f3bd45cb1f01c287ef00f97b98e0d5c5d66576d5be34baaefce5c108242ad1cf - nabe2030: https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md - Document revision: 8a161f2bee2e2d85b689a2d457ae68bead3929a4 - SHA-256: cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297 - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv - Document revision: b384f4a0e3ff0d563980befe52eb3e6964a5b6b1 - SHA-256: a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/BENCHMARKS.md - Document revision: b384f4a0e3ff0d563980befe52eb3e6964a5b6b1 - SHA-256: 5645ff4e129b3e2766672e092c65333b073b5550ad89aa6374cbd81ce1da7776 - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/VERSIONS.md - Document revision: b384f4a0e3ff0d563980befe52eb3e6964a5b6b1 - SHA-256: eb9ec6435802430a1be7302b445c67ceb48da9cf5e1ba32e939b5f7aaf9b100e - jcagle: https://forums.developer.nvidia.com/t/385731/1 - Document revision: 3 - SHA-256: 95ae1019d350e10f941b22110e09c3d65372413fa3139ae2b676b7eb244267a0 Checked 2026-10-18. AI-assisted writing, source checked by Run AI Here. No independent model tests. Conditions and selected source facts are in the adjacent JSON export. ```json { "facts": [], "reported_results": [ { "label": "single Spark senfu C1 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 26.97, "unit": "tokens/s", "statistic": "median of three trials", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 1, "repetitions": 3, "summary_kind": "median of three trials" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 9, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L9", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C1 median_ttft", "metric": "median_ttft", "value": 221.23, "unit": "ms", "statistic": "median of three trials", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 1, "repetitions": 3, "summary_kind": "median of three trials" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 9, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L9", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C1 median_time_per_output_token", "metric": "median_time_per_output_token", "value": 36.14, "unit": "ms", "statistic": "median of three trials", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 1, "repetitions": 3, "summary_kind": "median of three trials" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 9, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L9", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C8 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 67.68, "unit": "tokens/s", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 8, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 10, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L10", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C8 median_ttft", "metric": "median_ttft", "value": 526.78, "unit": "ms", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 8, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 10, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L10", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C8 median_time_per_output_token", "metric": "median_time_per_output_token", "value": 113.46, "unit": "ms", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 8, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 10, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L10", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C16 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 77.48, "unit": "tokens/s", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 16, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 11, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L11", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C16 median_ttft", "metric": "median_ttft", "value": 686.27, "unit": "ms", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 16, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 11, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L11", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C16 median_time_per_output_token", "metric": "median_time_per_output_token", "value": 198.0, "unit": "ms", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 16, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 11, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L11", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C32 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 89.35, "unit": "tokens/s", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 32, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 12, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L12", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C32 median_ttft", "metric": "median_ttft", "value": 1017.98, "unit": "ms", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 32, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 12, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L12", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "single Spark senfu C32 median_time_per_output_token", "metric": "median_time_per_output_token", "value": 343.57, "unit": "ms", "statistic": "single trial", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "senfu/Qwen3.8-Flash-Next-NVFP4", "model_revision": "5d37b3b3711d8406174b96ff950c0aa16324b266", "engine": "SGLang", "runtime_execution_revision": "4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b", "executed_image_digest": null, "system": "one DGX Spark GB10, 128 GB unified memory; NVMe file-backed PLE", "node_count": 1, "weight_precision": "NVFP4 experts and LM head; FP8 dense path", "kv_dtype": null, "draft_tokens": 0, "thinking": false, "temperature": null, "top_p": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "request_count": 200, "cache_state": "radix/prefix cache explicitly disabled; one warmup discarded", "harness": "vllm bench serve, ShareGPT 200 prompts", "experiment_date": null, "context_limit_tokens": null, "executed_context_for_each_sweep_unreported": true, "PLE_backing": "NVMe mmap", "exact_prompt_and_output_distributions_unreported": true, "concurrency": 32, "repetitions": 1, "summary_kind": "single trial" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-sglang-results", "source_document_sha256": "cf77f1feb1a37ab7fcdd8073ca0ff3b5fe78b8c47cf261fd917ae69fcb018297", "source_line": 12, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/results/speed_senfu_vs_radixark.md#L12", "source_record_id": "qwen38-flash-next-sglang-concurrency" }, { "label": "TP2 v2.0.0 C1 2048 per_request_decode_rate", "metric": "per_request_decode_rate", "value": 88.697, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 2.562, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_req", "source_campaign": "k76-capability-matrix", "concurrency": 1, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1142, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1142", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C4 2048 per_request_decode_rate", "metric": "per_request_decode_rate", "value": 48.631, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 1.533, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_req", "source_campaign": "k76-capability-matrix", "concurrency": 4, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1146, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1146", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C8 2048 per_request_decode_rate", "metric": "per_request_decode_rate", "value": 33.236, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 0.977, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_req", "source_campaign": "k76-capability-matrix", "concurrency": 8, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1148, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1148", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C16 2048 per_request_decode_rate", "metric": "per_request_decode_rate", "value": 22.142, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 0.362, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_req", "source_campaign": "k76-capability-matrix", "concurrency": 16, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1150, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1150", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C1 2048 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 88.697, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 2.562, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_total", "source_campaign": "k76-capability-matrix", "concurrency": 1, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1152, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1152", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C4 2048 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 156.645, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 4.558, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_total", "source_campaign": "k76-capability-matrix", "concurrency": 4, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1156, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1156", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C8 2048 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 172.213, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 16.348, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_total", "source_campaign": "k76-capability-matrix", "concurrency": 8, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1158, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1158", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.0.0 C16 2048 aggregate_output_rate", "metric": "aggregate_output_rate", "value": 231.294, "unit": "tokens/s", "statistic": "mean ± sd of 3 runs, one boot", "uncertainty": { "standard_deviation": 4.91, "kind": "SD across runs in one boot" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "16c9bd54788d12390838a65ce4a4ecda97fa5f1d", "recipe_release": "v2.0.0", "experiment_date": "2026-10-10", "harness": "llama-benchy 0.4.1.dev4+g0d4de4271", "source_run": "2x:results/k76-capability-matrix-20261010-1059/2x/d0.json", "source_metric": "tg_total", "source_campaign": "k76-capability-matrix", "concurrency": 16, "input_tokens": 2048, "output_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "thinking": true, "cache_state": "prefix caching enabled; fresh 2048-token task, not a cold-cache campaign", "repetitions": "mean ± sd of 3 runs, one boot", "aggregate_denominator": "all output tokens / wall time including prompt processing", "per_request_denominator": "one output stream after first token", "release_image_digest": "sha256:8abb60eeeb79ca13680341c17d87975801da1218c06df2505856b23da7ed1821", "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1160, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1160", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.2.0 C1 16384 ttft", "metric": "ttft", "value": 5.487, "unit": "s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "standard_deviation": 0.045, "kind": "SD between boot means; four boots x three cold prompts" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "recipe_release": "v2.2.0", "experiment_date": "2026-10-15", "harness": "cold.py (k101)", "source_run": "1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/", "source_metric": "ttft_s", "source_campaign": "promotion A/B", "concurrency": 1, "input_tokens": 16384, "output_tokens": 1, "temperature": 0, "top_p": null, "top_k": null, "thinking": null, "cache_state": "cold prefix with no hit, not a cold OS page-cache claim", "repetitions": "mean of 4 boots x 3 cold prompts, sd between boots", "aggregate_denominator": null, "per_request_denominator": null, "release_image_digest": null, "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1262, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1262", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.2.0 C1 16384 prefill_rate", "metric": "prefill_rate", "value": 2986.0, "unit": "tokens/s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "standard_deviation": 24.6, "kind": "SD between boot means; four boots x three cold prompts" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "recipe_release": "v2.2.0", "experiment_date": "2026-10-15", "harness": "cold.py (k101)", "source_run": "1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/", "source_metric": "pp", "source_campaign": "promotion A/B", "concurrency": 1, "input_tokens": 16384, "output_tokens": 1, "temperature": 0, "top_p": null, "top_k": null, "thinking": null, "cache_state": "cold prefix with no hit, not a cold OS page-cache claim", "repetitions": "mean of 4 boots x 3 cold prompts, sd between boots", "aggregate_denominator": null, "per_request_denominator": null, "release_image_digest": null, "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1263, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1263", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.2.0 C1 65536 ttft", "metric": "ttft", "value": 24.041, "unit": "s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "standard_deviation": 0.057, "kind": "SD between boot means; four boots x three cold prompts" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "recipe_release": "v2.2.0", "experiment_date": "2026-10-15", "harness": "cold.py (k101)", "source_run": "1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/", "source_metric": "ttft_s", "source_campaign": "promotion A/B", "concurrency": 1, "input_tokens": 65536, "output_tokens": 1, "temperature": 0, "top_p": null, "top_k": null, "thinking": null, "cache_state": "cold prefix with no hit, not a cold OS page-cache claim", "repetitions": "mean of 4 boots x 3 cold prompts, sd between boots", "aggregate_denominator": null, "per_request_denominator": null, "release_image_digest": null, "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1264, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1264", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.2.0 C1 65536 prefill_rate", "metric": "prefill_rate", "value": 2726.1, "unit": "tokens/s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "standard_deviation": 6.4, "kind": "SD between boot means; four boots x three cold prompts" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "recipe_release": "v2.2.0", "experiment_date": "2026-10-15", "harness": "cold.py (k101)", "source_run": "1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/", "source_metric": "pp", "source_campaign": "promotion A/B", "concurrency": 1, "input_tokens": 65536, "output_tokens": 1, "temperature": 0, "top_p": null, "top_k": null, "thinking": null, "cache_state": "cold prefix with no hit, not a cold OS page-cache claim", "repetitions": "mean of 4 boots x 3 cold prompts, sd between boots", "aggregate_denominator": null, "per_request_denominator": null, "release_image_digest": null, "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1265, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1265", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.2.0 C1 131072 ttft", "metric": "ttft", "value": 54.037, "unit": "s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "standard_deviation": 0.256, "kind": "SD between boot means; four boots x three cold prompts" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "recipe_release": "v2.2.0", "experiment_date": "2026-10-15", "harness": "cold.py (k101)", "source_run": "1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/", "source_metric": "ttft_s", "source_campaign": "promotion A/B", "concurrency": 1, "input_tokens": 131072, "output_tokens": 1, "temperature": 0, "top_p": null, "top_k": null, "thinking": null, "cache_state": "cold prefix with no hit, not a cold OS page-cache claim", "repetitions": "mean of 4 boots x 3 cold prompts, sd between boots", "aggregate_denominator": null, "per_request_denominator": null, "release_image_digest": null, "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1266, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1266", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" }, { "label": "TP2 v2.2.0 C1 131072 prefill_rate", "metric": "prefill_rate", "value": 2425.6, "unit": "tokens/s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "standard_deviation": 11.5, "kind": "SD between boot means; four boots x three cold prompts" }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "executed_image_digest": null, "system": "two DGX Spark GB10, 128 GB unified memory each; ConnectX-7 direct-cabled", "node_count": 2, "parallelism": "TP2", "weight_precision": "NVFP4 routed experts and decode GDN; MXFP8 dense/attention and prefill GDN copies", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "kv_dtype": "fp8", "draft_tokens": 4, "context_limit_tokens": 262144, "configured_max_requests": 16, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "recipe_release": "v2.2.0", "experiment_date": "2026-10-15", "harness": "cold.py (k101)", "source_run": "1x:results/k101s-1x-gdn-prefill-views-ab-20261015-1600/tp2/2x/", "source_metric": "pp", "source_campaign": "promotion A/B", "concurrency": 1, "input_tokens": 131072, "output_tokens": 1, "temperature": 0, "top_p": null, "top_k": null, "thinking": null, "cache_state": "cold prefix with no hit, not a cold OS page-cache claim", "repetitions": "mean of 4 boots x 3 cold prompts, sd between boots", "aggregate_denominator": null, "per_request_denominator": null, "release_image_digest": null, "release_digest_is_proven_executed_digest": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen-tp2-csv", "source_document_sha256": "a82bffe4f0e33f4243db9549c02fb8e9379e5faa62825a35a7b087cd30b4996e", "source_line": 1267, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2/blob/b384f4a0e3ff0d563980befe52eb3e6964a5b6b1/docs/data/capability.csv#L1267", "source_record_id": "qwen38-flash-next-tp2-concurrency-context" } ], "conditions": { "cross_engine_controlled_comparison": false, "concurrency_sweep_is_full_window_test": false, "one_boot_SD_is_boot_SD": false, "independent_site_runs": 0 }, "case_reports": [ { "id": "qwen-dual-gmu080", "classification": "source_reported_host_oom", "model": "Qwen3.8-Flash-Next", "engine": "vLLM 0.30.0 plus MiaAI overlays", "node_count": 2, "model_revision": null, "executed_image_digest": null, "GMU": 0.8, "concurrency": 8, "prompt_tokens_per_request_reported": "64K-class deep-prefill burst; exact individual counts absent", "minimum_head_MemAvailable_GiB": 2.4, "outcome": "earlyoom killed engine", "source_post_version": 3, "source_updated_at": "2026-10-13T22:17:54.906Z", "source_url": "https://forums.developer.nvidia.com/t/385731/1", "source_snapshot_sha256": "95ae1019d350e10f941b22110e09c3d65372413fa3139ae2b676b7eb244267a0", "immutable_public_revision_URL": false }, { "id": "qwen-dual-gmu078", "classification": "source_reported_deep_prefill_completion", "model": "Qwen3.8-Flash-Next", "engine": "vLLM 0.30.0 plus MiaAI overlays", "node_count": 2, "model_revision": null, "executed_image_digest": null, "GMU": 0.78, "concurrency": 8, "input_length_source_label": "64K each; exact token accounting absent", "aggregate_prefill_tokens_per_second": 2510, "TTFT_seconds": 117, "minimum_head_MemAvailable_GiB": 3.8, "output_tokens": null, "thinking": null, "temperature": null, "repetitions": null, "source_post_version": 3, "source_updated_at": "2026-10-13T22:17:54.906Z", "source_url": "https://forums.developer.nvidia.com/t/385731/1", "source_snapshot_sha256": "95ae1019d350e10f941b22110e09c3d65372413fa3139ae2b676b7eb244267a0", "immutable_public_revision_URL": false } ], "memory_observations": [ { "kind": "source_reported_KV_allocation_at_context", "context_limit_tokens": 131072, "KV_source_GB": 3.04, "GB_byte_convention_verified": false, "whole_model_peak_RAM": null, "source_document_id": "qwen-sglang-readme", "source_document_sha256": "8a27445102511702ff42fa60cfcc469583de37c4c0bd031c43cd40db7c0db9f4", "source_line": 19, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/README.md#L19" }, { "kind": "source_recipe_slots_not_a_completed_long_batch", "per_request_window_tokens": 49152, "shared_total_tokens": 393216, "request_slots": 8, "all_slots_full_simultaneously_proven": false, "source_document_id": "qwen-sglang-readme", "source_document_sha256": "8a27445102511702ff42fa60cfcc469583de37c4c0bd031c43cd40db7c0db9f4", "source_line": 134, "evidence_url": "https://github.com/nabe2030/flashnext-dgx-spark/blob/8a161f2bee2e2d85b689a2d457ae68bead3929a4/README.md#L134" } ] } ```