# Two-Spark MoE: served-rate and warmup audit Two-node serving study that distinguishes end-to-end served rate from decode rate, and audits cache, warmup and checkpoint identity. - Type: Performance report - Evidence: source_reported; site reproduced: false - Model family: Multiple - Model: Qwen3.8-Flash-Next / GLM-5.3-Flash / DeepSeek-V4-Flash-0731 - Original author: WeZZard - Original project: https://github.com/WeZZard/dgx-spark-bench - Pinned document: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md - Source revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - Source commit time (not experiment date): 2026-09-06T00:35:29Z - Checked: 2026-10-17 - Experiment date: None - Runtime topics: vLLM, SGLang - Hardware topics: spark, nvidia - Reported setup: {"system": "2 x DGX Spark; 121 GiB unified memory each; 200 GbE", "node_count": 2, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": null} - Performance figures: attributed selections with per-point conditions; no cross-source ranking ## Conditions Unknown is null. Recipe options are not measured benchmark settings. ```json { "conditions": { "model_revision": null, "weight_hash": null, "runtime_execution_revision": null, "input_tokens": null, "output_tokens": 512, "concurrency": null, "cache_state": null, "quantization": null, "speculative_decoding": null, "temperature": null, "top_p": null }, "test_profiles": [ { "label": "Fixed-length served rate", "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md", "conditions": { "model_revision": null, "weight_hash": null, "runtime_execution_revision": null, "input_tokens": null, "output_tokens": 512, "concurrency": null, "cache_state": "full-shape warmup; per-node JIT cache; see row prefix-cache controls", "quantization": null, "speculative_decoding": "model-dependent NEXTN / DFlash2 / no-spec arms", "temperature": null, "top_p": null, "measurement_denominator": "whole request wall clock including TTFT" } } ], "recipe_parameters": null, "deployment_options": [] } ``` ## Reading notes - Each request is forced to emit 512 tokens; served rate includes time to first token. - Every result retains model, weights, KV type, engine, users and KV pool. ## Limits - Text-0731 and Vision-Exp are separate checkpoints. - A draft model consumes KV capacity; compare the exact per-row configuration. ## License declaration {"declared_spdx": "MIT", "scope": "Repository metadata only; per-file, model, data and conversion terms remain separate."} ## Editorial answer The report divides all generated tokens by the complete request wall clock, including first-token delay. Every current table request is forced to 512 tokens with ignore_eos. Earlier naturally stopped outputs and 64-token correctness probes have different denominators and remain in the historical report rather than being merged into this cohort. By Run AI Here editorial ยท 2026-10-17 AI-assisted source reading and original editorial synthesis; no independent model execution. ## What does served rate include? The report divides all generated tokens by the complete request wall clock, including first-token delay. Every current table request is forced to 512 tokens with ignore_eos. Earlier naturally stopped outputs and 64-token correctness probes have different denominators and remain in the historical report rather than being merged into this cohort. Sources: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L25 ## Why were earlier results replaced? The author remeasured the campaign on 5 September after discovering a slow-clock GPU and unequal prefix-cache policy. The corrected runs disable prefix caching for all three models, exercise complete workload shapes during warmup and retain per-node JIT caches. Earlier measurements remain visible as history; they should not be treated as the corrected cohort. Sources: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L45 ## What can the long-context rows establish? At one user in the long agentic workload, the reported medians are 22.5, 21.0 and 17.7 served tokens/s, with different between-campaign spreads. The author labels this row not separated by the data. Different draft models and KV pools remain attached to each model, and the high-concurrency rows do not even use the same number of users across models. Sources: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L70 ## Why are failures and checkpoint identity useful? The README reports a Qwen FP8-KV configuration that failed on chunked input and GLM backend attempts that did not complete a forward pass. Its DeepSeek current rows use the text 0731 checkpoint, not Vision-Exp and not V4.1 Flash 1017. These failures and identities explain why a recipe label cannot substitute for the actual six-field serving configuration. Sources: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L50 ## Selected source-reported figures Each point retains its own configuration, measurement unit and source. No site reproduction or cross-source ranking. ```json [ { "label": "DeepSeek V4 Flash 0731, long agentic", "metric": "served_rate", "value": 22.5, "unit": "tokens/s", "statistic": "median of three campaigns", "uncertainty": { "spread_ratio": 1.04 }, "conditions": { "system": "2 x DGX Spark; 121 GiB unified memory each; 200 GbE", "node_count": 2, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": "2026-09-05", "output_tokens": 512, "ignore_eos": true, "prefix_cache": false, "warmup": "all workload/user shapes, full generation", "repetitions": 3, "concurrency": 1, "measurement_denominator": "whole request wall clock including TTFT", "workload": "agentic 33k context", "model": "DeepSeek-V4-Flash-0731", "kv": "fp8_ds_mla", "kv_pool_tokens": 517484, "draft": "DSpark k5 probabilistic" }, "evidence_state": "source_reported", "site_reproduced": false, "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L70", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469" }, { "label": "Qwen3.8 Flash-Next, long agentic", "metric": "served_rate", "value": 21.0, "unit": "tokens/s", "statistic": "median of three campaigns", "uncertainty": { "spread_ratio": 1.01 }, "conditions": { "system": "2 x DGX Spark; 121 GiB unified memory each; 200 GbE", "node_count": 2, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": "2026-09-05", "output_tokens": 512, "ignore_eos": true, "prefix_cache": false, "warmup": "all workload/user shapes, full generation", "repetitions": 3, "concurrency": 1, "measurement_denominator": "whole request wall clock including TTFT", "workload": "agentic 33k context", "model": "Qwen3.8-Flash-Next NVFP4", "kv": "BF16", "kv_pool_tokens": 161728, "draft": "NEXTN" }, "evidence_state": "source_reported", "site_reproduced": false, "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L70", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469" }, { "label": "GLM5.3 Flash, long agentic", "metric": "served_rate", "value": 17.7, "unit": "tokens/s", "statistic": "median of three campaigns", "uncertainty": { "spread_ratio": 1.32 }, "conditions": { "system": "2 x DGX Spark; 121 GiB unified memory each; 200 GbE", "node_count": 2, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": "2026-09-05", "output_tokens": 512, "ignore_eos": true, "prefix_cache": false, "warmup": "all workload/user shapes, full generation", "repetitions": 3, "concurrency": 1, "measurement_denominator": "whole request wall clock including TTFT", "workload": "agentic 33k context", "model": "GLM-5.3-Flash", "kv": "BF16", "kv_pool_tokens": 162688, "draft": "DFlash2" }, "evidence_state": "source_reported", "site_reproduced": false, "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L70", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469" } ] ```