# Two DGX Sparks: Read the Deployment Conditions A two-node model deployment is a combination of weights, runtime, attention cache, speculation and transport. A model name and a cable speed leave too much unspecified to reproduce it. This note reads [WeZZard’s pinned two-Spark campaign](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md), remeasured on 5 September 2026, alongside its detailed configuration ledger. It explains what to record before trying a similar deployment. These are community observations; Run AI Here has not run the models. ![Conceptual two-node deployment with separate local-memory spaces, a network link and configuration symbols.](/assets/dual-spark-conditions-concept.webp) AI-generated conceptual illustration. It is not a measured topology, a combined-memory claim or a photograph of the author’s cluster. ## Start with the campaign boundary The source reports two GB10 DGX Sparks, 121 GiB per node and a back-to-back 200 GbE link. Its corrected campaign has prefix caching off, 512 output tokens per request with `--ignore-eos`, full-length warmup for each workload and user shape, and median summaries from three campaigns. Served rate includes first-token waiting time in the request wall clock. [Original conditions](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L15). Each node retains its own memory and runtime state. Do not translate the two memory specifications into a promise of one freely allocatable memory pool. A launch-time context cap, an allocated KV pool and measured simultaneous users describe different constraints. ## Preserve the actual artifact and launch profile The artifact names and checkpoint revisions below come from the [detailed REPORT ledger](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md). Short revision prefixes are displayed here; the full revisions and separate source-document hashes are retained in JSON. | Campaign profile | Artifact / revision prefix | Weight format | Runtime image tag / parallelism | |---|---|---|---| | `ds-v3` | `deepseek-ai/DeepSeek-V4-Flash-0731` / `7872f01b` | MXFP4 experts | `vllm-dsv4:ray` / TP2, Ray | | `qwen-v3` | `RadixArk/Qwen3.8-Flash-Next-NVFP4` / `7b719225` | NVFP4 | `sglang-qwen38fn:sm121-ours` / TP2 | | `glm-v3` | `LibertAIDAI/GLM-5.3-Flash-NVFP4` / `aa28e1f5` | NVFP4 experts | `sglang-glm53:gb10-tilelang` / TP2, EP2 | The local image tags identify the reported environments, but a tag is not an immutable image digest. They are not a general installation command or an endorsement of an unverified image with the same name. ## Keep context, KV pool and concurrency separate | Profile | Allocated KV dtype / pool tokens | Speculation | Launch context cap | Measured users / launch maximum | |---|---|---|---:|---| | `ds-v3` | `fp8_ds_mla` / 517,484 | DSpark k=5, probabilistic | 262,144 | 1 and 6 / 12 | | `qwen-v3` | `bfloat16` / 161,728 | NEXTN, 2 draft tokens | 120,000 | 1 and 6 / unspecified | | `glm-v3` | `bfloat16` / 162,688 | DFlash2, 8 draft tokens | 131,072 | 1 and 8 / 8 | Sources: [corrected pools](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L90) and launch profiles [DeepSeek](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L928), [Qwen](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L562), [GLM](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L270). GLM’s static memory fraction is 0.90. The configured maximum is not a successful measurement at that load. Nor does a pool value establish a usable per-request context by division: engine accounting, shared state and scheduling need inspection. Preserve the input-token count for each measured cell; a workload label is insufficient. ## Verify the transport inside the container The source’s [interconnect investigation](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/interconnect.md) found that host networking worked while container verbs devices were absent and NCCL used sockets. A visible HCA listing alone did not prove active RDMA. For a comparable setup, inspect the devices exposed to the running containers and the transport actually used during a request. Retain observations from both nodes. A successful host bandwidth test and a server-ready message answer different questions from a completed distributed inference request. ## Separate historical tuning incidents from the final campaign The earlier [4 September speculation investigation](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/speculation.md#L44) recorded a Qwen pool change from 600,000 to 145,024 tokens after enabling MTP. That earlier value is not the corrected campaign’s 161,728-token pool. It illustrates why the allocated pool must be re-read after a configuration change, rather than supplying a universal MTP memory formula. That investigation also corrected an FP8-KV failure explanation: a single long prompt could trigger the chunked-prefill path. “Short prompt succeeded” therefore did not establish that the tested image supported long-context FP8 KV. The successful profile in the table uses bfloat16 KV. Keep failure records attached to their own build and settings. ## What to capture before calling a reproduction successful Record the exact artifact revision, conversion, image digest and executed runtime version. Read the allocated KV dtype and pool from startup evidence. Preserve speculation settings, context cap, prefix-cache state, actual input/output lengths, measured users and request failures. Warm the shapes you will measure on both nodes and retain clock and power observations under load. The [engine inventory](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/engines.md#L14) separately records a Qwen-image version observation on 4 September. It does not supply an immutable image digest binding all three corrected campaign launches, so those execution fields remain unknown here. GLM’s exact draft-model revision also remains unknown in this extraction. These profiles do not establish a cross-model ranking, current best settings or DeepSeek 1017 compatibility. Use the separate [1017 evidence note](/guides/deepseek-v41-flash-1017/) for checkpoint identity and the [Spark preflight checklist](/hardware/dgx-spark/deployment-checklist/) to prepare a run record. ## Original sources and conditions - WeZZard: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md - Document revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - SHA-256: b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469 - WeZZard: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md - Document revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - SHA-256: d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e - WeZZard: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/interconnect.md - Document revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - SHA-256: 9e7904db6d48fc44691c9f170c03f402c77480dcbf54f66bc107ef57ef0ea94f - WeZZard: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/speculation.md - Document revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - SHA-256: 46c64755b468e6e996e84fd1d19727160a9130bb3565731e7b5f26185c1dfe7f - WeZZard: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/engines.md - Document revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - SHA-256: 421632af313b5cfe7fca8b65ddabe0afb7b13e34229aca03e2af00c6a6f5452e Checked 2026-10-18. AI-assisted writing, source checked by Run AI Here. No independent model tests. Conditions and selected source facts are in the adjacent JSON export. ```json { "facts": [ { "label": "Physical link versus actual transport", "value": "200 GbE wire did not establish container RDMA; verbs devices and active NCCL transport had to be checked", "classification": "source_reported_configuration_incident", "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/interconnect.md#L9", "source_document_sha256": "9e7904db6d48fc44691c9f170c03f402c77480dcbf54f66bc107ef57ef0ea94f", "source_document_id": "spark-docs-interconnect", "source_line": 9 }, { "label": "Earlier speculation campaign pool", "value": { "date": "2026-09-04", "no_speculation_tokens": 600000, "with_speculation_tokens": 145024, "not_the_September_5_pool": true }, "classification": "source_reported_configuration_not_general_capacity", "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/speculation.md#L44", "source_document_sha256": "46c64755b468e6e996e84fd1d19727160a9130bb3565731e7b5f26185c1dfe7f", "source_document_id": "spark-docs-speculation", "source_line": 44 }, { "label": "Qwen image version observation", "value": { "observed_on": "2026-09-04", "image_tag": "sglang-qwen38fn:sm121-ours", "reported_engine_version": "0.0.0.dev1+gd91c3682b", "campaign_immutable_digest": null }, "classification": "source_reported_image_observation_not_immutable_campaign_version", "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/engines.md#L14", "source_document_sha256": "421632af313b5cfe7fca8b65ddabe0afb7b13e34229aca03e2af00c6a6f5452e", "source_document_id": "spark-docs-engines", "source_line": 14 }, { "label": "Qwen long-prompt FP8 KV failure", "value": "A single 30,301-token prompt also failed; the source identifies chunked prefill with a nonzero prefix, not concurrency itself, as the trigger in that image", "classification": "source_reported_failure_bound_to_build", "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/docs/speculation.md#L100", "source_document_sha256": "46c64755b468e6e996e84fd1d19727160a9130bb3565731e7b5f26185c1dfe7f", "source_document_id": "spark-docs-speculation", "source_line": 100 } ], "reported_results": [], "conditions": { "hardware": "2 x DGX Spark GB10 sm_121", "node_count": 2, "memory_per_node_gib": 121, "physical_interconnect_gbps": 200, "campaign_date": "2026-09-05", "prefix_cache": false, "output_tokens_per_request": 512, "ignore_eos": true, "warmup": "every workload/concurrency shape, full generation; each node has its own JIT cache", "repetitions": 3, "summary_statistic": "median of three campaigns", "served_rate_denominator": "all output tokens / whole request wall clock, including TTFT", "runtime_execution_immutable_digest": null, "shared_memory_pool_claim": false, "cross_model_ranking_allowed": false, "current_1017_reproduction_claim": false }, "deployment_profiles": [ { "id": "ds-v3", "model_artifact": "deepseek-ai/DeepSeek-V4-Flash-0731", "checkpoint_revision": "7872f01b1d1fe23eabc4c98b48bffcef5a386062", "weight_precision": "MXFP4 experts (block 32; source weight audit)", "engine": "vLLM", "image_tag": "vllm-dsv4:ray", "executed_engine_version": null, "image_digest": null, "parallelism": "TP2; Ray across two nodes", "context_limit_tokens": 262144, "configured_max_requests": 12, "measured_concurrency": [ 1, 6 ], "kv_dtype": "fp8_ds_mla", "kv_pool_tokens": 517484, "speculation": "DSpark k=5, probabilistic", "prefix_cache": false, "configuration_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L928", "source_document_sha256": "d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e", "source_document_id": "spark-REPORT", "source_line": 928 }, "artifact_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L591", "source_document_sha256": "d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e", "source_document_id": "spark-REPORT", "source_line": 591 }, "pool_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L97", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469", "source_document_id": "spark-pair-moe-served-rate", "source_line": 97 } }, { "id": "qwen-v3", "model_artifact": "RadixArk/Qwen3.8-Flash-Next-NVFP4", "checkpoint_revision": "7b719225242aacd3dbd3f9407468c2ee9a9d2594", "weight_precision": "NVFP4", "engine": "SGLang", "image_tag": "sglang-qwen38fn:sm121-ours", "executed_engine_version": null, "image_digest": null, "parallelism": "TP2; two nodes", "context_limit_tokens": 120000, "configured_max_requests": null, "measured_concurrency": [ 1, 6 ], "kv_dtype": "bfloat16", "kv_pool_tokens": 161728, "speculation": "NEXTN; one step, top-k 1, draft tokens 2", "prefix_cache": false, "configuration_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L562", "source_document_sha256": "d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e", "source_document_id": "spark-REPORT", "source_line": 562 }, "artifact_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L303", "source_document_sha256": "d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e", "source_document_id": "spark-REPORT", "source_line": 303 }, "pool_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L103", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469", "source_document_id": "spark-pair-moe-served-rate", "source_line": 103 } }, { "id": "glm-v3", "model_artifact": "LibertAIDAI/GLM-5.3-Flash-NVFP4", "checkpoint_revision": "aa28e1f54130286c95fee10d0705c74ce8743734", "weight_precision": "NVFP4 experts (block 16; source weight audit)", "engine": "SGLang", "image_tag": "sglang-glm53:gb10-tilelang", "executed_engine_version": null, "image_digest": null, "parallelism": "TP2 / EP2; two nodes", "context_limit_tokens": 131072, "configured_max_requests": 8, "measured_concurrency": [ 1, 8 ], "kv_dtype": "bfloat16", "kv_pool_tokens": 162688, "speculation": "DFlash2; draft tokens 8", "prefix_cache": false, "mem_fraction_static": 0.9, "draft_artifact_revision": null, "configuration_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L270", "source_document_sha256": "d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e", "source_document_id": "spark-REPORT", "source_line": 270 }, "artifact_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/REPORT.md#L18", "source_document_sha256": "d08b9489656537f7b4a5d11cf4f07ed9f58c9b86ba6c7badd20ca52260a8224e", "source_document_id": "spark-REPORT", "source_line": 18 }, "pool_evidence": { "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L106", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469", "source_document_id": "spark-pair-moe-served-rate", "source_line": 106 } } ] } ```