GB10 Long Context: Read the Failure Before Changing Flags

A long prompt can expose a kernel path that a short chat never exercises. On a pair of GB10 Sparks, first identify the failed layer: dtype handling, architecture support, per-block shared memory, correctness or container transport. Preserve the exact build and request shape before changing a setting.

This note interprets historical records from WeZZard’s dgx-spark-bench, pinned at 25e8dbf1b6113dd0b3f12d27a670fb284944d260. The relevant incidents are from September 2026; the documents were checked on 18 October. They do not establish the behavior of today’s upstream releases. Run AI Here did not reproduce these model runs.

Figure: a conceptual incident-review diagram generated as SVG by the local Spark model and reviewed for publication. It contains no measured timing or hardware topology.

Four evidence states, four different conclusions

Case 1: a single user can trigger the FP8 KV crash

The Qwen probe used RadixArk/Qwen3.8-Flash-Next-NVFP4, revision 7b719225242aacd3dbd3f9407468c2ee9a9d2594, with sglang-qwen38fn:sm121-ours. Its log identifies SGLang 0.0.0.dev1+gd91c3682b, float8_e4m3fn KV and a 310,656-token allocated pool. A short single-user control completed 64 output tokens and retrieved its needle. A 30,301-token single-user prompt then hit Unsupported rhs dtype fp8e4nv and terminated the scheduler. Probe and diagnosis.

The source traces this to nonzero prefix lengths in chunked prefill. With an 8,192-token chunk size, a longer request reaches a path reading the FP8 paged pool instead of fresh bf16 K/V. Concurrency alone is therefore the wrong explanation. The document proposes a compatible build or dtype conversion; it supplies no verified fix commit. Raising the chunk size moves the boundary. It does not demonstrate a corrected kernel.

The failed probe’s output count and the image digest remain unknown. Do not borrow the 332,480-token pool from an earlier campaign: it is a different observation in the same document.

Case 2: the 120,000-token guard is not a correctness result

The QSA document carries a warning about corruption beyond roughly 120K and contrasts it with a recipe requesting 262,144 tokens. The author inspected the image, did not find the named successor attention path, and capped the launcher at 120,000. The proposed 131K/262K retrieval ladder is explicitly still pending in this snapshot. Warning, guard and unperformed experiment.

Preserve the distinction between an operating guard and a measured maximum safe context. No completed ladder, corruption threshold or upstream fix verification is available here. The named pr36845.diff is a successor reference, not proof of a merged or validated fix. A future retrieval pass would be useful evidence for that task; it would not establish universal long-context correctness.

Case 3: unified RAM and kernel shared memory are different limits

The GLM records concern LibertAIDAI/GLM-5.3-Flash-NVFP4, revision aa28e1f54130286c95fee10d0705c74ce8743734. FP8 attempts encountered an architecture allowlist rejection and a separate TensorRT-LLM unsupported-architecture error. A stock bf16 tilelang path requested 169,984 bytes of dynamic shared memory per block against a reported 101,376-byte device limit. This is not the machine’s 121 GiB unified-memory pool. Attempt sequence.

The source reports success with bf16 KV in sglang-glm53:gb10-tilelang, using block_I=32, num_stages=1, threads=128. The early run allocated 800,448 KV tokens and served 26,535-token prompts with eight users. The pinned community build patch identifies the modified v1 path; it is not an upstream release-fix claim.

Read the chronology. Although the early GLM document says DFlash remains untested, the later speculation document reports clean eight-token draft-verify capture for batch sizes one through eight on the tuned image. Keep that follow-up separate from the early no-speculation pool. Do not present the earlier warning as the final outcome.

Case 4: verify transport inside the running container

The source’s September 4 Qwen investigation found working host fabric and visible container HCA entries in sysfs, while /dev/infiniband was absent. NCCL used sockets. Mapping the required verbs devices, setting IPC_LOCK/memlock and selecting the connected HCA led to channel logs showing NET/IB. Container evidence and remediation.

A host link test cannot confirm the serving container’s transport. Inspect the running service after startup; device visibility alone is also insufficient. This note does not promote the earlier sweep’s speed ratios to a current benchmark, and the original author still identifies remaining unexplained latency.

A useful incident record

Before tuning, save the first relevant error, artifact revision, engine version, image digest when known, attention backend, KV dtype and allocated pool. Add prompt/output lengths, concurrency, chunk size, cache state and speculative mode. An omitted runtime identifier stays unknown even when the documentation is pinned.

Then write one falsifiable explanation and one change that tests it. Keep a short control and a representative longer correctness task. A server that starts proves startup; it does not prove long-context correctness. Preserve failed runs, successful follow-ups and unperformed proposals as separate records.

The adjacent JSON exposes four case_reports with explicit evidence_status, per-case conditions, source pointers, remediation state and unresolved fields. Unknown values remain null; the warning is never exported as a completed experiment. Use the two-Spark conditions guide and For AI reference alongside these cases.