COMMUNITY COLLECTION / PERFORMANCE REPORT
Speed, quality and transport reports retain their own workloads, controls and caveats.
An original operator repository describing a three-node TP=3 deployment and its benchmark checkpoint. It also retains a four-node baseline, so the two sections must be read as different conditions.
An operator-maintained K3s/Ansible repository for a four-Spark cluster. It contains deployment notes and reported transport/inference comparisons. Select the exact model profile and experiment before reusing a number.
Paired serving-optimization report on eight RTX 5090 D cards. Interactive, wide-batch and held-out text sections have different workloads.
A clearly identified TP4 baseline using disk-backed Engram and DSpark. Read the boot-10 section separately from later EXL3 and SGLang lanes.
Versioned decode campaign comparing one- and two-RTX coordinators with four Spark expert workers, using interleaved sessions.
Two-node serving study that distinguishes end-to-end served rate from decode rate, and audits cache, warmup and checkpoint identity.
A dated 80-scenario quality cohort with two repeats, model-card sampling and per-deployment reports.
A fixed March M4 Pro report with repeated generation, first-token and memory measurements across available precisions.
Original Spark report separating prompt processing from single-batch generation at different context depths.
Maintainer benchmark document with M4 Max hardware and explicit package versions, separating capability and efficiency columns.
A traceable 1017 EXL3 deployment and two named decode configurations, with a separate later prefill cohort and explicit correctness limits.
A pinned single-Spark NVFP4/DFlash2 recipe with separate speed cohorts, task-level BF16 comparisons and negative non-inferiority flags.
The v2.4.0 promotion arm serves the pinned GDN-MSE checkpoint, with an extra MXFP8 prefill shard. NVFP4 expert weights, MXFP8 paths and a refitted four-draft MTP head are separate components. The experimental image tag is retained; the later release-image digest does not prove the exact cold-test image digest. Repository release names are specific to this one-Spark recipe.
This experiment changes the n-gram PLE table backing, rather than quantizing every model component. The source compares a BF16 table held in RAM with file-mapped sidecars on one 96 GB RTX PRO 6000 and a 176 GB host. Main-table throughput uses 8K input, 512 requested output tokens and no prefix-cache hits. Full checkpoint revision and executed image digest are not supplied.
The four selected C1 decode observations come from the raw quick-matrix CSVs, with 520-token thinking-off or 546-token effort-75 prompts and 64 actual output tokens. Each is one pass without variance estimates. Their first-to-last streamed-delta rate excludes initial prefill; end-to-end wall throughput remains a different field. This is not a long-context or sustained-response experiment.
Two saved runs found one exact-string needle at 63 percent depth in random filler, with 1034753 and 1035020 prompt tokens. Each completion is seven tokens, with zero cached prompt tokens. These are two different build/setup observations, not repeated measurements of one unchanged configuration or evidence of general long-document reasoning.
One 128 GB unified-memory DGX Spark serves senfu NVFP4 at its full recorded revision through SGLang commit 4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b. The recipe keeps PLE on NVMe through mmap, uses an FP8 dense path and an NVFP4 LM head, and disables MTP. The repository documentation pin is separate from the engine and weight revisions.
The k76 v2.0.0 concurrency series uses the earlier 16c9bd54788d12390838a65ce4a4ecda97fa5f1d GDN-MSE checkpoint, while v2.2.0 cold-prefix measurements use 03f4a0570496bbe741189f4a2a7d611679e85406. Both employ two directly connected Sparks, vLLM plus b12x and mixed NVFP4/MXFP8 paths. Release image digests and abbreviated engine commits do not establish an exact executed image digest.
The pinned MXFP4 GGUF runs through ik_llama.cpp merge d854dfd36a7262fc46cb7542716b04a9f6d4b47d on one RTX 5090 with WSL2, a 128 GiB-class host and 96 GiB VM ceiling. The baseline puts 38 expert layers on CPU, keeps mmap, disables the CUDA pinned-host override and uses Q8_0 KV. One request and one slot are used throughout; this is not full-model GPU residence.
Read 128K and 490K Qwen batches, a four-GPU 128K matrix, sampled peak VRAM and a Mamba crash with original versions and measurement limits.
A pinned NVFP4 target, DFlash2 draft and community image reach sustained decode and 120124-token retrieval, while strict output failures remain visible.
Separate a July lane-A reproduction from proposed speed/context recipes, retaining request timing, startup memory brackets and a negative NCCL result.
Audit a publisher-labelled K3 layer-split deployment, approximate memory and small GSM8K/HumanEval subsets, retaining first-pass failures and timeout retries.
Read two 250K-request memory controls, a 500K single-request peak, a production OOM and invalidated/repeated recovery tests without mixing hardware or cohorts.
Audit the 753B non-Flash EXL3 TP4 recipe, replicated and shared Q5 cache, million-token retrieval, BF16-reference fidelity and later RDMA fixes.
Separate a full historical MMLU run and small task subsets from the corrected EXL3 router stack, whose old task scores were withdrawn and await reruns.
Separate a full historical MMLU run and small task subsets from the corrected EXL3 router stack, whose old task scores were withdrawn and await reruns.
Read a13/a14 cold extraction logs, the queued eighth-request fix, full model/base-engine pins, a GSM8K subset and older-build needle checks.
Optional Google Analytics helps us understand which sources and tools are useful. It stays off unless you allow it. You can change your choice at any time. Read our privacy notice.
