SOURCES / ATTRIBUTION
Fresh repository metadata is separate from an actual deployment or benchmark reproduction.
The 17 October checkpoint: official weights, protocol details and minimal local inference. Start here to identify 1017 before reading a deployment recipe.
Publisher model card with serving links and distinct thinking-mode settings. Use its exact artifact identity alongside the selected engine recipe.
Official Flash checkpoint and local-serving references. Its reasoning effort and chat-template controls matter when attempting evaluation reproduction.
The full GLM-5.3 model card, with serving routes and evaluation notes. Keep it separate from Flash and from quantized community conversions.
Publisher checkpoint and deployment references for Kimi K3, including its preserved-thinking conversation requirements.
Original M3 weights and inference guidance, linking several local engines. This entry identifies the upstream model rather than a Spark-specific conversion.
Official Step 3.7 Flash card with serving examples and high-memory workstation options, including distinct engine paths.
The RL checkpoint’s original card, with engine examples, parallelism and parser requirements. Keep this artifact separate from the MOPD variant.
The current recipe selects the 1017 checkpoint by default. Check precision, engine version and hardware-specific flags before serving.
Model-specific serving recipe with NVFP4, FP8 and BF16 paths, hardware overrides and optional acceleration.
Dense Qwen3.8 deployment reference with separate quantization artifacts and workstation or accelerator profiles.
Large MoE recipe with explicit multi-GPU strategies and distinct native versus extended context claims.
Serving recipe for Flash with FP8, NVFP4 and BF16 choices and model-specific engine settings.
Full GLM-5.3 recipe, including quantization alternatives and accelerator-specific serving controls.
Kimi K3 deployment recipe with separate MXFP4, NVFP4 and other artifact paths and parallelism strategies.
M3 serving recipe covering full and converted weights, hardware overrides and optional acceleration.
Step 3.7 Flash engine recipe with separate BF16, FP8 and NVFP4 artifacts.
MiMo serving reference whose default artifact is MOPD and whose RL variant preserves the separate RL checkpoint.
Nemotron Lightning recipe with distinct NVFP4 and BF16 choices and engine constraints.
Recent multimodal Nemotron recipe with per-artifact hardware restrictions and selected GB200 layouts.
A smaller MoE deployment recipe with publisher and third-party FP8/NVFP4 alternatives.
Compact Qwen3.5 recipe with explicit Intel CPU/GPU artifact allowlists in the pinned version.
MXFP4 serving recipe with accelerator allowlists and model-specific serving controls.
Gemma 4 serving recipe with BF16 and multiple converted precision paths.
Large DeepSeek Pro recipe with a dated default checkpoint and separate draft/quantized variants.
Image-generation recipe with vLLM-Omni requirements and separate precision choices.
Start with the upstream build and server documentation for a GGUF deployment. Choose the backend for your own platform and keep the model artifact separate from the runtime revision.
A maintainer starting point for native installation and local model serving. Use its linked documentation to check platform support, installed model identity and local/cloud behavior.
Follow the serving project and its documentation when planning a compatible model-serving environment. Backend, architecture, package and model support are version-dependent.
An upstream reference for model-serving deployments. Begin with the project documentation, then verify the exact backend, model and container combination you intend to use.
An original operator repository describing a three-node TP=3 deployment and its benchmark checkpoint. It also retains a four-node baseline, so the two sections must be read as different conditions.
An operator-maintained K3s/Ansible repository for a four-Spark cluster. It contains deployment notes and reported transport/inference comparisons. Select the exact model profile and experiment before reusing a number.
Paired serving-optimization report on eight RTX 5090 D cards. Interactive, wide-batch and held-out text sections have different workloads.
A clearly identified TP4 baseline using disk-backed Engram and DSpark. Read the boot-10 section separately from later EXL3 and SGLang lanes.
Versioned decode campaign comparing one- and two-RTX coordinators with four Spark expert workers, using interleaved sessions.
Two-node serving study that distinguishes end-to-end served rate from decode rate, and audits cache, warmup and checkpoint identity.
A dated 80-scenario quality cohort with two repeats, model-card sampling and per-deployment reports.
A fixed March M4 Pro report with repeated generation, first-token and memory measurements across available precisions.
Original Spark report separating prompt processing from single-batch generation at different context depths.
Maintainer benchmark document with M4 Max hardware and explicit package versions, separating capability and efficiency columns.
A traceable 1017 EXL3 deployment and two named decode configurations, with a separate later prefill cohort and explicit correctness limits.
A pinned single-Spark NVFP4/DFlash2 recipe with separate speed cohorts, task-level BF16 comparisons and negative non-inferiority flags.
The v2.4.0 promotion arm serves the pinned GDN-MSE checkpoint, with an extra MXFP8 prefill shard. NVFP4 expert weights, MXFP8 paths and a refitted four-draft MTP head are separate components. The experimental image tag is retained; the later release-image digest does not prove the exact cold-test image digest. Repository release names are specific to this one-Spark recipe.
This experiment changes the n-gram PLE table backing, rather than quantizing every model component. The source compares a BF16 table held in RAM with file-mapped sidecars on one 96 GB RTX PRO 6000 and a 176 GB host. Main-table throughput uses 8K input, 512 requested output tokens and no prefix-cache hits. Full checkpoint revision and executed image digest are not supplied.
The four selected C1 decode observations come from the raw quick-matrix CSVs, with 520-token thinking-off or 546-token effort-75 prompts and 64 actual output tokens. Each is one pass without variance estimates. Their first-to-last streamed-delta rate excludes initial prefill; end-to-end wall throughput remains a different field. This is not a long-context or sustained-response experiment.
Two saved runs found one exact-string needle at 63 percent depth in random filler, with 1034753 and 1035020 prompt tokens. Each completion is seven tokens, with zero cached prompt tokens. These are two different build/setup observations, not repeated measurements of one unchanged configuration or evidence of general long-document reasoning.
The converter reports a 1000-tensor base, its exact assembled file size and a successful loader binding check. The separate vision sidecar and official 1017 Engram files retain individual identities. These are artifact and loader observations. The selected source explicitly has no 1017-specific serving speed, peak RAM or model-quality measurement.
One 128 GB unified-memory DGX Spark serves senfu NVFP4 at its full recorded revision through SGLang commit 4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b. The recipe keeps PLE on NVMe through mmap, uses an FP8 dense path and an NVFP4 LM head, and disables MTP. The repository documentation pin is separate from the engine and weight revisions.
The k76 v2.0.0 concurrency series uses the earlier 16c9bd54788d12390838a65ce4a4ecda97fa5f1d GDN-MSE checkpoint, while v2.2.0 cold-prefix measurements use 03f4a0570496bbe741189f4a2a7d611679e85406. Both employ two directly connected Sparks, vLLM plus b12x and mixed NVFP4/MXFP8 paths. Release image digests and abbreviated engine commits do not establish an exact executed image digest.
The pinned MXFP4 GGUF runs through ik_llama.cpp merge d854dfd36a7262fc46cb7542716b04a9f6d4b47d on one RTX 5090 with WSL2, a 128 GiB-class host and 96 GiB VM ceiling. The baseline puts 38 expert layers on CPU, keeps mmap, disables the CUDA pinned-host override and uses Q8_0 KV. One request and one slot are used throughout; this is not full-model GPU residence.
Read 128K and 490K Qwen batches, a four-GPU 128K matrix, sampled peak VRAM and a Mamba crash with original versions and measurement limits.
A pinned NVFP4 target, DFlash2 draft and community image reach sustained decode and 120124-token retrieval, while strict output failures remain visible.
Separate a July lane-A reproduction from proposed speed/context recipes, retaining request timing, startup memory brackets and a negative NCCL result.
Audit a publisher-labelled K3 layer-split deployment, approximate memory and small GSM8K/HumanEval subsets, retaining first-pass failures and timeout retries.
Read two 250K-request memory controls, a 500K single-request peak, a production OOM and invalidated/repeated recovery tests without mixing hardware or cohorts.
Audit the 753B non-Flash EXL3 TP4 recipe, replicated and shared Q5 cache, million-token retrieval, BF16-reference fidelity and later RDMA fixes.
Separate a full historical MMLU run and small task subsets from the corrected EXL3 router stack, whose old task scores were withdrawn and await reruns.
Separate a full historical MMLU run and small task subsets from the corrected EXL3 router stack, whose old task scores were withdrawn and await reruns.
Read a13/a14 cold extraction logs, the queued eighth-request fix, full model/base-engine pins, a GSM8K subset and older-build needle checks.
Optional Google Analytics helps us understand which sources and tools are useful. It stays off unless you allow it. You can change your choice at any time. Read our privacy notice.
