# DeepSeek 1017: Quantization, Memory and Context Evidence DeepSeek 1017 has several usable community evidence trails, but they answer different questions. A dual-Spark EXL3 matrix measures short endpoint requests. Two near-million-token runs retrieve one string and return seven tokens. A VQ conversion reports file layout and a loader check while explicitly leaving 1017 serving speed unmeasured. Keep those states separate before choosing a quantization or budgeting memory. ![DeepSeek 1017 short-run EXL3 decode, separate near-million-token needle latency and a VQ artifact-size receipt with unknown runtime performance.](/assets/deepseek-1017-quant-memory-context-source.webp) *The chart preserves three source states. Short decode rates are not near-1M generation rates, and VQ file bytes are not a runtime RAM requirement. No independent site model benchmark was performed.* ## Identify the EXL3 model and matching Engrams The [dual-Spark deployment](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/README.md) uses `LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw` at `afc7aa0aab98ee735c0f54e0a122019af1ac08f1`. Its [manifest](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/config/manifest.json) separately pins official Engram tables to `deepseek-ai/DeepSeek-V4.1-Flash-1017` at `1bae0e1088993b9b82de08f59de5be5a34e42170`. A document commit, converted-weight commit and Engram commit identify different objects. This deployment uses two 128 GB GB10 systems and TP2 over RoCE, with checkpoint reads through a read-only NFS export and rank-local Engram lookup. The TensorFold path is patched Python 0.6.0; the vLLM arm uses an EXL3 overlay. These are concrete integrations, not an instruction that an arbitrary current installation of either engine accepts the files unchanged. ## What do the short-run rates actually measure? The [method](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/docs/benchmark-method.md) uses deterministic ledger prompts: 520 input tokens with thinking disabled and 546 at reasoning effort 75. Selected C1 runs produce 64 output tokens. Each is one pass; no run-to-run variance estimate is available. | C1 arm | Thinking off | Effort 75 | |---|---:|---:| | TensorFold Python 0.6.0 | 58.1 tokens/s | 77.5 tokens/s | | vLLM with EXL3 overlay | 36.9 tokens/s | 41.4 tokens/s | The table rounds the [TensorFold CSV](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/tensorfold-quick.csv) and [vLLM CSV](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/vllm-quick.csv) to one decimal; JSON retains the original numeric precision. Decode divides completion tokens after the first by the first-to-last streamed-delta interval. Initial prefill is excluded. Batch wall throughput is a separate quantity and must not be replaced by this decode rate. An effort-75 response capped at 64 tokens can finish its budget inside thinking. A faster short interval therefore does not establish better answers or sustained long-generation capacity. The text matrix also predates the explicit native-image-routing flag. A later red-image smoke check in the deployment report is a serving-path check, not an image-understanding benchmark. ## A shared pool is an admission setting The [TensorFold deployment description](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/docs/deployment.md) reports eight slots, a 524,288-token shared pool and a 262,144-token per-request ceiling. Its [settings](https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/config/tensorfold-knobs.env.example) configure 5 GiB and 4 GiB memory floors, full prefill, and disabled session/prepared-cache writes. These are this profile's controls, not peak RAM measurements. They do not demonstrate eight simultaneous maximum-length requests. They also do not govern the separate near-1M experiment below. Comparing its lower free-memory reading with these floors would incorrectly combine different builds, launch settings and memory policies. ## Read the near-1M result as one specific retrieval task The [18 October run index](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/INDEX.md) and two saved JSON records describe one exact-string needle at 63 percent depth in random filler. Both runs found their requested string, with zero cached prompt tokens and seven completion tokens. | Saved run | Actual input tokens | Output tokens | First token | |---|---:|---:|---:| | Earlier build, `fd2365042f56` | 1,034,753 | 7 | 830.4 s | | Final setup, `92aa710b533d` | 1,035,020 | 7 | 817.7 s | The [earlier record](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m.json) and [final record](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m-final.json) retain exact counts and timestamps. These are different build/setup observations, not two unchanged-configuration repetitions. Runtime identifiers are reported prefixes; exact image digests and sampling values remain unknown. Zero reasoning tokens in the output is preserved without inventing an unlogged thinking flag. The result supports a narrow conclusion: each named setup accepted that prompt and found that string. It does not establish broad reasoning over a million tokens, multiple-needle recall, long-answer stability, multimodal context or concurrent full-window requests. The seven-token completion matters as much as the large input count. ## Keep the small memory margin attached to the result During the final fill, minimum host `MemAvailable` was 1,779 MiB on the head and 2,542 MiB on the worker. These are observed minimum available-memory readings, not the model's allocated memory or a recommended spare-RAM budget. The source describes swap restrictions, host reclaim/compaction settings and Engram page dropping. Its earlyoom threshold was 1,536 MiB during the reported run and was later changed to 1,024 MiB. This is a tuned, narrow-margin case. The [known limits](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/docs/KNOWN-LIMITS.md) and run index also retain slower prefill states and later final-image runs below earlier results. The site does not turn those whole-host changes into general deployment defaults. ## VQ conversion: file inspection before speed claims The [1017 VQ artifact card](https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/README.md) reports a 113,556,639,424-byte base and a 971,066,016-byte vision sidecar. The base excludes the two Engram files. A 1,000-tensor loader binding check passed, but this snapshot explicitly has no 1017-specific serving speed, peak runtime RAM or model-quality result. Its [recipe](https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/RECIPE.md) pins the quantizer and vision converter separately. Its [provenance](https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/SOURCE.md) explains why matching names and sizes are insufficient: 1017 Engram and vision contents differ from older V4.1 files. Preserve revision and the supplied [SHA-256 identities](https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/SHA256SUMS), including when a loader accepts an older file without error. The native 1,048,576-token metadata field is not a measured VQ serving result. Older V4.1 speed/fidelity examples in the card remain older-checkpoint examples. They cannot fill the missing 1017 values, and file bytes cannot account for all runtime state, buffers, cache residency or service overhead. ## Use the evidence without extending it The [previous TensorFold explanation](/guides/deepseek-1017-tensorfold-community/) discusses other named configurations; it does not make their settings interchangeable with these cases. For AI access, `deepseek-1017-long-context` preserves both narrow retrieval cases and headroom, while `deepseek-1017-quantization` returns the short EXL3 report beside the VQ inspection. See [For AI](/for-ai/) for executable examples. Choose the evidence type needed for the next decision: verified file identity, endpoint behaviour, memory observation or task correctness. Keep actual input/output tokens, cache state, runtime and weight identities beside every result. Where the source supplies no execution result, the corresponding field stays null rather than becoming zero or an inherited benchmark. ## VQ follow-up checked on October 18 The publisher card was rechecked at revision `f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5`. It still explicitly leaves 1017-specific GB10 speed and quality unmeasured. The selected [runtime guide](https://github.com/Baekpica/ds4-dfm-rs/blob/c9425237669057937178e0598b3114ed9f8a15ec/docs/deepseek41-flash.md) and [performance document](https://github.com/Baekpica/ds4-dfm-rs/blob/c9425237669057937178e0598b3114ed9f8a15ec/docs/performance.md) do not supply a 1017 VQ run that fills those gaps. The Hub discussion listing was empty at this check. This is a review of those named sources, not proof that no report exists anywhere. File identity remains useful; old V4.1 speed, quality and residency figures remain outside the 1017 measurement set. No new VQ serving result is added. ## Further VQ source check on October 18 A later bounded check retains the same [1017 VQ card revision](https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/README.md). The runtime has advanced to `9feb2d79fd0f84066d5c06ace4f4423e49fbda99`, but its [model guide](https://github.com/Baekpica/ds4-dfm-rs/blob/9feb2d79fd0f84066d5c06ace4f4423e49fbda99/docs/deepseek41-flash.md) and [performance document](https://github.com/Baekpica/ds4-dfm-rs/blob/9feb2d79fd0f84066d5c06ace4f4423e49fbda99/docs/performance.md) still contain no 1017-specific report. Hub discussions remain empty in this snapshot. No new 1017 VQ throughput, quality score or peak runtime RAM can be added from these checked sources; those fields remain null. Older V4.1 performance and the unrelated EXL3 results are not transferred. This describes the named-source review, not an exhaustive absence claim across the community. ## Original sources and conditions - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/README.md - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: 839509cc2365ea030d94360454437e66165c1087182ce6ccb85d23d85d5bb66d - Elevatormusic: https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/INDEX.md - Document revision: 9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32 - SHA-256: e69d8c270e6407ddbaacb82da36cc48994b6f20fd4300d176753371b4e23fb8e - Baekpica: https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/README.md - Document revision: f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5 - SHA-256: b9f85bae000a8aa0e270555d17b1f9bd859a79bc3f5f2d7fde7eb00607cc990f - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/config/manifest.json - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: 6b3da5443112253502befe2a866983a31d188af234aef5e03865a724dbe9b65a - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/config/tensorfold-knobs.env.example - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: e26dc4761458be238d048c5a03b524993e7c632c27f8457527e3ba0030bc58c0 - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/config/vllm-knobs.env.example - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: ee4eb1535b45d28c47fdd5b136e4b04268a7955d2ac9ca83174afeb35c0d62d8 - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/docs/deployment.md - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: 72fb210773604c15560e73d6db8ada144b0e953354201e42b3b7aebd4c018799 - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/docs/benchmark-method.md - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: 33d83fe761ea2eaafaadee937fa212ffcec19d69a2528614776ad7fb1139ba3b - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/README.md - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: 380393cc1e86d175f21af1c216b613a2aad741ed90c851b7e8f491bed5231acf - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/tensorfold-quick.csv - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: 3005b3be00581a962f9a269211eef2d598a6358d83c41e565ee1549b28e03c33 - x00byte: https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/vllm-quick.csv - Document revision: e85ed7ec3a403d8820a83e5aa33abe0985b58896 - SHA-256: befbf211d4f143798e12053ec21291cc77e0893284ccdb8d5a0aa19ef4722c6e - Elevatormusic: https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m.json - Document revision: 9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32 - SHA-256: 9b81971c1378ab5b0bef466219c3a7aa7e2f78315bddbfc5bd0837cfec2fde40 - Elevatormusic: https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m-final.json - Document revision: 9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32 - SHA-256: c476006b019d46c68a334ad3763f3e8e0f0532d01dfc6fb800c043c225709ab1 - Elevatormusic: https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/docs/KNOWN-LIMITS.md - Document revision: 9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32 - SHA-256: eeb6bc6f7a3fce20654f977581e9718b124e70fc01ab358efebefe9afd9dc942 - Baekpica: https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/RECIPE.md - Document revision: f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5 - SHA-256: 1b66bfc44386bd6b708b1d4250cdd0dd825d2b45eee18c74d2bebfbff9e62486 - Baekpica: https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/SOURCE.md - Document revision: f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5 - SHA-256: d039fb41a620380e352de9794271a6b2855e5d6931979d43695f4d3a236d443e - Baekpica: https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/SHA256SUMS - Document revision: f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5 - SHA-256: 6b98a0a8923558f69ad53a9d60c29332b984a59f8109a6115c7c88e684763b19 Checked 2026-10-18. AI-assisted writing, source checked by Run AI Here. No independent model tests. Conditions and selected source facts are in the adjacent JSON export. ```json { "facts": [], "reported_results": [ { "label": "tensorfold C1 thinking off", "metric": "decode_rate", "value": 58.05409336065285, "unit": "tokens/s", "statistic": "one pass; no run-to-run uncertainty estimate", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_artifact": "deepseek-ai/DeepSeek-V4.1-Flash-1017", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "Engram_shard_SHA256": { "model-00047-of-00048.safetensors": "76a05106ece259be83d1bfea0dcb54f23377864ee60fd321690f35e6698a2af5", "model-00048-of-00048.safetensors": "9dc51433e195bc78b5ad846b68643cc0351d3b174e5e4309222d3b7402c83afd" }, "weight_precision": "EXL3 2.9bpw; 6-bit dense MTP handling in vLLM overlay", "system": "two DGX Spark GB10, 128 GB unified memory each, TP2, both available RoCE links", "node_count": 2, "parallelism": "TP2", "experiment_date": null, "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_depth_cap": 3, "concurrency": 1, "input_lines": 27, "output_tokens": 64, "max_output_tokens": 64, "repetitions": 1, "cache_state": "startup and earlier generation checks; no per-case warmup; TensorFold session/prepared caches disabled", "temperature": null, "top_p": null, "kv_dtype": null, "measurement_denominator": "(completion_tokens - 1) / first-to-last streamed delta interval; excludes initial prefill", "quality_evaluation": false, "long_response_steady_state_measured": false, "native_image_routing_in_text_matrix": false, "engine": "TensorFold Python 0.6.0 patched", "runtime_execution_revision": "c4646171139ee8a3c38103eaa1699dad226ec12b", "overlay_source_revision": "2dcce0303451da2a59f80e6c7413da8b3b0f7a57", "executed_image_digest": null, "thinking": false, "reasoning_effort": null, "input_tokens": 520, "actual_output_tokens": 64, "batch_wall_seconds": 2.4011132870218717, "end_to_end_output_wall_tps": 26.65430254620761, "TTFT_seconds": 1.313945789996069 }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsdual-results-tensorfold-quick.csv", "source_document_sha256": "3005b3be00581a962f9a269211eef2d598a6358d83c41e565ee1549b28e03c33", "source_line": 2, "evidence_url": "https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/tensorfold-quick.csv#L2", "source_record_id": "deepseek-1017-dual-spark-quick" }, { "label": "tensorfold C1 thinking 75", "metric": "decode_rate", "value": 77.49652509362433, "unit": "tokens/s", "statistic": "one pass; no run-to-run uncertainty estimate", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_artifact": "deepseek-ai/DeepSeek-V4.1-Flash-1017", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "Engram_shard_SHA256": { "model-00047-of-00048.safetensors": "76a05106ece259be83d1bfea0dcb54f23377864ee60fd321690f35e6698a2af5", "model-00048-of-00048.safetensors": "9dc51433e195bc78b5ad846b68643cc0351d3b174e5e4309222d3b7402c83afd" }, "weight_precision": "EXL3 2.9bpw; 6-bit dense MTP handling in vLLM overlay", "system": "two DGX Spark GB10, 128 GB unified memory each, TP2, both available RoCE links", "node_count": 2, "parallelism": "TP2", "experiment_date": null, "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_depth_cap": 3, "concurrency": 1, "input_lines": 27, "output_tokens": 64, "max_output_tokens": 64, "repetitions": 1, "cache_state": "startup and earlier generation checks; no per-case warmup; TensorFold session/prepared caches disabled", "temperature": null, "top_p": null, "kv_dtype": null, "measurement_denominator": "(completion_tokens - 1) / first-to-last streamed delta interval; excludes initial prefill", "quality_evaluation": false, "long_response_steady_state_measured": false, "native_image_routing_in_text_matrix": false, "engine": "TensorFold Python 0.6.0 patched", "runtime_execution_revision": "c4646171139ee8a3c38103eaa1699dad226ec12b", "overlay_source_revision": "2dcce0303451da2a59f80e6c7413da8b3b0f7a57", "executed_image_digest": null, "thinking": true, "reasoning_effort": 75, "input_tokens": 546, "actual_output_tokens": 64, "batch_wall_seconds": 2.141774678020738, "end_to_end_output_wall_tps": 29.881761446140466, "TTFT_seconds": 1.326543439005036 }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsdual-results-tensorfold-quick.csv", "source_document_sha256": "3005b3be00581a962f9a269211eef2d598a6358d83c41e565ee1549b28e03c33", "source_line": 10, "evidence_url": "https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/tensorfold-quick.csv#L10", "source_record_id": "deepseek-1017-dual-spark-quick" }, { "label": "vllm C1 thinking off", "metric": "decode_rate", "value": 36.8753802051545, "unit": "tokens/s", "statistic": "one pass; no run-to-run uncertainty estimate", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_artifact": "deepseek-ai/DeepSeek-V4.1-Flash-1017", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "Engram_shard_SHA256": { "model-00047-of-00048.safetensors": "76a05106ece259be83d1bfea0dcb54f23377864ee60fd321690f35e6698a2af5", "model-00048-of-00048.safetensors": "9dc51433e195bc78b5ad846b68643cc0351d3b174e5e4309222d3b7402c83afd" }, "weight_precision": "EXL3 2.9bpw; 6-bit dense MTP handling in vLLM overlay", "system": "two DGX Spark GB10, 128 GB unified memory each, TP2, both available RoCE links", "node_count": 2, "parallelism": "TP2", "experiment_date": null, "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_depth_cap": 3, "concurrency": 1, "input_lines": 27, "output_tokens": 64, "max_output_tokens": 64, "repetitions": 1, "cache_state": "startup and earlier generation checks; no per-case warmup; TensorFold session/prepared caches disabled", "temperature": null, "top_p": null, "kv_dtype": null, "measurement_denominator": "(completion_tokens - 1) / first-to-last streamed delta interval; excludes initial prefill", "quality_evaluation": false, "long_response_steady_state_measured": false, "native_image_routing_in_text_matrix": false, "engine": "vLLM + EXL3 overlay", "runtime_execution_revision": null, "overlay_source_revision": "6f7d1590ad49a2b8995188e45d7b9db31e677452", "executed_image_digest": "sha256:2f0cf3adc0f989c1d446be274df864eb799630175f604c3b22b71b7205971dce", "thinking": false, "reasoning_effort": null, "input_tokens": 520, "actual_output_tokens": 64, "batch_wall_seconds": 2.5652991679962724, "end_to_end_output_wall_tps": 24.948357212460998, "TTFT_seconds": 0.8555283379973844 }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsdual-results-vllm-quick.csv", "source_document_sha256": "befbf211d4f143798e12053ec21291cc77e0893284ccdb8d5a0aa19ef4722c6e", "source_line": 2, "evidence_url": "https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/vllm-quick.csv#L2", "source_record_id": "deepseek-1017-dual-spark-quick" }, { "label": "vllm C1 thinking 75", "metric": "decode_rate", "value": 41.447290817386964, "unit": "tokens/s", "statistic": "one pass; no run-to-run uncertainty estimate", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_artifact": "deepseek-ai/DeepSeek-V4.1-Flash-1017", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "Engram_shard_SHA256": { "model-00047-of-00048.safetensors": "76a05106ece259be83d1bfea0dcb54f23377864ee60fd321690f35e6698a2af5", "model-00048-of-00048.safetensors": "9dc51433e195bc78b5ad846b68643cc0351d3b174e5e4309222d3b7402c83afd" }, "weight_precision": "EXL3 2.9bpw; 6-bit dense MTP handling in vLLM overlay", "system": "two DGX Spark GB10, 128 GB unified memory each, TP2, both available RoCE links", "node_count": 2, "parallelism": "TP2", "experiment_date": null, "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_depth_cap": 3, "concurrency": 1, "input_lines": 27, "output_tokens": 64, "max_output_tokens": 64, "repetitions": 1, "cache_state": "startup and earlier generation checks; no per-case warmup; TensorFold session/prepared caches disabled", "temperature": null, "top_p": null, "kv_dtype": null, "measurement_denominator": "(completion_tokens - 1) / first-to-last streamed delta interval; excludes initial prefill", "quality_evaluation": false, "long_response_steady_state_measured": false, "native_image_routing_in_text_matrix": false, "engine": "vLLM + EXL3 overlay", "runtime_execution_revision": null, "overlay_source_revision": "6f7d1590ad49a2b8995188e45d7b9db31e677452", "executed_image_digest": "sha256:2f0cf3adc0f989c1d446be274df864eb799630175f604c3b22b71b7205971dce", "thinking": true, "reasoning_effort": 75, "input_tokens": 546, "actual_output_tokens": 64, "batch_wall_seconds": 2.4199117650277913, "end_to_end_output_wall_tps": 26.447245277666145, "TTFT_seconds": 0.8985815939959139 }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsdual-results-vllm-quick.csv", "source_document_sha256": "befbf211d4f143798e12053ec21291cc77e0893284ccdb8d5a0aa19ef4722c6e", "source_line": 10, "evidence_url": "https://github.com/x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark/blob/e85ed7ec3a403d8820a83e5aa33abe0985b58896/results/vllm-quick.csv#L10", "source_record_id": "deepseek-1017-dual-spark-quick" }, { "label": "fd2365042f56 near-1M first token", "metric": "ttft", "value": 830.4, "unit": "s", "statistic": "single prompt, no variance estimate", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "weight_precision": "EXL3 2.9bpw", "system": "two DGX Spark GB10, TP2", "node_count": 2, "engine": "TensorFold Python 0.6.6", "runtime_execution_revision": null, "executed_image_digest": null, "context_limit_tokens": 1048576, "concurrency": 1, "needle_depth_fraction": 0.63, "workload": "one exact-string needle in random filler lines", "temperature": null, "top_p": null, "thinking": null, "repetitions": 1, "general_quality_evaluation": false, "multi_request_long_context_validated": false, "runtime_revision_prefix": "fd2365042f56", "input_tokens": 1034753, "output_tokens": 7, "cached_tokens": 0, "cache_state": "zero prefix-cache hit tokens; not a cold OS page-cache guarantee", "experiment_date": "2026-10-18T01:51:54Z", "needle_found": true, "reasoning_tokens": 0, "total_seconds": 835.8 }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsneedle-runs-2026-10-18-needle-1m.json", "source_document_sha256": "9b81971c1378ab5b0bef466219c3a7aa7e2f78315bddbfc5bd0837cfec2fde40", "source_line": 8, "evidence_url": "https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m.json#L8", "source_record_id": "deepseek-1017-million-token-needle" }, { "label": "92aa710b533d near-1M first token", "metric": "ttft", "value": 817.7, "unit": "s", "statistic": "single prompt, no variance estimate", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "weight_precision": "EXL3 2.9bpw", "system": "two DGX Spark GB10, TP2", "node_count": 2, "engine": "TensorFold Python 0.6.6", "runtime_execution_revision": null, "executed_image_digest": null, "context_limit_tokens": 1048576, "concurrency": 1, "needle_depth_fraction": 0.63, "workload": "one exact-string needle in random filler lines", "temperature": null, "top_p": null, "thinking": null, "repetitions": 1, "general_quality_evaluation": false, "multi_request_long_context_validated": false, "runtime_revision_prefix": "92aa710b533d", "input_tokens": 1035020, "output_tokens": 7, "cached_tokens": 0, "cache_state": "zero prefix-cache hit tokens; not a cold OS page-cache guarantee", "experiment_date": "2026-10-18T04:47:07Z", "needle_found": true, "reasoning_tokens": 0, "total_seconds": 825.9 }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsneedle-runs-2026-10-18-needle-1m-final.json", "source_document_sha256": "c476006b019d46c68a334ad3763f3e8e0f0532d01dfc6fb800c043c225709ab1", "source_line": 8, "evidence_url": "https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m-final.json#L8", "source_record_id": "deepseek-1017-million-token-needle" }, { "label": "near-1M final head minimum MemAvailable", "metric": "minimum_memavailable", "value": 1779, "unit": "MiB", "statistic": "minimum observed in this one fill, not a safe capacity guarantee", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "weight_precision": "EXL3 2.9bpw", "system": "two DGX Spark GB10, TP2", "node_count": 2, "engine": "TensorFold Python 0.6.6", "runtime_execution_revision": null, "executed_image_digest": null, "context_limit_tokens": 1048576, "concurrency": 1, "needle_depth_fraction": 0.63, "workload": "one exact-string needle in random filler lines", "temperature": null, "top_p": null, "thinking": null, "repetitions": 1, "general_quality_evaluation": false, "multi_request_long_context_validated": false, "runtime_revision_prefix": "92aa710b533d", "input_tokens": 1035020, "output_tokens": 7, "cache_state": "same final needle; cached_tokens 0", "experiment_date": "2026-10-18", "memory_kind": "minimum_host_MemAvailable_during_prefill", "node": "head", "earlyoom_limit_MiB_at_measurement": 1536, "earlyoom_later_limit_MiB": 1024, "OS_memory_controls_changed": true, "ordinary_host_baseline_inferred": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsneedle-runs-2026-10-18-INDEX.md", "source_document_sha256": "e69d8c270e6407ddbaacb82da36cc48994b6f20fd4300d176753371b4e23fb8e", "source_line": 59, "evidence_url": "https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/INDEX.md#L59", "source_record_id": "deepseek-1017-million-token-needle" }, { "label": "near-1M final worker minimum MemAvailable", "metric": "minimum_memavailable", "value": 2542, "unit": "MiB", "statistic": "minimum observed in this one fill, not a safe capacity guarantee", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "model_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "weight_precision": "EXL3 2.9bpw", "system": "two DGX Spark GB10, TP2", "node_count": 2, "engine": "TensorFold Python 0.6.6", "runtime_execution_revision": null, "executed_image_digest": null, "context_limit_tokens": 1048576, "concurrency": 1, "needle_depth_fraction": 0.63, "workload": "one exact-string needle in random filler lines", "temperature": null, "top_p": null, "thinking": null, "repetitions": 1, "general_quality_evaluation": false, "multi_request_long_context_validated": false, "runtime_revision_prefix": "92aa710b533d", "input_tokens": 1035020, "output_tokens": 7, "cache_state": "same final needle; cached_tokens 0", "experiment_date": "2026-10-18", "memory_kind": "minimum_host_MemAvailable_during_prefill", "node": "worker", "earlyoom_limit_MiB_at_measurement": 1536, "earlyoom_later_limit_MiB": 1024, "OS_memory_controls_changed": true, "ordinary_host_baseline_inferred": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "dsneedle-runs-2026-10-18-INDEX.md", "source_document_sha256": "e69d8c270e6407ddbaacb82da36cc48994b6f20fd4300d176753371b4e23fb8e", "source_line": 59, "evidence_url": "https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/INDEX.md#L59", "source_record_id": "deepseek-1017-million-token-needle" }, { "label": "1017 VQ base GGUF", "metric": "artifact_size", "value": 113556639424, "unit": "bytes", "statistic": "source-reported file bytes; not a RAM requirement", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF", "model_revision": "f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5", "base_model_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "quantizer_revision": "7fb26cc332d740416792204a29c7d13d2986caf5", "vision_converter_revision": "e4ea07889144d85acd33b2490ec4f44032644fb0", "quantization": "YoungAi vq8sh14-q4k-mtpnative; routed-expert VQ, q4_K skeleton; native MTP towers", "measurement_kind": "publisher artifact layout inspection, not runtime evaluation", "experiment_date": "2026-10-17 status in card", "system": null, "engine": "target ds4-dfm-rs; loader check passed, serving unmeasured on 1017", "runtime_execution_revision": null, "input_tokens": null, "output_tokens": null, "concurrency": null, "context_limit_tokens": 1048576, "context_limit_origin": "model metadata, not measured serving result", "speed_1017_measured": false, "quality_1017_measured": false, "peak_runtime_memory_measured": false, "base_model_speed_inherited": false, "filename": "DeepSeek-V4.1-Flash-vq8sh14-q4k-mtpnative.gguf", "memory_kind": "artifact_disk_file_bytes", "Engram_included": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "ds1017vq-readme", "source_document_sha256": "b9f85bae000a8aa0e270555d17b1f9bd859a79bc3f5f2d7fde7eb00607cc990f", "source_line": 97, "evidence_url": "https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/README.md#L97", "source_record_id": "deepseek-1017-vq-artifact-inspection" }, { "label": "1017 BF16 vision sidecar", "metric": "artifact_size", "value": 971066016, "unit": "bytes", "statistic": "source-reported file bytes; not a RAM requirement", "uncertainty": null, "conditions": { "model": "DeepSeek-V4.1-Flash-1017", "model_artifact": "Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF", "model_revision": "f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5", "base_model_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "quantizer_revision": "7fb26cc332d740416792204a29c7d13d2986caf5", "vision_converter_revision": "e4ea07889144d85acd33b2490ec4f44032644fb0", "quantization": "YoungAi vq8sh14-q4k-mtpnative; routed-expert VQ, q4_K skeleton; native MTP towers", "measurement_kind": "publisher artifact layout inspection, not runtime evaluation", "experiment_date": "2026-10-17 status in card", "system": null, "engine": "target ds4-dfm-rs; loader check passed, serving unmeasured on 1017", "runtime_execution_revision": null, "input_tokens": null, "output_tokens": null, "concurrency": null, "context_limit_tokens": 1048576, "context_limit_origin": "model metadata, not measured serving result", "speed_1017_measured": false, "quality_1017_measured": false, "peak_runtime_memory_measured": false, "base_model_speed_inherited": false, "filename": "DeepSeek-V4.1-Flash.vision.gguf", "memory_kind": "artifact_disk_file_bytes", "Engram_included": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "ds1017vq-readme", "source_document_sha256": "b9f85bae000a8aa0e270555d17b1f9bd859a79bc3f5f2d7fde7eb00607cc990f", "source_line": 98, "evidence_url": "https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF/blob/f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5/README.md#L98", "source_record_id": "deepseek-1017-vq-artifact-inspection" } ], "conditions": { "one_needle_is_general_long_context_quality": false, "EXL3_and_VQ_results_interchangeable": false, "VQ_1017_speed_measured": false, "VQ_1017_quality_measured": false, "independent_site_runs": 0 }, "deployment_profiles": [ { "id": "dsdual-python-profile", "model_artifact": "LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw", "checkpoint_revision": "afc7aa0aab98ee735c0f54e0a122019af1ac08f1", "Engram_revision": "1bae0e1088993b9b82de08f59de5be5a34e42170", "engine": "TensorFold Python 0.6.0 patched", "runtime_revision": "c4646171139ee8a3c38103eaa1699dad226ec12b", "context_limit_tokens": 262144, "configured_shared_pool_tokens": 524288, "configured_request_slots": 8, "configured_MemAvailable_floor_GiB": 5, "configured_hard_floor_GiB": 4, "memory_floors_are_measured_peaks": false, "session_cache": false, "prepared_cache_writes": false, "source_current_recipe_has_native_image_routing": true, "text_matrix_predates_image_flag": true } ], "memory_observations": [ { "record_id": "deepseek-1017-million-token-needle", "kind": "minimum_host_MemAvailable_during_final_fill", "head_MiB": 1779, "worker_MiB": 2542, "host_tuning_required_in_source": true, "safe_capacity_guarantee": false }, { "record_id": "deepseek-1017-vq-artifact-inspection", "kind": "file_inventory_not_RAM", "base_bytes": 113556639424, "vision_bytes": 971066016, "Engram_shard_bytes": [ 101535150936, 101537926640 ], "actual_peak_RAM": null, "speed_and_quality_1017": null } ], "long_context_cases": [ { "id": "dsneedle-runs-2026-10-18-needle-1m.json", "classification": "source_reported_single_needle_success", "configured_limit_tokens": 1048576, "actual_input_tokens": 1034753, "actual_output_tokens": 7, "needle_depth_fraction": 0.63, "exact_match": true, "TTFT_seconds": 830.4, "cached_tokens": 0, "runtime_revision_prefix": "fd2365042f56", "executed_image_digest": null, "sampling": null, "quality_scope": "one needle at one depth; not broad reasoning, multi-needle, long-response or concurrency validation", "source_document_id": "dsneedle-runs-2026-10-18-needle-1m.json", "source_document_sha256": "9b81971c1378ab5b0bef466219c3a7aa7e2f78315bddbfc5bd0837cfec2fde40", "source_line": 3, "evidence_url": "https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m.json#L3" }, { "id": "dsneedle-runs-2026-10-18-needle-1m-final.json", "classification": "source_reported_single_needle_success", "configured_limit_tokens": 1048576, "actual_input_tokens": 1035020, "actual_output_tokens": 7, "needle_depth_fraction": 0.63, "exact_match": true, "TTFT_seconds": 817.7, "cached_tokens": 0, "runtime_revision_prefix": "92aa710b533d", "executed_image_digest": null, "sampling": null, "quality_scope": "one needle at one depth; not broad reasoning, multi-needle, long-response or concurrency validation", "source_document_id": "dsneedle-runs-2026-10-18-needle-1m-final.json", "source_document_sha256": "c476006b019d46c68a334ad3763f3e8e0f0532d01dfc6fb800c043c225709ab1", "source_line": 3, "evidence_url": "https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/9c6ab02340b770c6c3e1a9a4d5e70a56c5829d32/runs/2026-10-18/needle-1m-final.json#L3" } ] } ```