DeepSeek 1017: Quantization, Memory and Context Evidence

DeepSeek 1017 has several usable community evidence trails, but they answer different questions. A dual-Spark EXL3 matrix measures short endpoint requests. Two near-million-token runs retrieve one string and return seven tokens. A VQ conversion reports file layout and a loader check while explicitly leaving 1017 serving speed unmeasured. Keep those states separate before choosing a quantization or budgeting memory.

The chart preserves three source states. Short decode rates are not near-1M generation rates, and VQ file bytes are not a runtime RAM requirement. No independent site model benchmark was performed.

Identify the EXL3 model and matching Engrams

The dual-Spark deployment uses LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw at afc7aa0aab98ee735c0f54e0a122019af1ac08f1. Its manifest separately pins official Engram tables to deepseek-ai/DeepSeek-V4.1-Flash-1017 at 1bae0e1088993b9b82de08f59de5be5a34e42170. A document commit, converted-weight commit and Engram commit identify different objects.

This deployment uses two 128 GB GB10 systems and TP2 over RoCE, with checkpoint reads through a read-only NFS export and rank-local Engram lookup. The TensorFold path is patched Python 0.6.0; the vLLM arm uses an EXL3 overlay. These are concrete integrations, not an instruction that an arbitrary current installation of either engine accepts the files unchanged.

What do the short-run rates actually measure?

The method uses deterministic ledger prompts: 520 input tokens with thinking disabled and 546 at reasoning effort 75. Selected C1 runs produce 64 output tokens. Each is one pass; no run-to-run variance estimate is available.

The table rounds the TensorFold CSV and vLLM CSV to one decimal; JSON retains the original numeric precision. Decode divides completion tokens after the first by the first-to-last streamed-delta interval. Initial prefill is excluded. Batch wall throughput is a separate quantity and must not be replaced by this decode rate.

An effort-75 response capped at 64 tokens can finish its budget inside thinking. A faster short interval therefore does not establish better answers or sustained long-generation capacity. The text matrix also predates the explicit native-image-routing flag. A later red-image smoke check in the deployment report is a serving-path check, not an image-understanding benchmark.

A shared pool is an admission setting

The TensorFold deployment description reports eight slots, a 524,288-token shared pool and a 262,144-token per-request ceiling. Its settings configure 5 GiB and 4 GiB memory floors, full prefill, and disabled session/prepared-cache writes.

These are this profile's controls, not peak RAM measurements. They do not demonstrate eight simultaneous maximum-length requests. They also do not govern the separate near-1M experiment below. Comparing its lower free-memory reading with these floors would incorrectly combine different builds, launch settings and memory policies.

Read the near-1M result as one specific retrieval task

The 18 October run index and two saved JSON records describe one exact-string needle at 63 percent depth in random filler. Both runs found their requested string, with zero cached prompt tokens and seven completion tokens.

The earlier record and final record retain exact counts and timestamps. These are different build/setup observations, not two unchanged-configuration repetitions. Runtime identifiers are reported prefixes; exact image digests and sampling values remain unknown. Zero reasoning tokens in the output is preserved without inventing an unlogged thinking flag.

The result supports a narrow conclusion: each named setup accepted that prompt and found that string. It does not establish broad reasoning over a million tokens, multiple-needle recall, long-answer stability, multimodal context or concurrent full-window requests. The seven-token completion matters as much as the large input count.

Keep the small memory margin attached to the result

During the final fill, minimum host MemAvailable was 1,779 MiB on the head and 2,542 MiB on the worker. These are observed minimum available-memory readings, not the model's allocated memory or a recommended spare-RAM budget.

The source describes swap restrictions, host reclaim/compaction settings and Engram page dropping. Its earlyoom threshold was 1,536 MiB during the reported run and was later changed to 1,024 MiB. This is a tuned, narrow-margin case. The known limits and run index also retain slower prefill states and later final-image runs below earlier results. The site does not turn those whole-host changes into general deployment defaults.

VQ conversion: file inspection before speed claims

The 1017 VQ artifact card reports a 113,556,639,424-byte base and a 971,066,016-byte vision sidecar. The base excludes the two Engram files. A 1,000-tensor loader binding check passed, but this snapshot explicitly has no 1017-specific serving speed, peak runtime RAM or model-quality result.

Its recipe pins the quantizer and vision converter separately. Its provenance explains why matching names and sizes are insufficient: 1017 Engram and vision contents differ from older V4.1 files. Preserve revision and the supplied SHA-256 identities, including when a loader accepts an older file without error.

The native 1,048,576-token metadata field is not a measured VQ serving result. Older V4.1 speed/fidelity examples in the card remain older-checkpoint examples. They cannot fill the missing 1017 values, and file bytes cannot account for all runtime state, buffers, cache residency or service overhead.

Use the evidence without extending it

The previous TensorFold explanation discusses other named configurations; it does not make their settings interchangeable with these cases. For AI access, deepseek-1017-long-context preserves both narrow retrieval cases and headroom, while deepseek-1017-quantization returns the short EXL3 report beside the VQ inspection. See For AI for executable examples.

Choose the evidence type needed for the next decision: verified file identity, endpoint behaviour, memory observation or task correctness. Keep actual input/output tokens, cache state, runtime and weight identities beside every result. Where the source supplies no execution result, the corresponding field stays null rather than becoming zero or an inherited benchmark.

VQ follow-up checked on October 18

The publisher card was rechecked at revision f81ee2e9801fcc58fd51ae75dd827467ffc0b9e5. It still explicitly leaves 1017-specific GB10 speed and quality unmeasured. The selected runtime guide and performance document do not supply a 1017 VQ run that fills those gaps. The Hub discussion listing was empty at this check. This is a review of those named sources, not proof that no report exists anywhere. File identity remains useful; old V4.1 speed, quality and residency figures remain outside the 1017 measurement set. No new VQ serving result is added.

Further VQ source check on October 18

A later bounded check retains the same 1017 VQ card revision. The runtime has advanced to 9feb2d79fd0f84066d5c06ace4f4423e49fbda99, but its model guide and performance document still contain no 1017-specific report. Hub discussions remain empty in this snapshot. No new 1017 VQ throughput, quality score or peak runtime RAM can be added from these checked sources; those fields remain null. Older V4.1 performance and the unrelated EXL3 results are not transferred. This describes the named-source review, not an exhaustive absence claim across the community.