# Qwen Flash-Next: Quantization, Memory and Long Context A smaller quantized file, a lower RSS reading and a longer configured window describe different capabilities. For Qwen3.8-Flash-Next, the useful evidence here comes from two separate deployments: a one-Spark NVFP4/MXFP8 recipe with measured cold-prefix latency, and an RTX PRO 6000 experiment that changes how the n-gram PLE table is stored. Read their hardware, allocations and request shapes before estimating whether a setup fits. ![One-Spark cold-prefix first-token latency at 16K, 64K and 128K, beside a separate PLE host-RSS comparison on RTX PRO 6000.](/assets/qwen-flash-next-memory-context-source.webp) *The left panel shows original means and between-boot SD. The right panel retains source-labelled GB and distinguishes reclaimable mapped pages. The panels describe different machines; neither is whole-model peak RAM.* ## Start with the complete single-Spark artifact The [one-Spark recipe](https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/README.md) serves `ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE` at `03f4a0570496bbe741189f4a2a7d611679e85406`. It also reads an MXFP8 prefill shard from `local-inference-lab/Qwen3.8-Flash-Next-NVFP4` at `7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd`. Record both identities. “NVFP4” alone omits the dense, attention and prefill paths, as well as the refitted four-draft MTP head. The selected experiment is the v2.4.0 promotion arm on one 128 GB unified-memory DGX Spark, using vLLM and b12x. Its saved [experimental recipe](https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/arm-p1/recipe.yaml) fixes the checkpoint and image tag. The [build report](https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/BENCHMARKS.md) describes the subsequent release image with an added compiled-cache layer. That release digest is useful provenance, but it is not silently substituted for an unreported digest of the exact cold-test image. Engine commit prefixes likewise remain prefixes when the complete executed revision is unavailable. ## Read the actual 16K–128K cold-prefix workload The [source capability CSV](https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv) selects C1, temperature zero and one requested output token. Each value averages four boots with three cold prompts per boot. The uncertainty is SD between boot means, not a confidence interval or the spread of every individual request. | Actual input tokens | First-token latency, mean ± SD | Prefill, mean ± SD | |---:|---:|---:| | 16,384 | 7.104 ± 0.016 s | 2,306.4 ± 5.2 tokens/s | | 65,536 | 30.723 ± 0.060 s | 2,133.2 ± 4.2 tokens/s | | 131,072 | 70.000 ± 0.137 s | 1,872.5 ± 3.7 tokens/s | These observations describe the cost of reading a fresh prefix. They do not measure a long answer after that prefix. “Cold prefix” excludes a matching prompt-cache hit; it does not say that weights, filesystem pages or compiled kernels were cold. The selected fields do not supply every request's actual completion count or all sampling parameters, so those gaps stay visible in JSON. The author reports a separate retrieval gate through 128K and explicitly leaves larger one-Spark depths unmeasured in this snapshot. Keep a retrieval gate, this latency curve and the configured window as separate evidence. None proves general answer quality over the whole configured window. ## Keep the KV pool separate from the request ceiling The experimental recipe sets FP8 KV and an explicit 15,032,385,536-byte pool, or 14 GiB. The build report records an allocated capacity of 993,754 tokens. The per-request ceiling is 262,144 tokens and the scheduler accepts up to eight request slots. Those three values are not a claim that eight full windows fit at once. Admission also depends on shared pool use, state and request scheduling. A token-capacity figure is not measured peak process memory. The recipe's `gpu_memory_utilization=0.01` is a startup guard in conjunction with explicit KV bytes, as its own comment explains; it does not mean the model consumes one percent of unified memory. The PLE table is paged from NVMe, introducing another distinct working set. ## Compare PLE memory on its own hardware The [PLE sidecar report](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md) uses one 96 GB RTX PRO 6000 Blackwell, 176 GB host RAM and local NVMe. Its main table uses the mixed NVFP4/FP8 checkpoint, 8K input, 512 requested output tokens and no prefix-cache hits. The reported launch window is 32,768 with 36 request slots. These conditions are not the Spark experiment's conditions. Two selected memory rows make the distinction concrete. The BF16 PLE table is labelled 95.4 GB, with approximately 95 GB host RSS. The INT4 group-16 sidecar is labelled 32 GB, with 32.9 GB host RSS. Its [format metadata](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/ples_int4/META.json) specifies group-16 INT4 with FP16 scales. Mapped RSS contains resident file pages that the operating system can reclaim. It is not a permanent anonymous allocation, and table size is not whole-model memory use. The source's GB labels are preserved; the site does not infer their exact byte convention or silently equate them with GiB. The full checkpoint revision, executed image digest and timing of the RSS sample remain unreported. ## Separate first-request residence from warm throughput An optional experiment drops page cache before each arm and compares INT4 mmap with an anonymous resident copy. The source reports 19.0 tokens/s on the first mmap request and 74.3 on the first resident request without MTP. These are first-request observations from a different cohort, whose exact request shape is not given; the main table's 8K/512 shape is not copied into it. The resident sidecar adds 29.8 GiB of anonymous memory above roughly 8 GiB already held by the worker. This is another allocation, not the INT4 mmap RSS restated in different units. Anonymous memory also has different reclaim and swap behaviour; it should not be described as universally unreclaimable. Within this optional cohort, both residence modes retain the same reported KV allocation for each MTP setting: 426,639 tokens without MTP and 134,192 with three drafts. The observation does not isolate a universal memory formula. The source also reports much closer warm rates and limited quality/sanity checks, so neither a cold-request advantage nor a smaller sidecar establishes general quality equivalence. ## What should accompany a deployment decision? Keep the exact main checkpoint, extra prefill weights, PLE format, drafter and executed runtime together. Name the memory measurement: file bytes, mapped RSS, anonymous allocation, KV capacity or minimum free memory. Record actual input/output length, concurrency, cache state and whether variation is within one boot or between boots. For automated reading, [the two complete source records](/for-ai/) are available through `qwen-flash-next-memory`; the adjacent article JSON retains the same conditions. Missing values remain null. Use the [runtime choice guide](/guides/inference-runtime-choice/) and [KV/speculation explanation](/guides/kv-cache-speculative-decoding/) to frame a follow-up test. Run AI Here has not executed either deployment or independently measured these models. ## Original sources and conditions - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/README.md - Document revision: 348ede7e28a01f6aa6b501aaf2392d20a286a012 - SHA-256: bb618a879867f416c51cfc3fa8b699da427d28ec324b14af7d1279a2501ab37f - primitive-ai: https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md - Document revision: a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763 - SHA-256: 0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9 - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv - Document revision: 348ede7e28a01f6aa6b501aaf2392d20a286a012 - SHA-256: 4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205 - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/BENCHMARKS.md - Document revision: 348ede7e28a01f6aa6b501aaf2392d20a286a012 - SHA-256: ac2128626b607b2e4571bdb45fb7e2639e5c65c83841a8f28367bb5692986fc0 - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/VERSIONS.md - Document revision: 348ede7e28a01f6aa6b501aaf2392d20a286a012 - SHA-256: c513523883d95b716707697a0633fb9c5e63c9cf748aeaf37a38d4f771913abd - ursuciprian: https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/arm-p1/recipe.yaml - Document revision: 348ede7e28a01f6aa6b501aaf2392d20a286a012 - SHA-256: 33ecbeea74e85987bd4dfa7418543438e98ff93ea2270c103deccab44a43fa7e - primitive-ai: https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/ples_int4/META.json - Document revision: a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763 - SHA-256: e9a1863008450b7ac329d50a3b6b908f21122b8bf5da999bdeff05004161884a - primitive-ai: https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/ples_fp8/META.json - Document revision: a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763 - SHA-256: de757efa51030ac1e4edb6acdb29507521f8ace860a3be511661a26e5017b632 - primitive-ai: https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/ples_nvfp4/META.json - Document revision: a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763 - SHA-256: 5aa4d53703f67c8c664d674c95f16c29c8ce7a94a2d7d9ac27ae8afe4c0b56ef Checked 2026-10-18. AI-assisted writing, source checked by Run AI Here. No independent model tests. Conditions and selected source facts are in the adjacent JSON export. ```json { "facts": [], "reported_results": [ { "label": "1x v2.4.0 16384-token ttft", "metric": "ttft", "value": 7.104, "unit": "s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "kind": "SD between boot means, not confidence interval", "standard_deviation": 0.016 }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": 1, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": 0, "thinking": null, "output_tokens": 1, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": 16384, "repetitions": "4 boots x 3 cold prompts; SD between boots", "harness": "cold.py (k101)", "source_run_directory": "1x:results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/", "source_metric": "ttft_s", "workload": "long-prompt" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-data-capability.csv", "source_document_sha256": "4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205", "source_line": 1262, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv#L1262", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "1x v2.4.0 16384-token prefill_rate", "metric": "prefill_rate", "value": 2306.4, "unit": "tokens/s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "kind": "SD between boot means, not confidence interval", "standard_deviation": 5.2 }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": 1, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": 0, "thinking": null, "output_tokens": 1, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": 16384, "repetitions": "4 boots x 3 cold prompts; SD between boots", "harness": "cold.py (k101)", "source_run_directory": "1x:results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/", "source_metric": "pp", "workload": "long-prompt" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-data-capability.csv", "source_document_sha256": "4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205", "source_line": 1263, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv#L1263", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "1x v2.4.0 65536-token ttft", "metric": "ttft", "value": 30.723, "unit": "s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "kind": "SD between boot means, not confidence interval", "standard_deviation": 0.06 }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": 1, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": 0, "thinking": null, "output_tokens": 1, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": 65536, "repetitions": "4 boots x 3 cold prompts; SD between boots", "harness": "cold.py (k101)", "source_run_directory": "1x:results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/", "source_metric": "ttft_s", "workload": "long-prompt" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-data-capability.csv", "source_document_sha256": "4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205", "source_line": 1264, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv#L1264", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "1x v2.4.0 65536-token prefill_rate", "metric": "prefill_rate", "value": 2133.2, "unit": "tokens/s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "kind": "SD between boot means, not confidence interval", "standard_deviation": 4.2 }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": 1, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": 0, "thinking": null, "output_tokens": 1, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": 65536, "repetitions": "4 boots x 3 cold prompts; SD between boots", "harness": "cold.py (k101)", "source_run_directory": "1x:results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/", "source_metric": "pp", "workload": "long-prompt" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-data-capability.csv", "source_document_sha256": "4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205", "source_line": 1265, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv#L1265", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "1x v2.4.0 131072-token ttft", "metric": "ttft", "value": 70.0, "unit": "s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "kind": "SD between boot means, not confidence interval", "standard_deviation": 0.137 }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": 1, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": 0, "thinking": null, "output_tokens": 1, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": 131072, "repetitions": "4 boots x 3 cold prompts; SD between boots", "harness": "cold.py (k101)", "source_run_directory": "1x:results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/", "source_metric": "ttft_s", "workload": "long-prompt" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-data-capability.csv", "source_document_sha256": "4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205", "source_line": 1266, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv#L1266", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "1x v2.4.0 131072-token prefill_rate", "metric": "prefill_rate", "value": 1872.5, "unit": "tokens/s", "statistic": "mean of 4 boots x 3 cold prompts, sd between boots", "uncertainty": { "kind": "SD between boot means, not confidence interval", "standard_deviation": 3.7 }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": 1, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": 0, "thinking": null, "output_tokens": 1, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": 131072, "repetitions": "4 boots x 3 cold prompts; SD between boots", "harness": "cold.py (k101)", "source_run_directory": "1x:results/k103-1x-prefill-combo-ab-20261016-1206/cb-01/", "source_metric": "pp", "workload": "long-prompt" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-data-capability.csv", "source_document_sha256": "4376de2ea68ca23d5208731ce948a28801cd1da2dc026c9415b2b122e9339205", "source_line": 1267, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/data/capability.csv#L1267", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "1x v2.4.0 allocated KV pool", "metric": "kv_pool_tokens", "value": 993754, "unit": "tokens", "statistic": "boot allocation observed on every reported boot", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "model_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_artifact": "local-inference-lab/Qwen3.8-Flash-Next-NVFP4", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "weight_precision": "NVFP4 experts; MXFP8 dense/attention; GDN NVFP4 decode and MXFP8 prefill", "system": "one DGX Spark GB10 with 128 GB unified memory, local NVMe", "node_count": 1, "parallelism": "TP1", "engine": "vLLM + b12x", "engine_version": null, "runtime_execution_revision": null, "runtime_revision_prefix": "10d0acc8 files over 5dad364d", "b12x_revision_prefix": "a1ad1b69", "executed_image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "release_image_is_exact_cold_test_image": false, "recipe_release": "1x v2.4.0 promotion A/B; k103 cb test arm, not the release-image ship check", "experiment_date": "2026-10-16", "context_limit_tokens": 262144, "configured_max_requests": 8, "draft_tokens": 4, "kv_dtype": "fp8", "GPU_memory_utilization_startup_guard": 0.01, "configured_KV_pool_bytes": 15032385536, "PLE_backing": "NVMe mmap, page cache; reported 26.8 GiB PLE table", "concurrency": null, "cache_state": "cold prefix, no cache hit; not a claimed cold OS page cache", "temperature": null, "thinking": null, "output_tokens": null, "actual_output_tokens_per_request": null, "GPU_clock_cap_MHz": null, "input_tokens": null, "memory_kind": "allocated_KV_token_pool_not_measured_peak_RAM", "max_context_simultaneously_validated": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen1x-docs-BENCHMARKS.md", "source_document_sha256": "ac2128626b607b2e4571bdb45fb7e2639e5c65c83841a8f28367bb5692986fc0", "source_line": 106, "evidence_url": "https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark/blob/348ede7e28a01f6aa6b501aaf2392d20a286a012/docs/BENCHMARKS.md#L106", "source_record_id": "qwen38-flash-next-single-spark-context" }, { "label": "BF16 in RAM table size", "metric": "artifact_size", "value": 95.4, "unit": "GB", "statistic": "source-reported PLE table size, not whole model or peak RAM", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": 32768, "configured_max_requests": 36, "input_tokens": 8192, "output_tokens": 512, "actual_output_tokens": null, "concurrency": 1, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": true, "temperature": null, "top_p": null, "draft_tokens": null, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "BF16 in RAM", "memory_kind": "PLE_artifact_size", "memory_backing": "in RAM" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 96, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L96", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "BF16 in RAM host RSS", "metric": "host_rss", "value": 95, "unit": "GB", "statistic": "approximate source host RSS", "uncertainty": { "approximate": true }, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": 32768, "configured_max_requests": 36, "input_tokens": 8192, "output_tokens": 512, "actual_output_tokens": null, "concurrency": 1, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": true, "temperature": null, "top_p": null, "draft_tokens": null, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "BF16 in RAM", "memory_kind": "host_RSS_not_necessarily_anonymous", "memory_backing": "in RAM" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 96, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L96", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "INT4 group-16 mmap table size", "metric": "artifact_size", "value": 32, "unit": "GB", "statistic": "source-reported PLE table size, not whole model or peak RAM", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": 32768, "configured_max_requests": 36, "input_tokens": 8192, "output_tokens": 512, "actual_output_tokens": null, "concurrency": 1, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": true, "temperature": null, "top_p": null, "draft_tokens": null, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16 mmap", "memory_kind": "PLE_artifact_size", "memory_backing": "reclaimable mapped file pages" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 98, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L98", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "INT4 group-16 mmap host RSS", "metric": "host_rss", "value": 32.9, "unit": "GB", "statistic": "source host RSS including reclaimable file pages", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": 32768, "configured_max_requests": 36, "input_tokens": 8192, "output_tokens": 512, "actual_output_tokens": null, "concurrency": 1, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": true, "temperature": null, "top_p": null, "draft_tokens": null, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16 mmap", "memory_kind": "host_RSS_not_necessarily_anonymous", "memory_backing": "reclaimable mapped file pages" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 98, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L98", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "mmap cold first request", "metric": "decode_rate", "value": 19.0, "unit": "tokens/s", "statistic": "first-request observation; warm asymptote is a separate quantity", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": null, "configured_max_requests": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "concurrency": 1, "cache_state": "page cache dropped before each arm", "thinking": null, "temperature": null, "top_p": null, "draft_tokens": 0, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": null, "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16", "workload": "first request, exact request shape unreported; optional resident-vs-mmap cohort, not main 8K/512 table", "memory_backing": "NVMe mmap" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 68, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L68", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "resident cold first request", "metric": "decode_rate", "value": 74.3, "unit": "tokens/s", "statistic": "first-request observation; warm asymptote is a separate quantity", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": null, "configured_max_requests": null, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "concurrency": 1, "cache_state": "page cache dropped before each arm", "thinking": null, "temperature": null, "top_p": null, "draft_tokens": 0, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": null, "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16", "workload": "first request, exact request shape unreported; optional resident-vs-mmap cohort, not main 8K/512 table", "memory_backing": "anonymous resident copy" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 69, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L69", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "INT4 resident sidecar anonymous allocation", "metric": "anonymous_memory", "value": 29.8, "unit": "GiB", "statistic": "sidecar allocation only, on top of roughly 8 GiB held by worker", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": null, "configured_max_requests": 36, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "concurrency": null, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": null, "temperature": null, "top_p": null, "draft_tokens": null, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16", "memory_kind": "anonymous_resident_sidecar_not_total_RSS" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 81, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L81", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "optional residence cohort KV pool, MTP 0", "metric": "kv_pool_tokens", "value": 426639, "unit": "tokens", "statistic": "reported identical mmap/resident allocation for this MTP setting", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": null, "configured_max_requests": 36, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "concurrency": null, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": null, "temperature": null, "top_p": null, "draft_tokens": 0, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16", "memory_kind": "allocated_KV_token_pool_not_measured_RAM" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 80, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L80", "source_record_id": "qwen38-flash-next-ple-memory" }, { "label": "optional residence cohort KV pool, MTP 3", "metric": "kv_pool_tokens", "value": 134192, "unit": "tokens", "statistic": "reported identical mmap/resident allocation for this MTP setting", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8", "model_revision": null, "PLE_artifact": "primitive-ai/Qwen3.8-Flash-Next-PLE-quant", "PLE_artifact_revision": "a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763", "system": "one RTX PRO 6000 Blackwell 96 GB, 176 GB host RAM, local NVMe", "node_count": 1, "engine": "vLLM with exact-image PLE overlay", "engine_version": null, "runtime_execution_revision": null, "image_tag": "vllm/vllm-openai:qwen38-flash-next", "executed_image_digest": null, "experiment_date": null, "source_update_boundary": "2026-09-06 corrections present; individual table trial dates not stated", "model_weight_precision": "mixed NVFP4/FP8 checkpoint; PLE format varies", "context_limit_tokens": null, "configured_max_requests": 36, "input_tokens": null, "output_tokens": null, "actual_output_tokens": null, "concurrency": null, "cache_state": "prefix-cache-free; mmap page-cache residency not independently logged", "thinking": null, "temperature": null, "top_p": null, "draft_tokens": 3, "MTP_main_table_execution": null, "kv_dtype": null, "throughput_repetitions": "two seeds, a/b; not independent boots", "memory_sampling_instant": null, "reported_size_unit": "GB as labelled by source, byte convention not independently specified", "PLE_format": "INT4 group-16", "memory_kind": "allocated_KV_token_pool_not_measured_RAM" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwenple-readme", "source_document_sha256": "0085c022b212b241827a09f1a315170737f76298ca5af3bd8a46dea2032faab9", "source_line": 80, "evidence_url": "https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant/blob/a0fa93f2b9ed5fcab0cf01b60e88ff1ce13a4763/README.md#L80", "source_record_id": "qwen38-flash-next-ple-memory" } ], "conditions": { "cross_hardware_memory_profiles_merged": false, "native_window_is_measured_full_window": false, "one_Spark_cold_prefill_validated_through_tokens": 131072, "general_quality_equivalence_established": false, "independent_site_runs": 0 }, "deployment_profiles": [ { "id": "qwen-1x-v2.4-cold", "model_artifact": "ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE", "checkpoint_revision": "03f4a0570496bbe741189f4a2a7d611679e85406", "extra_prefill_revision": "7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd", "engine": "vLLM + b12x", "image_tag": "ghcr.io/ursuciprian/spark-vllm-b12x:k103-20261015-a1ad1b69-10d0acc8", "executed_image_digest": null, "release_image_digest": "sha256:30fe678ed7ae5d4deb277be137497bcf597e46e829471b7bed24edd4294a84f1", "context_limit_tokens": 262144, "measured_cold_input_tokens": [ 16384, 65536, 131072 ], "configured_max_requests": 8, "KV_pool_bytes": 15032385536, "KV_pool_tokens": 993754, "MTP_draft_tokens": 4, "PLE_file_reported_GiB": 26.8, "all_eight_full_window_requests_proven": false, "cold_prefix_is_not_cold_page_cache": true } ], "memory_observations": [ { "record_id": "qwen38-flash-next-ple-memory", "kind": "source_main_table_host_RSS", "BF16_in_RAM_GB_approx": 95, "INT4_mmap_GB": 32.9, "mapped_pages_reclaimable": true, "anonymous_memory_inferred_from_RSS": false, "GB_byte_convention_verified": false }, { "record_id": "qwen38-flash-next-ple-memory", "kind": "optional_anonymous_residence", "INT4_sidecar_GiB": 29.8, "additional_worker_GiB_approx": 8, "same_cohort_as_main_table": false } ] } ```