{ "id": "A14", "path": "/guides/qwen-flash-next-5090-context-memory/", "title": "Qwen on RTX 5090: What the 128K Recipe Tested", "summary": "Understand a 104805-token Qwen retrieval, 32K-only memory telemetry and rejected MTP promotion on an RTX 5090 with WSL2 and partial CPU expert offload.", "answer": "The recipe completes a 104805-token needle under a configured 131072 window at C1. Published baseline memory peaks belong to a shorter 32K gate. MTP speed promotion failed, ngram speculation was not run, and the embedding table is a separate component.", "published_on": "2026-10-18", "modified_on": "2026-10-18", "source_checked_on": "2026-10-18", "sources": [ { "publisher": "kutaelee", "url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe", "repository": "kutaelee/qwen38-flash-next-5090-128k-runtime-recipe", "platform": "github", "revision": "fca75dd108ce04b6a79db3b45934ed58bba893bb", "document_path": "README.md", "pinned_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md", "report_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md", "document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "last_commit_at": "2026-09-18T12:12:22Z", "checked_on": "2026-10-18", "metadata_status": "checked", "record_id": "qwen38-flash-next-5090-offload-gates" } ], "supplemental_sources": [ { "document_id": "qwen5090-method", "parent_record_id": "qwen38-flash-next-5090-offload-gates", "publisher": "kutaelee", "repository": "kutaelee/qwen38-flash-next-5090-128k-runtime-recipe", "revision": "fca75dd108ce04b6a79db3b45934ed58bba893bb", "document_path": "docs/methodology.md", "pinned_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/docs/methodology.md", "document_sha256": "54d61a37de6873bcebe34c2ad34978a9b0a7191232fb865c364540cdd3e4c706", "checked_on": "2026-10-18" }, { "document_id": "qwen5090-limits", "parent_record_id": "qwen38-flash-next-5090-offload-gates", "publisher": "kutaelee", "repository": "kutaelee/qwen38-flash-next-5090-128k-runtime-recipe", "revision": "fca75dd108ce04b6a79db3b45934ed58bba893bb", "document_path": "docs/limitations.md", "pinned_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/docs/limitations.md", "document_sha256": "029f3cfc9a4b8e2e03eba92137e4df3ca722782b13af4dd990295a8fe1c270d2", "checked_on": "2026-10-18" }, { "document_id": "qwen5090-config", "parent_record_id": "qwen38-flash-next-5090-offload-gates", "publisher": "kutaelee", "repository": "kutaelee/qwen38-flash-next-5090-128k-runtime-recipe", "revision": "fca75dd108ce04b6a79db3b45934ed58bba893bb", "document_path": "configs/ik-llama-baseline-128k.example.sh", "pinned_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/configs/ik-llama-baseline-128k.example.sh", "document_sha256": "fc5de996c4e03b80de59f21597ae543a37ff829822641bf2117426b9c6f09f28", "checked_on": "2026-10-18" } ], "facts": [], "reported_results": [ { "label": "32K needle gate prefill_rate", "metric": "prefill_rate", "value": 271.64, "unit": "tokens/s", "statistic": "one reported completed gate; no dispersion", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "workload": "32K needle gate", "memory_cohort": "completed 32K monitoring" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 53, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L53", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "32K needle gate decode_rate", "metric": "decode_rate", "value": 16.53, "unit": "tokens/s", "statistic": "one reported completed gate; no dispersion", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "workload": "32K needle gate", "memory_cohort": "completed 32K monitoring" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 53, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L53", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "128K server smoke prefill_rate", "metric": "prefill_rate", "value": 17.02, "unit": "tokens/s", "statistic": "one reported completed gate; no dispersion", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 61, "context_limit_tokens": 131072, "cache_state": null, "workload": "128K server smoke", "memory_cohort": "no paired 128K telemetry published" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 54, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L54", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "128K server smoke decode_rate", "metric": "decode_rate", "value": 15.26, "unit": "tokens/s", "statistic": "one reported completed gate; no dispersion", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 61, "context_limit_tokens": 131072, "cache_state": null, "workload": "128K server smoke", "memory_cohort": "no paired 128K telemetry published" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 54, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L54", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "128K long-context needle prefill_rate", "metric": "prefill_rate", "value": 245.72, "unit": "tokens/s", "statistic": "one reported completed gate; no dispersion", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 104805, "context_limit_tokens": 131072, "cache_state": null, "workload": "128K long-context needle", "memory_cohort": "no paired 128K telemetry published" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 55, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L55", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "128K long-context needle decode_rate", "metric": "decode_rate", "value": 13.94, "unit": "tokens/s", "statistic": "one reported completed gate; no dispersion", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 104805, "context_limit_tokens": 131072, "cache_state": null, "workload": "128K long-context needle", "memory_cohort": "no paired 128K telemetry published" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 55, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L55", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "32K telemetry startup", "metric": "startup", "value": 25.15, "unit": "s", "statistic": "source monitored peak/minimum; no distribution", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "memory_cohort": "completed 32K gate only; not paired with the 104805-token gate", "memory_kind": "startup_duration", "MemAvailable_sampling_scope": null, "VM_ceiling_and_host_MemAvailable_same_scope_verified": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 57, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L57", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "32K telemetry host_rss", "metric": "host_rss", "value": 53901956, "unit": "KiB", "statistic": "source monitored peak/minimum; no distribution", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "memory_cohort": "completed 32K gate only; not paired with the 104805-token gate", "memory_kind": "peak_host_RSS", "MemAvailable_sampling_scope": null, "VM_ceiling_and_host_MemAvailable_same_scope_verified": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 57, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L57", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "32K telemetry minimum_memavailable", "metric": "minimum_memavailable", "value": 86701016, "unit": "KiB", "statistic": "source monitored peak/minimum; no distribution", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "memory_cohort": "completed 32K gate only; not paired with the 104805-token gate", "memory_kind": "minimum_MemAvailable_in_reported_monitor_scope", "MemAvailable_sampling_scope": null, "VM_ceiling_and_host_MemAvailable_same_scope_verified": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 57, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L57", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "32K telemetry swap_use", "metric": "swap_use", "value": 63708, "unit": "KiB", "statistic": "source monitored peak/minimum; no distribution", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "memory_cohort": "completed 32K gate only; not paired with the 104805-token gate", "memory_kind": "maximum_swap_use", "MemAvailable_sampling_scope": null, "VM_ceiling_and_host_MemAvailable_same_scope_verified": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 57, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L57", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "32K telemetry GPU_memory", "metric": "GPU_memory", "value": 25813, "unit": "MiB", "statistic": "source monitored peak/minimum; no distribution", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 0, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": 26253, "context_limit_tokens": 32768, "cache_state": null, "memory_cohort": "completed 32K gate only; not paired with the 104805-token gate", "memory_kind": "peak_GPU_memory", "MemAvailable_sampling_scope": null, "VM_ceiling_and_host_MemAvailable_same_scope_verified": false }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 57, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L57", "source_record_id": "qwen38-flash-next-5090-offload-gates" }, { "label": "canonical MTP n1 peak GPU memory", "metric": "GPU_memory", "value": 31775, "unit": "MiB", "statistic": "maximum in completed MTP short gates", "uncertainty": null, "conditions": { "model": "Qwen3.8-Flash-Next", "model_artifact": "jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF", "model_revision": "4a768bf4458a83e1c80667b4d818ec721080546e", "artifact_SHA256": "5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2", "engine": "ik_llama.cpp", "runtime_execution_revision": "d854dfd36a7262fc46cb7542716b04a9f6d4b47d", "MTP_patch_head": "108dfaa77fdaa138c46b043bfff0e94fbe2adff7", "system": "single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling", "node_count": 1, "concurrency": 1, "configured_max_requests": 1, "weight_precision": "MXFP4 GGUF, Q8 n-gram table", "CPU_expert_layers": 38, "defer_experts": true, "mmap": true, "GGML_CUDA_NO_PINNED": true, "kv_dtype": "Q8_0", "draft_tokens": 1, "ngram_speculation": false, "ngram_embedding_table_present": true, "temperature": null, "top_p": null, "thinking": null, "output_tokens": null, "actual_output_tokens": null, "repetitions": null, "experiment_date": null, "throughput_origin": "server timing fields, not a client estimate", "input_tokens": null, "context_limit_tokens": 131072, "cache_state": null, "memory_cohort": "canonical MTP n1 short gates; not baseline or long-context peak", "memory_kind": "peak_GPU_memory" }, "evidence_state": "source_reported", "site_reproduced": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 61, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L61", "source_record_id": "qwen38-flash-next-5090-offload-gates" } ], "conditions": { "concurrency": 1, "hardware_sample_size": 1, "full_131072_input_tested": false, "all_gate_memory_peaks_measured": false, "MTP_promoted": false, "ngram_state": "NOT_RUN_BY_GATE", "independent_site_runs": 0 }, "memory_observations": [ { "kind": "unpaired_memory_telemetry", "measured_context_limit_tokens": 32768, "measured_input_tokens": 26253, "longer_needle_input_tokens": 104805, "longer_needle_peak_RAM": null, "host_MemAvailable_scope": null, "CPU_expert_layers": 38, "source_GGUF_bytes": 126846153856 } ], "generation": { "text_draft": "local Spark Qwen; receipt in editorial-batch6 report; source-checked and rewritten by Codex", "review": "Run AI Here with Codex", "image": "deterministic matplotlib figures and evidence maps; no generated measured data", "local_first": true }, "image": { "url": "/assets/qwen-flash-next-5090-context-memory-source.webp", "alt": "The configured 131072-token window, completed 104805-token retrieval and 32K-only peak-memory telemetry are separate claims.", "width": 1600, "height": 900, "sha256": "aa8d2332386b7195f9c991d36179b6f62ad32c9166d30d0a46e9d94992499ccb", "caption": "The original recipe preserves one GPU, one request, mmap loading and partial CPU expert offload; missing long-context telemetry stays unknown." }, "long_context_cases": [ { "classification": "source_reported_exact_needle_success", "actual_input_tokens": 26253, "configured_limit_tokens": 32768, "actual_output_tokens": null, "exact_match": true, "needle_depth_fraction": null, "general_reasoning_quality_proven": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 53, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L53" }, { "classification": "source_reported_short_smoke", "actual_input_tokens": 61, "configured_limit_tokens": 131072, "actual_output_tokens": null, "exact_match": true, "needle_depth_fraction": null, "general_reasoning_quality_proven": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 54, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L54" }, { "classification": "source_reported_exact_needle_success", "actual_input_tokens": 104805, "configured_limit_tokens": 131072, "actual_output_tokens": null, "exact_match": true, "needle_depth_fraction": null, "general_reasoning_quality_proven": false, "source_document_id": "qwen5090-readme", "source_document_sha256": "0668bd3d2e2ca6ed48b4ef13e9d9c56e4ce5c37c45ebbf8dc54e359bd9575de0", "source_line": 55, "evidence_url": "https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md#L55" } ], "schema_version": "1.0.0", "record_type": "source_backed_explainer", "evidence_state": "source_reported", "site_reproduced": false, "metrics": null, "performance_ranking_allowed": false, "body_markdown": "# Qwen on RTX 5090: What the 128K Recipe Tested\n\nA Qwen3.8-Flash-Next server can advertise a 131,072-token window while its longest published request contains 104,805 tokens. That is useful long-context evidence with a precise boundary. The original RTX 5090 WSL2 recipe also publishes memory telemetry from a shorter 32K gate, rejects MTP promotion and leaves ngram speculation unrun. Preserving these separate outcomes makes the report more useful than a single “fits on 32 GB” headline.\n\n![The configured 131072-token window, completed 104805-token retrieval and 32K-only peak-memory telemetry are separate claims.](/assets/qwen-flash-next-5090-context-memory-source.webp)\n\n## Start with the CPU offload and the exact artifact\n\nThe [original kutaelee repository](https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md) describes one RTX 5090 reporting 32,607 MiB, a 16-core/32-thread desktop CPU, a 128 GiB-class host and Ubuntu under WSL2 with a 96 GiB VM ceiling. The selected model is `jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF` at `4a768bf4458a83e1c80667b4d818ec721080546e`. Its file is 126,846,153,856 bytes, with a publisher-provided SHA-256 retained in the JSON conditions. Run AI Here inspected the public documents; it did not download or execute this weight file.\n\nThe tested engine is `ik_llama.cpp` merge commit `d854dfd36a7262fc46cb7542716b04a9f6d4b47d`, including the pinned MTP patch. A repository documentation revision and a runtime execution revision identify different things, so both are retained. Later changes at a repository's moving main branch are not automatically part of this test.\n\nMost importantly, the [baseline configuration](https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/configs/ik-llama-baseline-128k.example.sh) keeps 38 expert layers on the CPU, defers experts, enables mmap and uses Q8_0 KV. It allows one request and one server slot. This is partial CPU expert offload, not a claim that the whole model resides in the GPU's memory. A Q8 n-gram embedding table is part of the model artifact; ngram speculative decoding is a different feature. Keeping speculation off does not mean the model's embedding table is absent or unused.\n\n## Read the actual request lengths\n\nThe three [reported baseline gates](https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/README.md) use different prompts. Throughput comes from server timing fields, according to the [methodology](https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/docs/methodology.md). It is not a client-side elapsed-time estimate.\n\n| Completed gate | Configured window | Actual prompt tokens | Prompt processing, tokens/s | Decode, tokens/s |\n|---|---:|---:|---:|---:|\n| 32K exact needle | 32,768 | 26,253 | 271.64 | 16.53 |\n| 128K server smoke | 131,072 | 61 | 17.02 | 15.26 |\n| Longer exact needle | 131,072 | 104,805 | 245.72 | 13.94 |\n\n![Three completed 5090 gates use 26253, 61 and 104805 prompt tokens, with distinct prompt-processing rates.](/assets/qwen-5090-actual-input-gates.webp)\n\nThe 61-token request confirms that a server configured for 128K completed a short smoke test. It does not test a 128K input. The longer gate is more informative about retrieval at depth, while still falling short of a completely filled 131,072-token input. Actual completion lengths, needle depth, per-gate sampling and repeat counts are not supplied in the selected table. Do not silently insert the runtime's current defaults into those empty fields.\n\nThese three rows are not a controlled context-scaling experiment. Request content, startup state and task differ, and no uncertainty distribution accompanies these gate values. It would be misleading to fit a latency law or predict a maximum memory requirement from the table. A request passing also does not establish that several such requests can run together: every published row uses C1.\n\n## Keep the 32K memory telemetry in its own cohort\n\nThe source reports 25.15 seconds of startup and the following observations during the completed 32K monitoring gate:\n\n| Quantity | Original reported value | What it describes |\n|---|---:|---|\n| Peak host RSS | 53,901,956 KiB | Resident process memory in the reported monitor scope |\n| Minimum MemAvailable | 86,701,016 KiB | An available-memory sample whose precise boundary is unspecified |\n| Maximum swap use | 63,708 KiB | Reported swap use during that gate |\n| Peak GPU memory | 25,813 MiB | GPU allocation observation for that gate |\n\n![The 32K telemetry retains peak RSS, minimum MemAvailable, swap and peak GPU memory in original KiB or MiB.](/assets/qwen-5090-32k-memory-telemetry.webp)\n\nThere is no paired peak-RAM, swap or peak-GPU series for the 104,805-token needle in the selected documents. Those longer-test fields remain null. Do not relabel the shorter gate's peak as a “128K memory requirement,” and do not treat absent long-context swap data as zero swap.\n\nThe reported MemAvailable scope deserves particular care. The source names a 96 GiB WSL ceiling but does not establish that its available-memory sample and that ceiling describe the same boundary. The site therefore preserves the original number without calculating a supposed spare VM budget from it. Host RSS, mapped file pages, GPU allocations and a VM limit should not simply be added together. Units remain KiB or MiB rather than silently becoming decimal GB.\n\n## mmap success addresses a specific allocation problem\n\nThe source explains that the default CUDA-host override attempted a 98.61 GiB anonymous pinned allocation during validation. The accepted baseline sets `GGML_CUDA_NO_PINNED=1` and retains mmap. That observation explains the author's configuration choice; it does not guarantee identical memory behavior on another operating system, cache state or runtime build.\n\nThe [limitations document](https://github.com/kutaelee/qwen38-flash-next-5090-128k-runtime-recipe/blob/fca75dd108ce04b6a79db3b45934ed58bba893bb/docs/limitations.md) identifies cold expert-page faults and CPU expert work as important constraints for this workstation. Those are source interpretations with a clear configuration attached. They do not establish that every 5090 deployment is CPU limited, nor that every alternative conversion has the same behavior. The [Spark memory article](/guides/qwen-flash-next-memory-context/) describes different hardware and PLE backing experiments; those measurements cannot substitute for missing WSL telemetry.\n\n## Retain the failed and unrun speculation paths\n\nThe repository reports that MTP n=4 preserved eight correctness checks, but two completed comparable prose samples reached only 2.52 and 3.14 tokens/s. Canonical MTP n=1 also preserved tested correctness and was slower across its completed short gates. Its peak GPU memory reached 31,775 MiB. The memory point is a separate MTP short-gate observation, not the baseline's long-context peak.\n\nThe author consequently rejected MTP promotion for this exact MXFP4 conversion and 38-layer CPU-expert split. Ngram speculative decoding was not run because the protocol required a beneficial MTP lane first. “Not run” provides no measured ngram speed, memory improvement or failure rate. The n-gram embedding table remains a separate model component throughout this decision.\n\n![Baseline is retained, MTP speed promotion was rejected, and ngram was not run because its prerequisite failed.](/assets/qwen-5090-mtp-promotion-gates.webp)\n\nThis is a useful operational result: correctness, speed and headroom can disagree. Passing a small correctness gate does not compel deployment of a slower speculative path. It also does not prove general quality equivalence. Read the broader [KV/speculation explanation](/guides/kv-cache-speculative-decoding/) before transferring a favorable MTP result from a different checkpoint or engine.\n\n## What a useful follow-up report would add\n\nFor this configuration, the most valuable missing evidence would pair actual prompt and output counts with telemetry from the same long request. It would identify whether MemAvailable was sampled inside the VM or on the host, retain the cache state, and repeat an unchanged workload before summarizing variation. Any concurrency claim would need completed multi-request runs with their scheduling and memory outcomes. These are editorial suggestions for future evidence, not measurements performed by this site.\n\n### Has the full 128K input been validated here?\n\nThe selected source validates an exact needle in 104,805 input tokens under a 131,072-token configured ceiling. It does not publish a fully filled 131,072-token input gate.\n\n### Does the 104K pass establish repository-analysis quality?\n\nNo. Exact retrieval of one embedded answer is narrower than multi-document reasoning, trustworthy code changes or long output generation.\n\n### Is ngram missing because the embedding table was disabled?\n\nNo. The skipped experiment is speculative decoding. The Q8 n-gram embedding table belongs to the model artifact and is a separate mechanism. The [For AI reference](/for-ai/) keeps these states distinct in the exported conditions.\n" }