Qwen Flash-Next: RTX 5090 CPU offload and long-context gates

THE SOURCE IN CONTEXT

The pinned MXFP4 GGUF runs through ik_llama.cpp merge d854dfd36a7262fc46cb7542716b04a9f6d4b47d on one RTX 5090 with WSL2, a 128 GiB-class host and 96 GiB VM ceiling. The baseline puts 38 expert layers on CPU, keeps mmap, disables the CUDA pinned-host override and uses Q8_0 KV. One request and one slot are used throughout; this is not full-model GPU residence.

Run AI Here editorial · Published 2026-10-18 · Updated 2026-10-18

Which memory placement belongs to the report?

Source: kutaelee · pinned document kutaelee · pinned document

What input was actually completed?

The source reports a 26253-token needle under a 32768 window, a 61-token short smoke under a 131072 window, and a 104805-token exact needle under that longer window. Throughput comes from server timing. Completion lengths, needle depth, per-gate sampling and repeated-run dispersion are absent. A short smoke with a large configured ceiling is not a full-window input test.

Source: kutaelee · pinned document kutaelee · pinned document

Which memory observations are paired with which gate?

Startup, peak RSS, minimum MemAvailable, swap and peak GPU memory are reported for the completed 32K monitoring gate. The longer 104805-token gate has no paired peak telemetry in these selected documents. The MemAvailable sampling boundary is unspecified, so its relationship to the WSL ceiling cannot be assumed. Keep original KiB and MiB labels and do not sum overlapping allocations.

Source: kutaelee · pinned document kutaelee · pinned document

What happened to speculation?

MTP n=4 and canonical n=1 passed the tested correctness checks but failed speed promotion on this particular CPU-expert split. The canonical MTP short gates reached 31775 MiB peak GPU memory. Ngram speculative decoding was not run because its prerequisite failed; it has no measured benefit here. This is separate from the Q8 n-gram embedding table present in the artifact.

Source: kutaelee · pinned document kutaelee · pinned document

Reported figures and their conditions

Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.

Attributed source results; no cross-source ranking
Configuration / workload	Metric	Reported value	Evidence
32K needle gate prefill_rate	prefill rate	271.64 tokens/sone reported completed gate; no dispersion	Original row ↗
32K needle gate decode_rate	decode rate	16.53 tokens/sone reported completed gate; no dispersion	Original row ↗
128K server smoke prefill_rate	prefill rate	17.02 tokens/sone reported completed gate; no dispersion	Original row ↗
128K server smoke decode_rate	decode rate	15.26 tokens/sone reported completed gate; no dispersion	Original row ↗
128K long-context needle prefill_rate	prefill rate	245.72 tokens/sone reported completed gate; no dispersion	Original row ↗
128K long-context needle decode_rate	decode rate	13.94 tokens/sone reported completed gate; no dispersion	Original row ↗
32K telemetry startup	startup	25.15 ssource monitored peak/minimum; no distribution	Original row ↗
32K telemetry host_rss	host rss	53901956 KiBsource monitored peak/minimum; no distribution	Original row ↗
32K telemetry minimum_memavailable	minimum memavailable	86701016 KiBsource monitored peak/minimum; no distribution	Original row ↗
32K telemetry swap_use	swap use	63708 KiBsource monitored peak/minimum; no distribution	Original row ↗
32K telemetry GPU_memory	GPU memory	25813 MiBsource monitored peak/minimum; no distribution	Original row ↗
canonical MTP n1 peak GPU memory	GPU memory	31775 MiBmaximum in completed MTP short gates	Original row ↗

32K needle gate prefill_rate · prefill rate · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
workload	32K needle gate
memory_cohort	completed 32K monitoring

32K needle gate decode_rate · decode rate · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
workload	32K needle gate
memory_cohort	completed 32K monitoring

128K server smoke prefill_rate · prefill rate · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	61
context_limit_tokens	131072
cache_state	Not specified
workload	128K server smoke
memory_cohort	no paired 128K telemetry published

128K server smoke decode_rate · decode rate · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	61
context_limit_tokens	131072
cache_state	Not specified
workload	128K server smoke
memory_cohort	no paired 128K telemetry published

128K long-context needle prefill_rate · prefill rate · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	104805
context_limit_tokens	131072
cache_state	Not specified
workload	128K long-context needle
memory_cohort	no paired 128K telemetry published

128K long-context needle decode_rate · decode rate · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	104805
context_limit_tokens	131072
cache_state	Not specified
workload	128K long-context needle
memory_cohort	no paired 128K telemetry published

32K telemetry startup · startup · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	startup_duration
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

32K telemetry host_rss · host rss · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	peak_host_RSS
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

32K telemetry minimum_memavailable · minimum memavailable · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	minimum_MemAvailable_in_reported_monitor_scope
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

32K telemetry swap_use · swap use · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	maximum_swap_use
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

32K telemetry GPU_memory · GPU memory · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	peak_GPU_memory
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

canonical MTP n1 peak GPU memory · GPU memory · conditions

Conditions for this result only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	1
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	Not specified
context_limit_tokens	131072
cache_state	Not specified
memory_cohort	canonical MTP n1 short gates; not baseline or long-context peak
memory_kind	peak_GPU_memory

Original source-based interpretation; local Spark Qwen draft with Codex factual review, no site model benchmark.

Key reading notes

One source document can contain different experiments and versions; not an independent experiment count.

Source identity and full condition receipt

Source identity

Attribution and exact source identity
Original publisher	kutaelee
Reference type	Performance report
Model ID	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
Default recipe artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
Runtime topics	["llama.cpp"]
Source revision	fca75dd108ce04b6a79db3b45934ed58bba893bb
Source commit date — not experiment date	2026-09-18T12:12:22Z
Source checked	2026-10-18
Declared license	mit

Reported conditions

Unknown fields remain unspecified; context limits and measured input lengths are different
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
parallelism	Not specified
kv_dtype	Q8_0
context_limit_tokens	32768
experiment_date	Not specified
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
weight_hash	Not specified
runtime_execution_revision	Not specified
input_tokens	Not specified
output_tokens	Not specified
concurrency	Not specified
cache_state	Not specified
quantization	Not specified
speculative_decoding	Not specified
temperature	Not specified
top_p	Not specified

32K needle gate prefill_rate

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
workload	32K needle gate
memory_cohort	completed 32K monitoring

Original experiment section

32K needle gate decode_rate

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
workload	32K needle gate
memory_cohort	completed 32K monitoring

Original experiment section

128K server smoke prefill_rate

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	61
context_limit_tokens	131072
cache_state	Not specified
workload	128K server smoke
memory_cohort	no paired 128K telemetry published

Original experiment section

128K server smoke decode_rate

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	61
context_limit_tokens	131072
cache_state	Not specified
workload	128K server smoke
memory_cohort	no paired 128K telemetry published

Original experiment section

128K long-context needle prefill_rate

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	104805
context_limit_tokens	131072
cache_state	Not specified
workload	128K long-context needle
memory_cohort	no paired 128K telemetry published

Original experiment section

128K long-context needle decode_rate

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	104805
context_limit_tokens	131072
cache_state	Not specified
workload	128K long-context needle
memory_cohort	no paired 128K telemetry published

Original experiment section

32K telemetry startup

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	startup_duration
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

Original experiment section

32K telemetry host_rss

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	peak_host_RSS
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

Original experiment section

32K telemetry minimum_memavailable

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	minimum_MemAvailable_in_reported_monitor_scope
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

Original experiment section

32K telemetry swap_use

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	maximum_swap_use
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

Original experiment section

32K telemetry GPU_memory

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	0
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	26253
context_limit_tokens	32768
cache_state	Not specified
memory_cohort	completed 32K gate only; not paired with the 104805-token gate
memory_kind	peak_GPU_memory
MemAvailable_sampling_scope	Not specified
VM_ceiling_and_host_MemAvailable_same_scope_verified	False

Original experiment section

canonical MTP n1 peak GPU memory

Conditions of this profile only
model	Qwen3.8-Flash-Next
model_artifact	jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF
model_revision	4a768bf4458a83e1c80667b4d818ec721080546e
artifact_SHA256	5a4236a6eeda73d23c151f8ed19e5457b0bf1935e1c02c1d42d98a0e26dadde2
engine	ik_llama.cpp
runtime_execution_revision	d854dfd36a7262fc46cb7542716b04a9f6d4b47d
MTP_patch_head	108dfaa77fdaa138c46b043bfff0e94fbe2adff7
system	single RTX 5090 32607 MiB; 16-core/32-thread desktop CPU; 128 GiB-class host; Ubuntu WSL2 with 96 GiB VM ceiling
node_count	1
concurrency	1
configured_max_requests	1
weight_precision	MXFP4 GGUF, Q8 n-gram table
CPU_expert_layers	38
defer_experts	True
mmap	True
GGML_CUDA_NO_PINNED	True
kv_dtype	Q8_0
draft_tokens	1
ngram_speculation	False
ngram_embedding_table_present	True
temperature	Not specified
top_p	Not specified
thinking	Not specified
output_tokens	Not specified
actual_output_tokens	Not specified
repetitions	Not specified
experiment_date	Not specified
throughput_origin	server timing fields, not a client estimate
input_tokens	Not specified
context_limit_tokens	131072
cache_state	Not specified
memory_cohort	canonical MTP n1 short gates; not baseline or long-context peak
memory_kind	peak_GPU_memory

Original experiment section

Limits

No independent site model execution.
Missing sampling, exact executed image digests and measurement boundaries remain null.
Configured window, completed input, scheduler ceiling and measured full-window concurrency are separate quantities.

The complete original tables and raw evidence remain in the linked reports. Selected attributed figures retain their own conditions. Results from different models, weights, runtimes, workloads or cohorts must be read separately.

License scope

Source declaration; models, runtime and dependencies retain their exact terms.

This index publishes original explanations, limited attributed factual selections and source pointers. It does not distribute model weights or source repositories. Third-party material retains its own terms.

Related references — check the checkpoint

Qwen3.8-Flash-Next: official model and deployment card · Official model card
Qwen3.8-Flash-Next: vLLM recipe · Deployment recipe
Qwen3.8-27B: vLLM recipe · Deployment recipe
Qwen3.8-2.4T-A95B: vLLM recipe · Deployment recipe

Qwen Flash-Next: RTX 5090 CPU offload and long-context gates

Which memory placement belongs to the report?

What input was actually completed?

Which memory observations are paired with which gate?

What happened to speculation?

Reported figures and their conditions

Key reading notes

Source identity

Reported conditions

32K needle gate prefill_rate

32K needle gate decode_rate

128K server smoke prefill_rate

128K server smoke decode_rate

128K long-context needle prefill_rate

128K long-context needle decode_rate

32K telemetry startup

32K telemetry host_rss

32K telemetry minimum_memavailable

32K telemetry swap_use

32K telemetry GPU_memory

canonical MTP n1 peak GPU memory

Limits

License scope

Related references — check the checkpoint

Your privacy choices