THE SOURCE IN CONTEXT
The September NVFP4 report and October EXL3 correction are different deployment lines. The EXL3 author withdrew its old task scores after finding an unloaded router bias. Current evidence is a same-quant probability check; it does not revalidate old task scores or import the NVFP4 task panel.
Which scores remain historical NVFP4 observations?
The September 3 TP3+EP Marlin NVFP4 t10 campaign uses GMU 0.85, FP8 KV, DFlash2 k7 and a decimal-million configured window. Full MMLU is 14042 zero-shot loglikelihood questions with exact accuracy 0.8594217347956131 and harness SE 0.0028276391244761038. Tool-eval has eight trials over 88 hard scenarios, mean 87.8 and source SD 0.9. GSM8K is only 200 questions, 8-shot,188 correct. IFEval uses a custom unverified evaluator.
Source: NNNtrance · pinned document NNNtrance · pinned document NNNtrance · pinned document
Why did the EXL3 author withdraw its old scores?
The October 8 notice identifies missing e_score_correction_bias in the old cuda-exl3 plugin. Wrong expert routing invalidates that repository's old task panels. The new TP3 main stack loads the bias and fails boot if missing. TP2 remains unported and untested. This notice is scoped to the EXL3 plugin line, not proof of the same bug in the separate historical NVFP4 line.
Source: NNNtrance · pinned document NNNtrance · pinned document NNNtrance · pinned document
What was rerun on corrected EXL3?
The detailed KL table compares the same EXL3 checkpoint to exllamav3 1.5.4 on six author-written texts. Assistant KL is 0.1033 pre-fix and 0.00828 on production a4i. Perplexity 2.222 belongs to bias-fix a4g; production perplexity is not reported in that table. The reference is quantized, not BF16. Task benchmarks have not been rerun, and the prepared text set and per-position logits are not public.
Source: NNNtrance · pinned document NNNtrance · pinned document
How are failed attempts kept?
Historical NVFP4 MMLU initially hits a harness queue timeout, and needle tests initially exceed a 120s request cap; same-content reruns use longer timeouts with zero engine errors reported. These are harness retries, not model-accuracy zeros or engine recoveries. A separate model line's completed task scores cannot fill the corrected EXL3 missing scores.
Source: NNNtrance · pinned document NNNtrance · pinned document NNNtrance · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| Pre-fix assistant KL | KL reference to engine | 0.1033 natssource-reported observation; dispersion unreported | Original row ↗ |
| Production assistant KL | KL reference to engine | 0.00828 natssource-reported observation; dispersion unreported | Original row ↗ |
| Bias-fix assistant perplexity | perplexity | 2.222 perplexitysource-reported observation; dispersion unreported | Original row ↗ |
Pre-fix assistant KL · KL reference to engine · conditions
| model | GLM-5.3-Flash |
|---|---|
| model_artifact | turboderp/GLM-5.3-Flash-exl3 |
| model_revision | 2a30229e67012798ba9f0cd832bb78abf4c363d5 |
| engine | vLLM 0.30.1rc1.dev709+g21d93d0d8 plus cuda-exl3 and 14 patches |
| base_engine_revision | 21d93d0d8c0e9627900020382bfce4730e61cab7 |
| runtime_execution_revision | Not specified |
| production_plugin_revision | 448f1d6b8964b96e7d4e299494538bfc240149ef |
| executed_image_digest | Not specified |
| production_image_id | sha256:8c4f560d92ba190ed84983bdf12e9975c5b4f1ec6dc380b5b93041b2a19ac261 |
| system | three DGX Spark GB10 |
| node_count | 3 |
| parallelism | TP3+EP |
| weight_precision | EXL3 4.05 bpw full scope |
| kv_dtype | FP8 |
| experiment_date | 2026-10-07 to 2026-10-08 |
| concurrency | Not specified |
| input_tokens | 21072 |
| scored_positions | 21066 |
| assistant_positions | 6601 |
| quality_panel | six author-written texts; text set and per-position logits not published |
| reference | exllamav3 1.5.4 running SAME quantized checkpoint, not BF16 |
| metric_denominator | assistant-token KL, reference top512 + remainder bin; lower-bound estimate |
| temperature | Not specified |
| thinking | reference panel has reasoning effort low / clear_thinking |
| cache_state | prefill target log probabilities, before DFlash proposals |
| memory_utilization | 0.75 |
| host_peak_RAM | Not specified |
| current_task_benchmark_score | Not specified |
| old_EXL3_task_scores_withdrawn | True |
| TP2_corrected_or_tested | False |
| measured_build | a4f |
| production_perplexity | Not specified |
| pre_fix_comparison_is_retracted_task_score | False |
Production assistant KL · KL reference to engine · conditions
| model | GLM-5.3-Flash |
|---|---|
| model_artifact | turboderp/GLM-5.3-Flash-exl3 |
| model_revision | 2a30229e67012798ba9f0cd832bb78abf4c363d5 |
| engine | vLLM 0.30.1rc1.dev709+g21d93d0d8 plus cuda-exl3 and 14 patches |
| base_engine_revision | 21d93d0d8c0e9627900020382bfce4730e61cab7 |
| runtime_execution_revision | Not specified |
| production_plugin_revision | 448f1d6b8964b96e7d4e299494538bfc240149ef |
| executed_image_digest | Not specified |
| production_image_id | sha256:8c4f560d92ba190ed84983bdf12e9975c5b4f1ec6dc380b5b93041b2a19ac261 |
| system | three DGX Spark GB10 |
| node_count | 3 |
| parallelism | TP3+EP |
| weight_precision | EXL3 4.05 bpw full scope |
| kv_dtype | FP8 |
| experiment_date | 2026-10-07 to 2026-10-08 |
| concurrency | Not specified |
| input_tokens | 21072 |
| scored_positions | 21066 |
| assistant_positions | 6601 |
| quality_panel | six author-written texts; text set and per-position logits not published |
| reference | exllamav3 1.5.4 running SAME quantized checkpoint, not BF16 |
| metric_denominator | assistant-token KL, reference top512 + remainder bin; lower-bound estimate |
| temperature | Not specified |
| thinking | reference panel has reasoning effort low / clear_thinking |
| cache_state | prefill target log probabilities, before DFlash proposals |
| memory_utilization | 0.75 |
| host_peak_RAM | Not specified |
| current_task_benchmark_score | Not specified |
| old_EXL3_task_scores_withdrawn | True |
| TP2_corrected_or_tested | False |
| measured_build | a4i |
| production_perplexity | Not specified |
| pre_fix_comparison_is_retracted_task_score | False |
Bias-fix assistant perplexity · perplexity · conditions
| model | GLM-5.3-Flash |
|---|---|
| model_artifact | turboderp/GLM-5.3-Flash-exl3 |
| model_revision | 2a30229e67012798ba9f0cd832bb78abf4c363d5 |
| engine | vLLM 0.30.1rc1.dev709+g21d93d0d8 plus cuda-exl3 and 14 patches |
| base_engine_revision | 21d93d0d8c0e9627900020382bfce4730e61cab7 |
| runtime_execution_revision | Not specified |
| production_plugin_revision | 448f1d6b8964b96e7d4e299494538bfc240149ef |
| executed_image_digest | Not specified |
| production_image_id | sha256:8c4f560d92ba190ed84983bdf12e9975c5b4f1ec6dc380b5b93041b2a19ac261 |
| system | three DGX Spark GB10 |
| node_count | 3 |
| parallelism | TP3+EP |
| weight_precision | EXL3 4.05 bpw full scope |
| kv_dtype | FP8 |
| experiment_date | 2026-10-07 to 2026-10-08 |
| concurrency | Not specified |
| input_tokens | 21072 |
| scored_positions | 21066 |
| assistant_positions | 6601 |
| quality_panel | six author-written texts; text set and per-position logits not published |
| reference | exllamav3 1.5.4 running SAME quantized checkpoint, not BF16 |
| metric_denominator | assistant-token KL, reference top512 + remainder bin; lower-bound estimate |
| temperature | Not specified |
| thinking | reference panel has reasoning effort low / clear_thinking |
| cache_state | prefill target log probabilities, before DFlash proposals |
| memory_utilization | 0.75 |
| host_peak_RAM | Not specified |
| current_task_benchmark_score | Not specified |
| old_EXL3_task_scores_withdrawn | True |
| TP2_corrected_or_tested | False |
| measured_build | a4g |
| production_perplexity | Not specified |
| pre_fix_comparison_is_retracted_task_score | False |
Local Spark Qwen draft; Codex source audit and rewrite. No site model benchmark.