THE SOURCE IN CONTEXT
The recipe is a concrete deployment candidate, but speed and quality use different request shapes and thinking modes. The source custom quality bar passes while its non-inferiority flags are false; preserve both and leave the executed local image digest unknown.
What is pinned and what remains unknown?
The recipe names unsloth Qwen3.8-27B NVFP4, ten DFlash2 draft tokens, bf16 GDN state and chisel-passes 0.1.0. The measured target revision is only a short prefix. A recipe base-image digest is available, but the main tests used a local build and the saved server.image is empty; the executed image digest remains unknown.
Source: yuanlin2004 · pinned document
Which speed cohort is being quoted?
The mixed-prompt test reports median decode latency and TTFT with thinking on and medium effort, one request at a time, up to 256 output tokens and two fresh-start sessions. Local llama-benchy results instead use 16K retained context, 128 output tokens and prefix cache on; its two-user throughput is aggregate, not per user.
Source: yuanlin2004 · pinned document
How should the quality gate be read?
Quality tests use thinking off and compare four tasks against a separately recorded BF16 reference. The custom bar passes, but all four non_inferior flags and the overall all_non_inferior flag are false. Preserve both gates, the raw scores, sample counts, paired differences and intervals. These results do not establish equivalence or general quality preservation.
Source: yuanlin2004 · pinned document
How were the intervals computed?
The comparison code uses matched per-question differences and a normal approximation: mean difference plus or minus 1.96 sample standard errors. It is not a bootstrap interval. The non-inferiority flag requires the lower bound above a loss of one point, while the author custom bar uses score recovery, a clearly-worse check and answer-length limits.
Source: yuanlin2004 · pinned document
What should an operator verify next?
The configured 262144 context cap and four-sequence capacity are launch settings. They do not establish simultaneous near-limit fit. Verify the exact quantized artifact, drafter, plugin and image, then use representative requests and correctness tasks. Do not transfer candidate-specific image, clock or plugin settings to the BF16 reference when they were not reported.
Source: yuanlin2004 · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| Chisel replay mixed-prompt median decode latency | decode latency | 15.35 ms/tokensource-reported median over two fresh-start sessions | Original row ↗ |
| Chisel replay mixed-prompt median TTFT | ttft | 241 mssource-reported median on the same prompt cohort | Original row ↗ |
| Local llama-benchy 16K C1 | generation throughput | 30.0 tokens/ssource-reported summary; separate from mixed-prompt median latency | Original row ↗ |
| Local llama-benchy 16K C2 | generation throughput | 58.0 tokens/ssource-reported summary; separate from mixed-prompt median latency | Original row ↗ |
| gsm8k BF16 reference | task score | 90.06823351023503 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| gsm8k NVFP4 replay | task score | 89.46171341925702 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| ifeval BF16 reference | task score | 80.96118299445472 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| ifeval NVFP4 replay | task score | 81.88539741219964 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| humaneval_chat BF16 reference | task score | 95.1219512195122 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| humaneval_chat NVFP4 replay | task score | 94.51219512195121 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| longbench_narrativeqa BF16 reference | task score | 34.83234254995091 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
| longbench_narrativeqa NVFP4 replay | task score | 35.235443073646884 score pointssource paired-comparison raw score; retain task-specific definition | Original row ↗ |
Chisel replay mixed-prompt median decode latency · decode latency · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | 2026-10-15 source cohort; some recorded sessions cross UTC midnight |
| thinking | True |
| reasoning_effort | medium |
| greedy | True |
| temperature | 0 |
| top_p | Not specified |
| concurrency | 1 |
| input_tokens | Not specified |
| max_output_tokens | 256 |
| output_tokens | Not specified |
| prompt_count | 25 |
| prompt_mix | {"chat": 9, "code": 9, "JSON_schema": 6, "long_story": 1} |
| repeats_per_session | 3 |
| fresh_start_sessions | 2 |
| measurement_denominator | (last streamed token time - first) / (output tokens - 1); median latency, not whole-request served rate |
Chisel replay mixed-prompt median TTFT · ttft · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | 2026-10-15 source cohort; some recorded sessions cross UTC midnight |
| thinking | True |
| reasoning_effort | medium |
| greedy | True |
| temperature | 0 |
| top_p | Not specified |
| concurrency | 1 |
| input_tokens | Not specified |
| max_output_tokens | 256 |
| output_tokens | Not specified |
| prompt_count | 25 |
| prompt_mix | {"chat": 9, "code": 9, "JSON_schema": 6, "long_story": 1} |
| repeats_per_session | 3 |
| fresh_start_sessions | 2 |
| measurement_denominator | request start to first streamed token |
Local llama-benchy 16K C1 · generation throughput · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | 2026-10-15 source cohort; some recorded sessions cross UTC midnight |
| benchmark | llama-benchy 0.4.0, local execution of spark-arena-v2 shape; not arena scores |
| concurrency | 1 |
| input_tokens | 2048 |
| retained_context_tokens | 16384 |
| output_tokens | 128 |
| prefix_cache | True |
| repeats_per_session | 5 |
| fresh_start_sessions | 2 |
| thinking | Not specified |
| measurement_denominator | source local llama-benchy generation throughput; total across users for C2 |
Local llama-benchy 16K C2 · generation throughput · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | 2026-10-15 source cohort; some recorded sessions cross UTC midnight |
| benchmark | llama-benchy 0.4.0, local execution of spark-arena-v2 shape; not arena scores |
| concurrency | 2 |
| input_tokens | 2048 |
| retained_context_tokens | 16384 |
| output_tokens | 128 |
| prefix_cache | True |
| repeats_per_session | 5 |
| fresh_start_sessions | 2 |
| thinking | Not specified |
| measurement_denominator | source local llama-benchy generation throughput; total across users for C2 |
gsm8k BF16 reference · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | Qwen/Qwen3.8-27B |
| model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reported_model_revision_prefix | Not specified |
| system | Not specified |
| node_count | Not specified |
| driver | Not specified |
| GPU_clock_cap_MHz | Not specified |
| engine | Not specified |
| executed_engine_version | Not specified |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | Not specified |
| executed_local_image_digest | Not specified |
| executed_image_boundary | Reference model and bf16 KV are reported; candidate-specific image, plugin and clock settings are not imputed to this arm. |
| plugin | Not specified |
| draft_model | Not specified |
| draft_weight_sha256 | Not specified |
| draft_tokens | Not specified |
| GDN_state_dtype | Not specified |
| KV_dtype | Not specified |
| configured_context_limit_tokens | Not specified |
| configured_max_requests | Not specified |
| GPU_memory_utilization | Not specified |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | BF16 reference |
| task | gsm8k |
| n | 1319 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | 5-shot accuracy |
| paired_difference_normal_CI95_points | [-1.8137451343135735, 0.6007049523575464] |
| non_inferior | False |
| author_custom_bar_passed | True |
gsm8k NVFP4 replay · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | NVFP4 replay |
| task | gsm8k |
| n | 1319 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | 5-shot accuracy |
| paired_difference_normal_CI95_points | [-1.8137451343135735, 0.6007049523575464] |
| non_inferior | False |
| author_custom_bar_passed | True |
ifeval BF16 reference · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | Qwen/Qwen3.8-27B |
| model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reported_model_revision_prefix | Not specified |
| system | Not specified |
| node_count | Not specified |
| driver | Not specified |
| GPU_clock_cap_MHz | Not specified |
| engine | Not specified |
| executed_engine_version | Not specified |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | Not specified |
| executed_local_image_digest | Not specified |
| executed_image_boundary | Reference model and bf16 KV are reported; candidate-specific image, plugin and clock settings are not imputed to this arm. |
| plugin | Not specified |
| draft_model | Not specified |
| draft_weight_sha256 | Not specified |
| draft_tokens | Not specified |
| GDN_state_dtype | Not specified |
| KV_dtype | Not specified |
| configured_context_limit_tokens | Not specified |
| configured_max_requests | Not specified |
| GPU_memory_utilization | Not specified |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | BF16 reference |
| task | ifeval |
| n | 541 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | prompt-level strict |
| paired_difference_normal_CI95_points | [-1.560611007417151, 3.409039842906985] |
| non_inferior | False |
| author_custom_bar_passed | True |
ifeval NVFP4 replay · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | NVFP4 replay |
| task | ifeval |
| n | 541 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | prompt-level strict |
| paired_difference_normal_CI95_points | [-1.560611007417151, 3.409039842906985] |
| non_inferior | False |
| author_custom_bar_passed | True |
humaneval_chat BF16 reference · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | Qwen/Qwen3.8-27B |
| model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reported_model_revision_prefix | Not specified |
| system | Not specified |
| node_count | Not specified |
| driver | Not specified |
| GPU_clock_cap_MHz | Not specified |
| engine | Not specified |
| executed_engine_version | Not specified |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | Not specified |
| executed_local_image_digest | Not specified |
| executed_image_boundary | Reference model and bf16 KV are reported; candidate-specific image, plugin and clock settings are not imputed to this arm. |
| plugin | Not specified |
| draft_model | Not specified |
| draft_weight_sha256 | Not specified |
| draft_tokens | Not specified |
| GDN_state_dtype | Not specified |
| KV_dtype | Not specified |
| configured_context_limit_tokens | Not specified |
| configured_max_requests | Not specified |
| GPU_memory_utilization | Not specified |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | BF16 reference |
| task | humaneval_chat |
| n | 164 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | chat pass@1 |
| paired_difference_normal_CI95_points | [-2.6839968720184646, 1.4644846768965132] |
| non_inferior | False |
| author_custom_bar_passed | True |
humaneval_chat NVFP4 replay · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | NVFP4 replay |
| task | humaneval_chat |
| n | 164 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | chat pass@1 |
| paired_difference_normal_CI95_points | [-2.6839968720184646, 1.4644846768965132] |
| non_inferior | False |
| author_custom_bar_passed | True |
longbench_narrativeqa BF16 reference · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | Qwen/Qwen3.8-27B |
| model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reported_model_revision_prefix | Not specified |
| system | Not specified |
| node_count | Not specified |
| driver | Not specified |
| GPU_clock_cap_MHz | Not specified |
| engine | Not specified |
| executed_engine_version | Not specified |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | Not specified |
| executed_local_image_digest | Not specified |
| executed_image_boundary | Reference model and bf16 KV are reported; candidate-specific image, plugin and clock settings are not imputed to this arm. |
| plugin | Not specified |
| draft_model | Not specified |
| draft_weight_sha256 | Not specified |
| draft_tokens | Not specified |
| GDN_state_dtype | Not specified |
| KV_dtype | Not specified |
| configured_context_limit_tokens | Not specified |
| configured_max_requests | Not specified |
| GPU_memory_utilization | Not specified |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | BF16 reference |
| task | longbench_narrativeqa |
| n | 200 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | F1, full stories |
| paired_difference_normal_CI95_points | [-1.6421295670363527, 2.4483306144283086] |
| non_inferior | False |
| author_custom_bar_passed | True |
longbench_narrativeqa NVFP4 replay · task score · conditions
| model | Qwen3.8-27B |
|---|---|
| model_artifact | unsloth/Qwen3.8-27B-NVFP4 |
| model_revision | Not specified |
| reported_model_revision_prefix | f0b7c9e7 |
| system | 1 x DGX Spark GB10 sm_121; 128 GB unified memory |
| node_count | 1 |
| driver | 580.178.04 |
| GPU_clock_cap_MHz | 2000 |
| engine | vLLM |
| executed_engine_version | 0.31.1rc1.dev380 |
| runtime_execution_revision | Not specified |
| recipe_base_image_digest | sha256:b77010699ec42ab88b3409f4dd101f2288dfd8895e2778e5bb737e03a20953f0 |
| executed_local_image_digest | Not specified |
| executed_image_boundary | benchmark and quality used a local build with the same plugin code preinstalled; saved server.image is []; do not substitute the recipe base digest as the executed image digest |
| plugin | chisel-passes 0.1.0 |
| draft_model | yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter |
| draft_weight_sha256 | 8d464c9f597c41a2e8917c585454239be14f254592f114675fd9d5d5dbc92f4c |
| draft_tokens | 10 |
| GDN_state_dtype | bfloat16 |
| KV_dtype | Not specified |
| configured_context_limit_tokens | 262144 |
| configured_max_requests | 4 |
| GPU_memory_utilization | 0.6 |
| experiment_date | reference cohort 2026-10-09; candidate cohort filed 2026-10-15; compare file retains earlier original paths |
| quality_arm | NVFP4 replay |
| task | longbench_narrativeqa |
| n | 200 |
| thinking | False |
| greedy | True |
| concurrency | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| benchmark | lm-evaluation-harness 0.4.13; chat template; full task |
| reference_model | Qwen/Qwen3.8-27B |
| reference_model_revision | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| reference_weight_precision | bfloat16 |
| reference_KV_dtype | bfloat16 |
| metric_definition | F1, full stories |
| paired_difference_normal_CI95_points | [-1.6421295670363527, 2.4483306144283086] |
| non_inferior | False |
| author_custom_bar_passed | True |
Local Spark Qwen draft, source-audited and edited by Run AI Here with Codex; no independent model execution.