Qwen3.8-27B Chisel: Deployment, Speed and the Quality Gate

The Chisel community recipe is a concrete Qwen3.8-27B deployment candidate for one DGX Spark. Its value is the pinned serving recipe and separate speed and quality records. Read the workload and quality gate before repeating a headline rate: the published results use different prompt shapes, cache settings and thinking modes.

This note reads yuanlin2004’s replay recipe, checked on 18 October 2026. The underlying local cohort is filed under October 15; some sessions cross UTC midnight. Run AI Here has not installed the recipe or run the model.

Figure: calculated positions use the source comparison JSON directly. Intervals describe paired task-score differences, not speed variation. Passing the author’s custom bar does not establish non-inferiority.

Identify the target, drafter and actual image separately

The target is unsloth/Qwen3.8-27B-NVFP4, reported with revision prefix f0b7c9e7. The full executed weight revision is not provided. The drafter is yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter, with its weight-file SHA-256 retained in the export. Ten draft tokens, bf16 GDN state and chisel-passes 0.1.0 are part of the configuration. GDN state dtype does not fill an unreported attention-KV dtype.

The environment names vLLM 0.31.1rc1.dev380, driver 580.178.04 and a 2,000 MHz GPU cap. It explicitly says the main tests used a local build with the plugin preinstalled, while the published recipe installs it at startup. The base image has a digest; the saved server.image contains an empty array. Keep the executed local image digest unknown. Environment.

The YAML configures TP1, memory utilization 0.60, a 262,144-token cap and up to four sequences. Those settings do not prove four simultaneous near-limit requests fit. The plugins change recurrent-state handling and the draft output head; the target verification path retains its full vocabulary according to the author. Source code for these recipe plugins is public, while the broader Chisel project is described as not yet public. Pinned serving recipe.

Four numbers belong to two speed cohorts

The mixed test has 25 prompts: nine chat, nine code, six JSON-schema and one long story. Two fresh-start sessions each repeat that set three times. Decode latency excludes the first-token wait. The llama-benchy cohort uses version 0.4.0 and five runs per session. Its thinking override is not explicitly supplied in the method summary and remains unknown in the extraction. These are local runs of an arena-shaped benchmark, separate from the hosted arena entry. Method.

A median latency is not whole-request throughput. Nor can a two-user total be compared directly with the one-user latency. Use the timing guide when deciding which measure fits an application. Avoid a “fastest” verdict across these conditions.

Read the quality result beyond the pass flag

The source compares against Qwen/Qwen3.8-27B BF16 at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, with bf16 KV. Quality evaluation uses lm-evaluation-harness 0.4.13, chat templates, greedy decoding and thinking off. The BF16 cohort is filed under October 9; the comparison JSON retains older original candidate paths. Candidate image, clock and plugin settings must not be silently assigned to the reference arm.

The custom bar requires at least 97% recovery, a permitted answer-length ratio and no task flagged clearly worse. It passes. A stricter flag requires the lower interval bound to exceed a loss of one point; all four non_inferior flags are false. Both results must survive an AI summary. Exact comparison JSON.

The comparison code computes paired item differences and a normal-approximation interval, mean ± 1.96 standard errors. These are not bootstrap intervals. Intervals crossing zero do not prove equivalence, and scores on four tasks do not guarantee general quality preservation.

A useful local acceptance plan

Before adopting the recipe, identify the quantized artifact, drafter, plugins and executed image. Check the required API features and memory allocation at your actual context and concurrency. Then evaluate both timing and correctness on representative prompts. Keep speed thinking-on results separate from thinking-off quality tests. The source record and adjacent JSON/Markdown retain the twelve selected observations, four paired comparisons and missing identifiers.