THE SOURCE IN CONTEXT
The source reports 49/50 on a GSM8K subset and a HumanEval composite of 18/20 after retrying four timed-out problems. The initial HumanEval result was 14/20. Unequal cloud sampling, unpinned weight identity and small subsets do not prove lossless IQ1_S quantization or full 1M-input capability.
Which identity and topology are known?
The August 7 publisher labels the artifact Kimi K3/Kimi-Linear IQ1_S with nine GGUF shards. No exact upstream checkpoint revision or weight hash is given, so the label is not an independently verified official artifact. Three DGX Sparks use layer-split RPC through a pwilkin llama.cpp fork, not TP3; the actual transport configuration uses MTU4096, while the fabric overview separately says MTU9000.
Source: chishiki37 · pinned document
What is measured versus configured?
The report gives approximate steady decode and prefill summaries through a 131K-labelled input. The 1048576-token window and one slot are configured limits. About 119–120 GB per node is an approximate occupancy report with no peak-telemetry method; it cannot establish peak headroom at a completed million-token input. Exact engine commit and runtime image are also absent.
Source: chishiki37 · pinned document
How are scores and retries retained?
The local GSM8K subset has 49 correct of 50. Raw HumanEval results show 14 correct of 20 initially, including four 600-second timeouts. The selected timeout-only rerun passes all four, producing the reported composite 18/20. The two remaining assertion failures are IDs2 and7. Do not turn the composite into a first-pass result or call every failure a floating-point issue.
Source: chishiki37 · pinned document chishiki37 · pinned document chishiki37 · pinned document
Why is this not a quantization equivalence test?
Cloud Kimi uses temperature1 and a smaller GSM8K output cap while local uses temperature0.2; the GLM comparison raises its cap after output exhaustion. These are small fixed subsets with no unquantized local control. Matching selected scores and failure IDs does not prove equivalence, zero quantization cost or a general model ranking. No standardized full-suite leaderboard score is created here.
Source: chishiki37 · pinned document chishiki37 · pinned document chishiki37 · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| steady decode | decode rate | 2.9 tokens/sapproximate source summary | Original row ↗ |
| 8K prefill | prefill rate | 79.6 tokens/ssource summary; exact per-run token counts not supplied | Original row ↗ |
| 32K prefill | prefill rate | 43.9 tokens/ssource summary; exact per-run token counts not supplied | Original row ↗ |
| 131K prefill | prefill rate | 48.5 tokens/ssource summary; exact per-run token counts not supplied | Original row ↗ |
| GSM8K local subset | accuracy | 98.0 percentsource fixed-subset score, no confidence interval | Original row ↗ |
| HumanEval initial local subset | accuracy | 70.0 percentsource fixed-subset score, no confidence interval | Original row ↗ |
| HumanEval timeout retries | accuracy | 100.0 percentsource fixed-subset score, no confidence interval | Original row ↗ |
| HumanEval local after selected retries | accuracy | 90.0 percentsource composite score after retries | Original row ↗ |
steady decode · decode rate · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| workload | steady decode |
| input_length_source_label | Not specified |
8K prefill · prefill rate · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| workload | 8K prefill |
| input_length_source_label | 8K |
32K prefill · prefill rate · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| workload | 32K prefill |
| input_length_source_label | 32K |
131K prefill · prefill rate · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| workload | 131K prefill |
| input_length_source_label | 131K |
GSM8K local subset · accuracy · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | 0.2 |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | 8192 |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| benchmark | GSM8K canonical test subset |
| correct | 49 |
| total | 50 |
| subset_ids | 50 source-indexed questions; fixed subset, not full benchmark |
| timeout_seconds | 1800 |
| retry_policy | GLM comparison changed output cap; local retries unreported |
HumanEval initial local subset · accuracy · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | 0.2 |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | 8192 |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| benchmark | HumanEval subset IDs 0 through 19 |
| correct | 14 |
| total | 20 |
| subset_ids | [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] |
| timeout_seconds | 600 |
| timeouts | 4 |
| retry_policy | initial run; four timeouts retained as failures |
HumanEval timeout retries · accuracy · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | 0.2 |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | 8192 |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| benchmark | HumanEval timeout-only subset |
| correct | 4 |
| total | 4 |
| subset_ids | [1, 6, 10, 17] |
| timeout_seconds | Not specified |
| retry_policy | rerun only four timed-out problems with higher limits; not fresh 20-problem pass |
HumanEval local after selected retries · accuracy · conditions
| model | Kimi K3 IQ1_S (publisher label; exact upstream checkpoint not pinned) |
|---|---|
| model_artifact | k3-neuron-iq1s, nine GGUF shards |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | pwilkin llama.cpp fork, build-rdma |
| system | three DGX Spark GB10, 128 GB unified memory each |
| node_count | 3 |
| parallelism | layer-split RPC; two remote workers, not TP3 |
| weight_precision | IQ1_S |
| kv_dtype | f16 |
| context_limit_tokens | 1048576 |
| configured_max_requests | 1 |
| concurrency | 1 |
| load_mode | none |
| actual_transport_MTU | 4096 |
| fabric_overview_MTU | 9000 |
| temperature | 0.2 |
| top_p | Not specified |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | 8192 |
| actual_output_tokens | Not specified |
| cache_state | Not specified |
| repetitions | Not specified |
| experiment_date | 2026-08-07 |
| peak_runtime_RAM | Not specified |
| benchmark | HumanEval subset IDs 0 through 19 |
| correct | 18 |
| total | 20 |
| subset_ids | [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] |
| timeout_seconds | Not specified |
| retry_policy | 14 initial passes plus 4 successful timeout-only retries; not first-pass 18/20 |
| failed_ids | [2, 7] |
Local Spark Qwen writing draft; Codex source audit and rewrite. No site model benchmark.