THE SOURCE IN CONTEXT
The source runs full GLM-5.3 on four GB10 nodes with EXL3 and native MTP-3. Its two cache layouts trade reported aggregate code throughput for a larger shared window. Peak RAM and a task-accuracy suite are absent; the published quality panel measures quantization fidelity.
What must the deployment provide?
Four 128 GB GB10 nodes use TP4, direct dual-port 200G RoCE, TensorFold 0.6.0 plus the model branch, EXL3 average 3.04 bpw and native MTP-3. It is text-only and does not use DFlash. The approximately 69.86 GiB per-rank weight fit is shape-derived and excludes whole-run cache and working memory.
Source: bertholomus · pinned document
Which release produced the speed table?
October 4 release numbers are attributed to engine prefix 3eb35dd; the later head ddfad35 adds per-device RDMA completion matching and graph prewarm. The full executed commit and checkpoint revision are unreported. Replicated Q5 at 65536/stream reports 82 aggregate code tokens/s for four streams; shared DCP4 Q5 at a 1048576-token pool reports about 64.
Source: bertholomus · pinned document
How are HTTP accounting and fidelity limited?
The README calls the rates HTTP measurements with prompt time included. Its available serial streaming probe counts nonempty SSE chunks rather than verified tokenizer/usage tokens, while the exact concurrent release harness/raw accounting is not retained here. The fidelity panel uses BF16 activations and no cache quantization, with forward KL 0.109, reverse KL 0.123 and 90.2 percent top-1 agreement. It is not a task-accuracy test.
Source: bertholomus · pinned document bertholomus · pinned document bertholomus · pinned document
What does the long-window result establish?
The source finds a needle at 1039064 actual input tokens, reports 402 tokens/s at nominal 32K and about 76s first-token latency near 31K. Those do not demonstrate four full-million-token requests running together. Repetition counts, long-input peak RAM and raw per-request release accounting remain missing.
Source: bertholomus · pinned document bertholomus · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| replicated Q5 four code streams | aggregate output rate | 82 tokens/ssource-reported observation; dispersion unreported | Original row ↗ |
| DCP4 shared Q5 four code streams | aggregate output rate | 64 tokens/ssource-reported observation; dispersion unreported | Original row ↗ |
| 32K-labelled prefill | prefill rate | 402 tokens/ssource-reported observation; dispersion unreported | Original row ↗ |
| forward KL | KL reference to quant | 0.109 natssource-reported observation; dispersion unreported | Original row ↗ |
| reverse KL | KL quant to reference | 0.123 natssource-reported observation; dispersion unreported | Original row ↗ |
| top-1 agreement | top1 agreement | 90.2 percentsource-reported observation; dispersion unreported | Original row ↗ |
| quant perplexity | perplexity | 2.911 perplexitysource-reported observation; dispersion unreported | Original row ↗ |
| BF16 perplexity | perplexity | 2.75 perplexitysource-reported observation; dispersion unreported | Original row ↗ |
| served engine prompt-path KL | KL reference to engine | 0.111 natssource-reported observation; dispersion unreported | Original row ↗ |
replicated Q5 four code streams · aggregate output rate · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | Q5 MLA latent cache; FP8 indexer |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | 2026-10-04 |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | 4 |
| context_limit_tokens | 65536 |
| cache_state | replicated |
| metric_denominator | source reports HTTP prompt time included; release concurrent raw usage/harness not retained |
| token_accounting_verified | False |
| available_serial_stream_probe_counts_nonempty_SSE_chunks | True |
| release_was_computed_by_that_probe | Not specified |
| value_is_approximate | False |
DCP4 shared Q5 four code streams · aggregate output rate · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | Q5 MLA latent cache; FP8 indexer |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | 2026-10-04 |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | 4 |
| context_limit_tokens | 1048576 |
| cache_state | four streams share one DCP4 pool |
| metric_denominator | source reports HTTP prompt time included; release concurrent raw usage/harness not retained |
| token_accounting_verified | False |
| available_serial_stream_probe_counts_nonempty_SSE_chunks | True |
| release_was_computed_by_that_probe | Not specified |
| value_is_approximate | True |
32K-labelled prefill · prefill rate · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | Q5 MLA latent cache; FP8 indexer |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | 2026-10-04 |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | 1 |
| input_length_source_label | 32K |
| context_limit_tokens | 1048576 |
| cache_state | Not specified |
| metric_denominator | source HTTP prefill measurement, exact prompt count unreported |
forward KL · KL reference to quant · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | no cache quantization |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | Not specified |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | Not specified |
| quality_panel | wikitext-2 test, 32 x 2048 = 65536 positions |
| activation_dtype | BF16 |
| reference | BF16 full model |
| cache_state | Not specified |
| quality_is_task_accuracy | False |
| quality_arm | forward KL |
reverse KL · KL quant to reference · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | no cache quantization |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | Not specified |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | Not specified |
| quality_panel | wikitext-2 test, 32 x 2048 = 65536 positions |
| activation_dtype | BF16 |
| reference | BF16 full model |
| cache_state | Not specified |
| quality_is_task_accuracy | False |
| quality_arm | reverse KL |
top-1 agreement · top1 agreement · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | no cache quantization |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | Not specified |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | Not specified |
| quality_panel | wikitext-2 test, 32 x 2048 = 65536 positions |
| activation_dtype | BF16 |
| reference | BF16 full model |
| cache_state | Not specified |
| quality_is_task_accuracy | False |
| quality_arm | top-1 agreement |
quant perplexity · perplexity · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | no cache quantization |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | Not specified |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | Not specified |
| quality_panel | wikitext-2 test, 32 x 2048 = 65536 positions |
| activation_dtype | BF16 |
| reference | BF16 full model |
| cache_state | Not specified |
| quality_is_task_accuracy | False |
| quality_arm | quant perplexity |
BF16 perplexity · perplexity · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | no cache quantization |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | Not specified |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | Not specified |
| quality_panel | wikitext-2 test, 32 x 2048 = 65536 positions |
| activation_dtype | BF16 |
| reference | BF16 full model |
| cache_state | Not specified |
| quality_is_task_accuracy | False |
| quality_arm | BF16 perplexity |
served engine prompt-path KL · KL reference to engine · conditions
| model | GLM-5.3 (full 753B, not Flash) |
|---|---|
| model_artifact | bertholomus/GLM-5.3-EXL3-3.0bpw |
| model_revision | Not specified |
| engine | TensorFold 0.6.0 + glm-dsa-tp4 branch |
| runtime_execution_revision | Not specified |
| measured_runtime_short | 3eb35dd |
| later_branch_head_short | ddfad35 |
| executed_image_digest | Not specified |
| system | four DGX Spark GB10, 128 GB unified memory each, direct dual-port 200G RoCE |
| node_count | 4 |
| parallelism | TP4 |
| weight_precision | EXL3 3.04 bpw whole-model average; exllamav3 1.5.3 |
| kv_dtype | served path; panel-specific cache details not retained |
| draft | native MTP-3 |
| vision | False |
| DFlash | False |
| experiment_date | Not specified |
| temperature | 0 |
| thinking | Not specified |
| input_tokens | Not specified |
| output_tokens | Not specified |
| host_peak_RAM | Not specified |
| repetitions | Not specified |
| concurrency | Not specified |
| quality_panel | wikitext-2 test, 32 x 2048 = 65536 positions |
| activation_dtype | BF16 |
| reference | BF16 full model |
| cache_state | Not specified |
| quality_is_task_accuracy | False |
| quality_arm | served engine |
Local Spark Qwen draft; Codex source audit and rewrite. No site model benchmark.