THE SOURCE IN CONTEXT
Four submitted requests do not guarantee four running requests. The sources contain completed long-context batches, but their cache states, timing denominators and memory windows differ. Keep the four-GPU campaign peak separate from per-cell latency, and retain the original scheduler failure.
Which long-context batches were measured?
Penny Royal v2.5.0 tests one RTX PRO 6000 at TP1 with RadixArk NVFP4, RAM PLE, FP8 KV, FR-Spec and an 824384-token pool. Selected 128K and 490K C4 results divide all output by synchronized group wall time. Each cell is one batch, with stop or a 4096-token cap and differing completion totals. The online-FP8 arm records actual four-way overlap; exact prefix sharing and full checkpoint revision remain unreported here.
Source: jpezzulli · pinned document
What does the newer release add?
October 10 Penny Royal 3.0 has separate short regular screens and actual 63921/489915-token cold prompts. It uses another checkpoint, an adaptive policy and a larger token pool. The regular rate is an elapsed-weighted native active-interval mean, not whole task wall time. Public accepted source and local diagnostic checkout are separately identified. These cells do not replace the older long C4 experiment.
Source: jpezzulli · pinned document jpezzulli · pinned document
How are memory and sustained-load evidence bounded?
The external four-GPU report supplies a sampled summed used-VRAM peak for its entire matrix, not a per-cell peak or host peak. Penny Royal post-graph free VRAM and minimum free VRAM in a GPU-media window are different observations. A separate credited field report submits four approximately 240K requests over ten rounds, with queuing and 57 percent peak pool occupancy; HTTP 200 alone is not retrieval correctness.
Source: jpezzulli · pinned document jpezzulli · pinned document jpezzulli · pinned document
Which failure remains useful?
A first-person SGLang incident has approximately 140K cold input, HiCache write-through, 51 Mamba slots and only two running requests when Mamba usage reaches 1.00. The scheduler asserts while full token usage is 0.46. PR37619 is still open at this check; its CPU tests and proposed fallback do not establish that the GPU reproducer or all affected workloads are fixed.
Source: jpezzulli · pinned document
Reported figures and their conditions
Selected facts reported by the original author. Each row has its own configuration and measurement boundary. Run AI Here has not reproduced these tests.
| Configuration / workload | Metric | Reported value | Evidence |
|---|---|---|---|
| 128K C4 option off | aggregate output rate | 367.127 tokens/sone reported observation; no dispersion | Original row ↗ |
| 128K C4 online FP8 | aggregate output rate | 422.037 tokens/sone reported observation; no dispersion | Original row ↗ |
| 490K C4 option off | aggregate output rate | 236.678 tokens/sone reported observation; no dispersion | Original row ↗ |
| 490K C4 online FP8 | aggregate output rate | 329.01 tokens/sone reported observation; no dispersion | Original row ↗ |
| 128K online FP8 C1 decode_rate | decode rate | 195.634 tokens/sone reported observation; no dispersion | Original row ↗ |
| 128K online FP8 C1 ttft | ttft | 12.906 sone reported observation; no dispersion | Original row ↗ |
| 490K online FP8 C1 decode_rate | decode rate | 172.644 tokens/sone reported observation; no dispersion | Original row ↗ |
| 490K online FP8 C1 ttft | ttft | 63.296 sone reported observation; no dispersion | Original row ↗ |
| Post-graph free VRAM | available GPU memory | 7.52 GiBone reported observation; no dispersion | Original row ↗ |
| Media-window minimum free VRAM | available GPU memory | 1897 MiBone reported observation; no dispersion | Original row ↗ |
| 3.0 cold 64K prefill_rate | prefill rate | 17235.9 tokens/sone reported observation; no dispersion | Original row ↗ |
| 3.0 cold 64K ttft | ttft | 4.71 sone reported observation; no dispersion | Original row ↗ |
| 3.0 cold 490K prefill_rate | prefill rate | 8839.75 tokens/sone reported observation; no dispersion | Original row ↗ |
| 3.0 cold 490K ttft | ttft | 60.37 sone reported observation; no dispersion | Original row ↗ |
| 3.0 short C1 native mean | native decode rate | 218.63 tokens/snative mean; three regular repeats, no SD | Original row ↗ |
| 3.0 short C4 native mean | native decode rate | 489.35 tokens/snative mean; three regular repeats, no SD | Original row ↗ |
128K C4 option off · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 128K four-request three-key retrieval |
| input_length_source_label | 128K |
| concurrency | 4 |
| precision_option | option off |
| actual_output_tokens_group | 15725 |
| group_wall_seconds | 42.833 |
| requested_output_cap_per_request | 4096 |
| finish_reasons | ["stop", "length"] |
| all_four_overlap_seconds | Not specified |
| metric_denominator | all group output tokens / synchronized group makespan including TTFT |
| all_responses_three_keys_exact | True |
| all_four_full_distinct_prefixes_proven | False |
128K C4 online FP8 · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 128K four-request three-key retrieval |
| input_length_source_label | 128K |
| concurrency | 4 |
| precision_option | online FP8 |
| actual_output_tokens_group | 15898 |
| group_wall_seconds | 37.67 |
| requested_output_cap_per_request | 4096 |
| finish_reasons | ["stop", "length"] |
| all_four_overlap_seconds | 30.152 |
| metric_denominator | all group output tokens / synchronized group makespan including TTFT |
| all_responses_three_keys_exact | True |
| all_four_full_distinct_prefixes_proven | False |
490K C4 option off · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 490K four-request three-key retrieval |
| input_length_source_label | 490K |
| concurrency | 4 |
| precision_option | option off |
| actual_output_tokens_group | 15455 |
| group_wall_seconds | 65.3 |
| requested_output_cap_per_request | 4096 |
| finish_reasons | ["stop", "length"] |
| all_four_overlap_seconds | Not specified |
| metric_denominator | all group output tokens / synchronized group makespan including TTFT |
| all_responses_three_keys_exact | True |
| all_four_full_distinct_prefixes_proven | False |
490K C4 online FP8 · aggregate output rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 490K four-request three-key retrieval |
| input_length_source_label | 490K |
| concurrency | 4 |
| precision_option | online FP8 |
| actual_output_tokens_group | 15127 |
| group_wall_seconds | 45.977 |
| requested_output_cap_per_request | 4096 |
| finish_reasons | ["stop", "length"] |
| all_four_overlap_seconds | 24.85 |
| metric_denominator | all group output tokens / synchronized group makespan including TTFT |
| all_responses_three_keys_exact | True |
| all_four_full_distinct_prefixes_proven | False |
128K online FP8 C1 decode_rate · decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | 1024 |
| actual_output_tokens | 1024 |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 128K C1 three-key retrieval |
| concurrency | 1 |
| precision_option | online FP8 |
| input_length_source_label | 128K |
| metric_denominator | client generation after first token, excluding TTFT |
128K online FP8 C1 ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | 1024 |
| actual_output_tokens | 1024 |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 128K C1 three-key retrieval |
| concurrency | 1 |
| precision_option | online FP8 |
| input_length_source_label | 128K |
| metric_denominator | client generation after first token, excluding TTFT |
490K online FP8 C1 decode_rate · decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | 1024 |
| actual_output_tokens | 1024 |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 490K C1 three-key retrieval |
| concurrency | 1 |
| precision_option | online FP8 |
| input_length_source_label | 490K |
| metric_denominator | client generation after first token, excluding TTFT |
490K online FP8 C1 ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | 1024 |
| actual_output_tokens | 1024 |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| workload | 490K C1 three-key retrieval |
| concurrency | 1 |
| precision_option | online FP8 |
| input_length_source_label | 490K |
| metric_denominator | client generation after first token, excluding TTFT |
Post-graph free VRAM · available GPU memory · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| concurrency | Not specified |
| memory_kind | post_graph_available_not_peak_used |
| memory_cohort | startup |
| paired_long_text_batch_peak | False |
Media-window minimum free VRAM · available GPU memory · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | RadixArk/Qwen3.8-Flash-Next-NVFP4 |
| model_revision | Not specified |
| runtime_execution_revision | Not specified |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 824384 |
| mamba_slots | 24 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-09-11 |
| recipe_release | v2.5.0 |
| input_tokens | Not specified |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | post-warmup context tests; exact C4 cache sharing unreported |
| concurrency | Not specified |
| memory_kind | sampled_minimum_free_not_per_long_text_batch_peak |
| memory_cohort | ten selected GPU-media checks; ordinary/large JPEG, concurrent images and video history |
| paired_long_text_batch_peak | False |
3.0 cold 64K prefill_rate · prefill rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | OrcaRouter ModelOpt NVFP4 checkpoint; exact revision unreported here |
| model_revision | Not specified |
| runtime_execution_revision | 90b2b412e2bfd06f0ef3be1b41fbfbdee45bc14f |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 1067072 |
| mamba_slots | 36 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-10-10 |
| recipe_release | 3.0 |
| input_tokens | 63921 |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | 100% uncached prompt |
| accepted_public_core | b48b70f3f80494d78a778fd4d26958ede343075f |
| local_runtime_has_private_diagnostics | True |
| media_preprocessing | A4000 sidecar; not an inference TP rank |
| concurrency | 1 |
| metric_denominator | native mean input rate |
3.0 cold 64K ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | OrcaRouter ModelOpt NVFP4 checkpoint; exact revision unreported here |
| model_revision | Not specified |
| runtime_execution_revision | 90b2b412e2bfd06f0ef3be1b41fbfbdee45bc14f |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 1067072 |
| mamba_slots | 36 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-10-10 |
| recipe_release | 3.0 |
| input_tokens | 63921 |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | 100% uncached prompt |
| accepted_public_core | b48b70f3f80494d78a778fd4d26958ede343075f |
| local_runtime_has_private_diagnostics | True |
| media_preprocessing | A4000 sidecar; not an inference TP rank |
| concurrency | 1 |
| metric_denominator | client first-token latency |
3.0 cold 490K prefill_rate · prefill rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | OrcaRouter ModelOpt NVFP4 checkpoint; exact revision unreported here |
| model_revision | Not specified |
| runtime_execution_revision | 90b2b412e2bfd06f0ef3be1b41fbfbdee45bc14f |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 1067072 |
| mamba_slots | 36 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-10-10 |
| recipe_release | 3.0 |
| input_tokens | 489915 |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | 100% uncached prompt |
| accepted_public_core | b48b70f3f80494d78a778fd4d26958ede343075f |
| local_runtime_has_private_diagnostics | True |
| media_preprocessing | A4000 sidecar; not an inference TP rank |
| concurrency | 1 |
| metric_denominator | native mean input rate |
3.0 cold 490K ttft · ttft · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | OrcaRouter ModelOpt NVFP4 checkpoint; exact revision unreported here |
| model_revision | Not specified |
| runtime_execution_revision | 90b2b412e2bfd06f0ef3be1b41fbfbdee45bc14f |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 1067072 |
| mamba_slots | 36 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | Not specified |
| top_p | Not specified |
| thinking | Not specified |
| repetitions | 1 |
| experiment_date | 2026-10-10 |
| recipe_release | 3.0 |
| input_tokens | 489915 |
| output_tokens | Not specified |
| actual_output_tokens | Not specified |
| cache_state | 100% uncached prompt |
| accepted_public_core | b48b70f3f80494d78a778fd4d26958ede343075f |
| local_runtime_has_private_diagnostics | True |
| media_preprocessing | A4000 sidecar; not an inference TP rank |
| concurrency | 1 |
| metric_denominator | client first-token latency |
3.0 short C1 native mean · native decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | OrcaRouter ModelOpt NVFP4 checkpoint; exact revision unreported here |
| model_revision | Not specified |
| runtime_execution_revision | 90b2b412e2bfd06f0ef3be1b41fbfbdee45bc14f |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 1067072 |
| mamba_slots | 36 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | 0.8 |
| top_p | 0.95 |
| thinking | xhigh |
| repetitions | 3 |
| experiment_date | 2026-10-10 |
| recipe_release | 3.0 |
| input_tokens | 128 |
| output_tokens | 1024 |
| actual_output_tokens | 1024 |
| cache_state | regular warmed screens |
| accepted_public_core | b48b70f3f80494d78a778fd4d26958ede343075f |
| local_runtime_has_private_diagnostics | True |
| media_preprocessing | A4000 sidecar; not an inference TP rank |
| concurrency | 1 |
| top_k | 40 |
| min_p | 0.05 |
| metric_denominator | elapsed-weighted complete active native log intervals; excludes idle/prefill crossing and partial tail; aggregate if C4 |
3.0 short C4 native mean · native decode rate · conditions
| model | Qwen3.8-Flash-Next |
|---|---|
| model_artifact | OrcaRouter ModelOpt NVFP4 checkpoint; exact revision unreported here |
| model_revision | Not specified |
| runtime_execution_revision | 90b2b412e2bfd06f0ef3be1b41fbfbdee45bc14f |
| executed_image_digest | Not specified |
| engine | SGLang-derived Penny Royal |
| system | one RTX PRO 6000 Blackwell SM120, 96 GB; TP1 |
| node_count | 1 |
| parallelism | TP1 |
| weight_precision | ModelOpt NVFP4 experts; optional online FP8 projections |
| kv_dtype | FP8 |
| PLE | host RAM |
| context_limit_tokens | 524288 |
| KV_pool_tokens | 1067072 |
| mamba_slots | 36 |
| HiCache_NIXL_GiB | 32 |
| FR_Spec | True |
| temperature | 0.8 |
| top_p | 0.95 |
| thinking | xhigh |
| repetitions | 3 |
| experiment_date | 2026-10-10 |
| recipe_release | 3.0 |
| input_tokens | 128 |
| output_tokens | 1024 |
| actual_output_tokens | 1024 |
| cache_state | regular warmed screens |
| accepted_public_core | b48b70f3f80494d78a778fd4d26958ede343075f |
| local_runtime_has_private_diagnostics | True |
| media_preprocessing | A4000 sidecar; not an inference TP rank |
| concurrency | 4 |
| top_k | 40 |
| min_p | 0.05 |
| metric_denominator | elapsed-weighted complete active native log intervals; excludes idle/prefill crossing and partial tail; aggregate if C4 |
Local Spark Qwen writing draft; Codex source audit and rewrite. No site model benchmark.