# LLM Speed: TTFT, Decode and Served Rate TTFT tells you how long you wait for the first token. Decode rate describes output generation after that point. Served rate divides output tokens by the request's total elapsed time. Read all three with the workload and timing definition beside them: a fast stream can still follow a long wait. The examples below come from separate community experiments. Each explains a metric in its own configuration. RunAIHere has not independently reproduced these measurements. ![Conceptual timeline showing the three intervals used for TTFT, decode and served rate.](/assets/llm-speed-metrics-spark.webp) *Conceptual illustration generated as SVG by the local Spark text model and reviewed by RunAIHere. Not to scale; no benchmark values.* ## Read the clock before the headline Start by identifying when the author starts and stops the timer. Does the reported interval include queueing, prompt processing or a cold start? Was the server already warm? Does an output-rate calculation include the first-token wait? Those choices determine what a reader can infer from a tokens-per-second figure. For an interactive session, both the initial wait and subsequent delivery affect the experience. A batch-serving report also needs its concurrency and aggregation rules. Keep an aggregate throughput figure separate from a per-request or per-stream rate. ## TTFT: waiting before the first token One dated report used an Apple M4 Pro with 64 GB unified memory, MLX 0.31.0 and mlx-lm 0.30.7. For Qwen3.5-4B int4, the 4,096-token-input test reported mean TTFT of **6.14 seconds**, with standard deviation **0.25 seconds**, across 20 measurements after one warmup iteration. This describes that input shape and setup. The report's generation-rate test used a different input length, so keep those profiles distinct. [Original measurement row](https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P+4E+20GPU_64GB/2026-03-06__19:47:24/REPORT.md#L40) ## Decode: the generation phase A four-DGX-Spark report using TP4 and DSpark k5 reported **43.12 tokens/s** as mean per-stream decode at concurrency one, averaged across eight categories with counting excluded. Input and output shapes varied by category; the source reports one batch per cell. It concerns a DeepSeek V4.1 Flash launch checkpoint. The record does not establish performance for DeepSeek-V4.1-Flash-1017. [Original report and row](https://github.com/tonyd2wild/DeepSeek-V4.1-Flash-vLLM-DGX-Spark/blob/d45538f67366c62da668c35ec1afd4fcd0e4c8b4/README.md#L130) ## Served rate: the whole request window A different operator defines served rate as output tokens divided by whole-request wall-clock time, including first-token waiting. Its two-Spark DeepSeek-V4-Flash-0731 agentic test at approximately 33k context reported **22.5 tokens/s**, the median of three campaigns. That cell used concurrency one, 512 output tokens with ignore-eos enabled, prefix caching off and full-shape warmup. The reported spread ratio was **1.04**; this is a descriptive spread across those campaigns, not a confidence interval. [Original method and result](https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L70) The two DeepSeek examples use different checkpoints and hardware counts. Their values do not provide a controlled comparison between the machines or models. ## A checklist before choosing a deployment - Match the exact checkpoint, quantization, runtime and hardware topology. - Record input length, output length, concurrency and stopping rules. - Keep cache, warmup and speculative decoding settings with each result. - Retain the statistic and repetitions; distinguish standard deviation, observed spread and confidence intervals. - Check missing conditions in the original report before treating a nearby configuration as equivalent. The accompanying timeline is conceptual and has no measured scale. Its purpose is to show which interval each metric describes. The linked original reports remain the authority for their measurements. ## Original sources and conditions - weklund: https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md - Document revision: 5ae9b491f7a0539d5abef949eded6bd57785a981 - SHA-256: ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3 - tonyd2wild: https://github.com/tonyd2wild/DeepSeek-V4.1-Flash-vLLM-DGX-Spark/blob/d45538f67366c62da668c35ec1afd4fcd0e4c8b4/README.md - Document revision: d45538f67366c62da668c35ec1afd4fcd0e4c8b4 - SHA-256: 8c72dbb42a49a0f9b5b5f8a70055a467823993012bdb47cad748b5b0e04e36a7 - WeZZard: https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md - Document revision: 25e8dbf1b6113dd0b3f12d27a670fb284944d260 - SHA-256: b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469 Checked 2026-10-18. AI-assisted writing, source checked by Run AI Here. No independent model tests. Conditions and selected source facts are in the adjacent JSON export. ```json { "facts": [], "reported_results": [ { "label": "Qwen3.5-4B int4 first token", "metric": "ttft", "value": 6.14, "unit": "seconds", "statistic": "mean of 20 measurements", "uncertainty": { "standard_deviation": 0.25 }, "conditions": { "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory", "node_count": 1, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": "2026-03-06", "runtime": "MLX 0.31.0 / mlx-lm 0.30.7", "repetitions": 20, "warmup_iterations": 1, "concurrency": null, "model_revision": null, "cache_state": null, "model": "Qwen3.5-4B int4", "quantization": "int4", "input_tokens": 4096, "output_tokens": null }, "evidence_state": "source_reported", "site_reproduced": false, "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P+4E+20GPU_64GB/2026-03-06__19:47:24/REPORT.md#L40", "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3", "source_record_id": "qwen35-m4pro-20260306" }, { "label": "Eight-category mean, C1", "metric": "per_stream_decode", "value": 43.12, "unit": "tokens/s", "statistic": "mean across eight categories; counting excluded", "uncertainty": null, "conditions": { "system": "4 x NVIDIA DGX Spark", "node_count": 4, "parallelism": "TP4", "kv_dtype": null, "context_limit_tokens": 300000, "experiment_date": null, "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017", "speculative_decoding": "DSpark k5", "temperature": 0, "thinking": false, "warmup": "performed", "repetitions": "one batch per cell", "input_tokens": "about 30–120; summary prompt longer", "output_token_budget": "150–256; natural stopping", "weight_precision": "MXFP4 experts / MXFP8 dense", "concurrency": 1, "denominator": "after first token" }, "evidence_state": "source_reported", "site_reproduced": false, "evidence_url": "https://github.com/tonyd2wild/DeepSeek-V4.1-Flash-vLLM-DGX-Spark/blob/d45538f67366c62da668c35ec1afd4fcd0e4c8b4/README.md#L130", "source_document_sha256": "8c72dbb42a49a0f9b5b5f8a70055a467823993012bdb47cad748b5b0e04e36a7", "source_record_id": "deepseek-v41-four-spark-boot10" }, { "label": "DeepSeek V4 Flash 0731, long agentic", "metric": "served_rate", "value": 22.5, "unit": "tokens/s", "statistic": "median of three campaigns", "uncertainty": { "spread_ratio": 1.04 }, "conditions": { "system": "2 x DGX Spark; 121 GiB unified memory each; 200 GbE", "node_count": 2, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": "2026-09-05", "output_tokens": 512, "ignore_eos": true, "prefix_cache": false, "warmup": "all workload/user shapes, full generation", "repetitions": 3, "concurrency": 1, "measurement_denominator": "whole request wall clock including TTFT", "workload": "agentic 33k context", "model": "DeepSeek-V4-Flash-0731", "kv": "fp8_ds_mla", "kv_pool_tokens": 517484, "draft": "DSpark k5 probabilistic" }, "evidence_state": "source_reported", "site_reproduced": false, "evidence_url": "https://github.com/WeZZard/dgx-spark-bench/blob/25e8dbf1b6113dd0b3f12d27a670fb284944d260/README.md#L70", "source_document_sha256": "b815d32807147f9638b1abbeadc02af257c55bd1f7651a123e5a8090bebe7469", "source_record_id": "spark-pair-moe-served-rate" } ], "conditions": { "cross_source_ranking_allowed": false, "one_shared_experiment": false, "independent_site_runs": 0 } } ```