LLM Speed: TTFT, Decode and Served Rate

TTFT tells you how long you wait for the first token. Decode rate describes output generation after that point. Served rate divides output tokens by the request's total elapsed time. Read all three with the workload and timing definition beside them: a fast stream can still follow a long wait.

The examples below come from separate community experiments. Each explains a metric in its own configuration. RunAIHere has not independently reproduced these measurements.

Conceptual illustration generated as SVG by the local Spark text model and reviewed by RunAIHere. Not to scale; no benchmark values.

Read the clock before the headline

Start by identifying when the author starts and stops the timer. Does the reported interval include queueing, prompt processing or a cold start? Was the server already warm? Does an output-rate calculation include the first-token wait? Those choices determine what a reader can infer from a tokens-per-second figure.

For an interactive session, both the initial wait and subsequent delivery affect the experience. A batch-serving report also needs its concurrency and aggregation rules. Keep an aggregate throughput figure separate from a per-request or per-stream rate.

TTFT: waiting before the first token

One dated report used an Apple M4 Pro with 64 GB unified memory, MLX 0.31.0 and mlx-lm 0.30.7. For Qwen3.5-4B int4, the 4,096-token-input test reported mean TTFT of 6.14 seconds, with standard deviation 0.25 seconds, across 20 measurements after one warmup iteration. This describes that input shape and setup. The report's generation-rate test used a different input length, so keep those profiles distinct. Original measurement row

Decode: the generation phase

A four-DGX-Spark report using TP4 and DSpark k5 reported 43.12 tokens/s as mean per-stream decode at concurrency one, averaged across eight categories with counting excluded. Input and output shapes varied by category; the source reports one batch per cell. It concerns a DeepSeek V4.1 Flash launch checkpoint. The record does not establish performance for DeepSeek-V4.1-Flash-1017. Original report and row

Served rate: the whole request window

A different operator defines served rate as output tokens divided by whole-request wall-clock time, including first-token waiting. Its two-Spark DeepSeek-V4-Flash-0731 agentic test at approximately 33k context reported 22.5 tokens/s, the median of three campaigns. That cell used concurrency one, 512 output tokens with ignore-eos enabled, prefix caching off and full-shape warmup. The reported spread ratio was 1.04; this is a descriptive spread across those campaigns, not a confidence interval. Original method and result

The two DeepSeek examples use different checkpoints and hardware counts. Their values do not provide a controlled comparison between the machines or models.

A checklist before choosing a deployment

• Match the exact checkpoint, quantization, runtime and hardware topology.
• Record input length, output length, concurrency and stopping rules.
• Keep cache, warmup and speculative decoding settings with each result.
• Retain the statistic and repetitions; distinguish standard deviation, observed spread and confidence intervals.
• Check missing conditions in the original report before treating a nearby configuration as equivalent.

The accompanying timeline is conceptual and has no measured scale. Its purpose is to show which interval each metric describes. The linked original reports remain the authority for their measurements.