# DeepSeek V4.1: eight-5090 placement campaign

Paired serving-optimization report on eight RTX 5090 D cards. Interactive, wide-batch and held-out text sections have different workloads.

- Type: Performance report
- Evidence: source_reported; site reproduced: false
- Model family: DeepSeek
- Model: DeepSeek-V4.1-Flash (base checkpoint)
- Original author: devin-lai
- Original project: https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel
- Pinned document: https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md
- Source revision: 4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8
- Source commit time (not experiment date): 2026-09-16T07:56:48Z
- Checked: 2026-10-17
- Experiment date: 2026-09-16
- Runtime topics: vLLM
- Hardware topics: nvidia
- Reported setup: {"system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only", "node_count": 1, "parallelism": "TP8 + expert parallelism", "kv_dtype": null, "context_limit_tokens": null, "experiment_date": "2026-09-16"}
- Performance figures: attributed selections with per-point conditions; no cross-source ranking

## Conditions

Unknown is null. Recipe options are not measured benchmark settings.

```json
{
  "conditions": {
    "model_revision": null,
    "weight_hash": null,
    "runtime_execution_revision": null,
    "input_tokens": null,
    "output_tokens": null,
    "concurrency": null,
    "cache_state": null,
    "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
    "speculative_decoding": null,
    "temperature": null,
    "top_p": null
  },
  "test_profiles": [
    {
      "label": "Paired interactive placement",
      "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md",
      "conditions": {
        "model_revision": null,
        "weight_hash": null,
        "runtime_execution_revision": null,
        "input_tokens": null,
        "output_tokens": null,
        "concurrency": [
          1,
          8
        ],
        "cache_state": null,
        "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
        "speculative_decoding": null,
        "temperature": null,
        "top_p": null,
        "hardware": "8 x RTX 5090 D; 503 GiB RAM",
        "repetitions": 3
      }
    },
    {
      "label": "Wide-batch synthetic workload",
      "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md",
      "conditions": {
        "model_revision": null,
        "weight_hash": null,
        "runtime_execution_revision": null,
        "input_tokens": 1024,
        "output_tokens": 128,
        "concurrency": null,
        "cache_state": null,
        "quantization": null,
        "speculative_decoding": null,
        "temperature": null,
        "top_p": null,
        "repetitions": 1
      }
    },
    {
      "label": "Held-out text workload",
      "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md",
      "conditions": {
        "model_revision": null,
        "weight_hash": null,
        "runtime_execution_revision": null,
        "input_tokens": null,
        "output_tokens": 256,
        "concurrency": null,
        "cache_state": null,
        "quantization": null,
        "speculative_decoding": null,
        "temperature": null,
        "top_p": null,
        "repetitions": 1
      }
    }
  ],
  "recipe_parameters": null,
  "deployment_options": []
}
```

## Reading notes

- The comparison baseline is already patched vLLM, not stock upstream.
- MXFP4 experts, FP8 dense weights and BF16 expert activations are retained.

## Limits

- Interactive three-trial comparisons and single-trial wide batches have different evidence strength.
- These are base-checkpoint results, not 1017 results.

## License declaration

{"declared_spdx": "MIT", "scope": "Repository metadata only; per-file, model, data and conversion terms remain separate."}


## Editorial answer

The paired experiment runs on eight 32 GB RTX 5090 D GPUs, two Xeon Gold 6530 sockets and 503 GiB host memory over PCIe. Both arms already use custom vLLM patches and CUDA graphs. The placed arm moves the same offload budget onto rarely selected expert pages and enables exact staging; it is not a stock-versus-optimized upstream comparison.

By Run AI Here editorial · 2026-10-17

AI-assisted source reading and original editorial synthesis; no independent model execution.

## What is being compared?

The paired experiment runs on eight 32 GB RTX 5090 D GPUs, two Xeon Gold 6530 sockets and 503 GiB host memory over PCIe. Both arms already use custom vLLM patches and CUDA graphs. The placed arm moves the same offload budget onto rarely selected expert pages and enables exact staging; it is not a stock-versus-optimized upstream comparison.

Sources: https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L20

## What improves, and what regresses?

Across three trials, interactive C1 output rises from 75.37 to 97.23 tokens/s, while C8 rises from 166.55 to 247.77. The same placement reduces the random-token C1 result from 67.46 to 63.00. The negative result is central: calibration-dependent expert placement is useful when traffic resembles its routing profile and can be harmful when it does not.

Sources: https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L42

## Why is the largest number not interactive speed?

The 368.55-token/s held-out-text result is aggregate output at 128 concurrent requests, with 1,000-token inputs and 256-token outputs in a single trial. The corresponding latency and queueing behavior differ from C1. The synthetic 337.45-token/s result at 256 users is another workload and another memory budget. Do not collapse them into one headline for a single person.

Sources: https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L147

## How strong is the evidence?

The paired interactive comparison has three trials, while placement-only screens and wide-batch campaigns have one. The corpus and prompt shape differ across sections. The report measures the launch checkpoint, not the 1017 refresh, and the collection does not fill in missing prompt lengths or weight hashes from a different source.

Sources: https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L28

## Selected source-reported figures

Each point retains its own configuration, measurement unit and source. No site reproduction or cross-source ranking.

```json
[
  {
    "label": "Interactive C1, patched baseline",
    "metric": "output_throughput",
    "value": 75.37,
    "unit": "tokens/s",
    "statistic": "reported value",
    "uncertainty": null,
    "conditions": {
      "system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only",
      "node_count": 1,
      "parallelism": "TP8 + expert parallelism",
      "kv_dtype": null,
      "context_limit_tokens": null,
      "experiment_date": "2026-09-16",
      "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017",
      "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
      "runtime": "already patched graph-enabled vLLM",
      "calibration": "shipped mixed real-text profile",
      "concurrency": 1,
      "repetitions": 3,
      "placement": "layer offload"
    },
    "evidence_state": "source_reported",
    "site_reproduced": false,
    "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L35",
    "source_document_sha256": "b2d519cd355ea9c77692eab01e960ca98234e38fcf552589dacd5edb21475078"
  },
  {
    "label": "Interactive C1, expert placement + staging",
    "metric": "output_throughput",
    "value": 97.23,
    "unit": "tokens/s",
    "statistic": "reported value",
    "uncertainty": null,
    "conditions": {
      "system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only",
      "node_count": 1,
      "parallelism": "TP8 + expert parallelism",
      "kv_dtype": null,
      "context_limit_tokens": null,
      "experiment_date": "2026-09-16",
      "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017",
      "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
      "runtime": "already patched graph-enabled vLLM",
      "calibration": "shipped mixed real-text profile",
      "concurrency": 1,
      "repetitions": 3,
      "placement": "cold expert pages + exact staging"
    },
    "evidence_state": "source_reported",
    "site_reproduced": false,
    "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L35",
    "source_document_sha256": "b2d519cd355ea9c77692eab01e960ca98234e38fcf552589dacd5edb21475078"
  },
  {
    "label": "Interactive C8, expert placement + staging",
    "metric": "output_throughput",
    "value": 247.77,
    "unit": "tokens/s",
    "statistic": "reported value",
    "uncertainty": null,
    "conditions": {
      "system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only",
      "node_count": 1,
      "parallelism": "TP8 + expert parallelism",
      "kv_dtype": null,
      "context_limit_tokens": null,
      "experiment_date": "2026-09-16",
      "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017",
      "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
      "runtime": "already patched graph-enabled vLLM",
      "calibration": "shipped mixed real-text profile",
      "concurrency": 8,
      "repetitions": 3,
      "placement": "cold expert pages + exact staging"
    },
    "evidence_state": "source_reported",
    "site_reproduced": false,
    "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L36",
    "source_document_sha256": "b2d519cd355ea9c77692eab01e960ca98234e38fcf552589dacd5edb21475078"
  },
  {
    "label": "Random-token C1, baseline",
    "metric": "output_throughput",
    "value": 67.46,
    "unit": "tokens/s",
    "statistic": "reported value",
    "uncertainty": null,
    "conditions": {
      "system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only",
      "node_count": 1,
      "parallelism": "TP8 + expert parallelism",
      "kv_dtype": null,
      "context_limit_tokens": null,
      "experiment_date": "2026-09-16",
      "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017",
      "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
      "runtime": "already patched graph-enabled vLLM",
      "calibration": "shipped mixed real-text profile",
      "concurrency": 1,
      "repetitions": 3,
      "workload": "uniform random token IDs"
    },
    "evidence_state": "source_reported",
    "site_reproduced": false,
    "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L42",
    "source_document_sha256": "b2d519cd355ea9c77692eab01e960ca98234e38fcf552589dacd5edb21475078"
  },
  {
    "label": "Random-token C1, placed",
    "metric": "output_throughput",
    "value": 63.0,
    "unit": "tokens/s",
    "statistic": "reported value",
    "uncertainty": null,
    "conditions": {
      "system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only",
      "node_count": 1,
      "parallelism": "TP8 + expert parallelism",
      "kv_dtype": null,
      "context_limit_tokens": null,
      "experiment_date": "2026-09-16",
      "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017",
      "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
      "runtime": "already patched graph-enabled vLLM",
      "calibration": "shipped mixed real-text profile",
      "concurrency": 1,
      "repetitions": 3,
      "workload": "uniform random token IDs",
      "placement": "cold expert pages + staging"
    },
    "evidence_state": "source_reported",
    "site_reproduced": false,
    "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L42",
    "source_document_sha256": "b2d519cd355ea9c77692eab01e960ca98234e38fcf552589dacd5edb21475078"
  },
  {
    "label": "Held-out text C128, 128-sequence preset",
    "metric": "aggregate_output",
    "value": 368.55,
    "unit": "tokens/s",
    "statistic": "reported value",
    "uncertainty": null,
    "conditions": {
      "system": "8 x NVIDIA RTX 5090 D (32 GB each); 503 GiB host RAM; PCIe only",
      "node_count": 1,
      "parallelism": "TP8 + expert parallelism",
      "kv_dtype": null,
      "context_limit_tokens": null,
      "experiment_date": "2026-09-16",
      "model": "DeepSeek V4.1 Flash launch checkpoint, not 1017",
      "quantization": "MXFP4 experts / FP8 dense / BF16 expert activations",
      "runtime": "already patched graph-enabled vLLM",
      "calibration": "shipped mixed real-text profile",
      "concurrency": 128,
      "input_tokens": 1000,
      "output_tokens": 256,
      "repetitions": 1,
      "max_num_seqs": 128
    },
    "evidence_state": "source_reported",
    "site_reproduced": false,
    "evidence_url": "https://github.com/devin-lai/DeepSeek-V4.1-Flash-Accel/blob/4a4e88ec2cdcf2fc7c802e1f6b7c943d897770c8/benchmarks/results/2026-09-16-expert-placement-and-wide-batches.md#L158",
    "source_document_sha256": "b2d519cd355ea9c77692eab01e960ca98234e38fcf552589dacd5edb21475078"
  }
]
```