# Kimi-K3: vLLM recipe

Kimi K3 deployment recipe with separate MXFP4, NVFP4 and other artifact paths and parallelism strategies.

- Type: Deployment recipe
- Evidence: source_reported; site reproduced: false
- Model family: Kimi
- Model: moonshotai/Kimi-K3
- Original author: vllm-project
- Original project: https://github.com/vllm-project/recipes
- Pinned document: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml
- Source revision: d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94
- Source commit time (not experiment date): 2026-10-17T09:22:45Z
- Checked: 2026-10-17
- Experiment date: None
- Runtime topics: vLLM
- Hardware topics: amd, ascend, nvidia
- Reported setup: {"system": null, "node_count": null, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": null}
- Performance figures: attributed selections with per-point conditions; no cross-source ranking

## Conditions

Unknown is null. Recipe options are not measured benchmark settings.

```json
{
  "conditions": {
    "model_revision": null,
    "weight_hash": null,
    "runtime_execution_revision": null,
    "input_tokens": null,
    "output_tokens": null,
    "concurrency": null,
    "cache_state": null,
    "quantization": null,
    "speculative_decoding": null,
    "temperature": null,
    "top_p": null
  },
  "test_profiles": [],
  "recipe_parameters": {
    "minimum_runtime_version": "0.29.0",
    "nightly_required": false,
    "model_context_limit_tokens": 1048576,
    "compatible_strategies": [
      "single_node_tp",
      "multi_node_tp",
      "multi_node_tep",
      "multi_node_tp_pp",
      "multi_node_dep",
      "pd_cluster"
    ],
    "hardware_claims": [
      {
        "profile": "h200",
        "maintainer_status": "verified"
      },
      {
        "profile": "b200",
        "maintainer_status": "verified"
      },
      {
        "profile": "b300",
        "maintainer_status": "verified"
      },
      {
        "profile": "gb200",
        "maintainer_status": "verified"
      },
      {
        "profile": "gb300",
        "maintainer_status": "verified"
      },
      {
        "profile": "mi355x",
        "maintainer_status": "verified"
      },
      {
        "profile": "ascend_910c",
        "maintainer_status": "verified"
      }
    ],
    "recipe_updated_on": "2026-10-16",
    "effective_default_minimum_runtime_version": "0.29.0"
  },
  "deployment_options": [
    {
      "variant": "default",
      "model_id": "moonshotai/Kimi-K3",
      "precision": "mxfp4",
      "supported_hardware": [
        "h100",
        "h200",
        "b200",
        "b300",
        "gb200",
        "gb300",
        "mi300x",
        "mi325x",
        "mi355x"
      ],
      "model_revision": null,
      "declared_planning_minimum_gb": 1680,
      "minimum_runtime_version": "0.29.0",
      "extra_args": [],
      "tensor_parallel_by_hardware": null
    },
    {
      "variant": "nvfp4",
      "model_id": "RedHatAI/Kimi-K3-NVFP4",
      "precision": "nvfp4",
      "supported_hardware": [
        "b200",
        "b300",
        "gb200",
        "gb300"
      ],
      "model_revision": null,
      "declared_planning_minimum_gb": 1650,
      "minimum_runtime_version": "0.29.0",
      "extra_args": [],
      "tensor_parallel_by_hardware": null
    },
    {
      "variant": "ascend_w4a8",
      "model_id": "Eco-Tech/Kimi-K3-w4a8",
      "precision": "int4",
      "supported_hardware": [
        "ascend_910c"
      ],
      "model_revision": null,
      "declared_planning_minimum_gb": 1711,
      "minimum_runtime_version": "0.29.0",
      "extra_args": [
        "--quantization",
        "ascend",
        "--enable-prefix-caching",
        "--enable-expert-parallel",
        "--max-num-seqs",
        "16",
        "--max-model-len",
        "131072",
        "--max-num-batched-tokens",
        "24576",
        "--gpu-memory-utilization",
        "0.9",
        "--compilation-config",
        "{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}",
        "--mm-processor-cache-gb",
        "0",
        "--additional-config",
        "{\"enable_cpu_binding\":true, \"enable_flashcomm1\":true}",
        "--mm-encoder-tp-mode",
        "data",
        "--limit-mm-per-prompt",
        "{\"vision_chunk\": 2}",
        "--allowed-local-media-path",
        "/"
      ],
      "tensor_parallel_by_hardware": null
    },
    {
      "variant": "fp8",
      "model_id": "RedHatAI/Kimi-K3-FP8-BLOCK",
      "precision": "fp8",
      "supported_hardware": null,
      "model_revision": null,
      "declared_planning_minimum_gb": 3360,
      "minimum_runtime_version": "0.29.0",
      "extra_args": [],
      "tensor_parallel_by_hardware": null
    }
  ]
}
```

## Reading notes

- Recipe minimum engine version: 0.29.0.
- Default artifact: moonshotai/Kimi-K3.
- Variant and hardware settings are source declarations; select the matching original profile.

## Limits

- Do not merge measurements made with different quantizations or parallelism.
- Input length, output policy, concurrency and cache are workload-specific; no single benchmark is inferred from the recipe.

## License declaration

{"declared_spdx": "Apache-2.0", "scope": "Recipe repository only; each model and conversion has independent distribution terms."}


## Editorial answer

The pinned Kimi-K3 recipe defaults to moonshotai/Kimi-K3 (mxfp4). Its effective default-variant minimum is vLLM 0.29.0. Select the exact variant, hardware override and topology before treating it as a launch plan. No deployment was run by this site.

By Run AI Here editorial · 2026-10-17

AI-assisted source reading and original editorial synthesis; no independent model execution.

## Which artifact and engine does this recipe select?

The recipe identifies moonshotai/Kimi-K3 as its model family entry and selects moonshotai/Kimi-K3 by default. The default precision label is mxfp4; model-wide and variant-specific runtime requirements are kept separately below. The document revision fixes what the recipe said, while the weight revision and executed runtime revision remain separate fields. A model-family title alone cannot establish either artifact identity.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132

## What needs special attention for this model?

The default MXFP4 footprint is explicitly described as estimated in the pinned recipe. Do not promote that estimate to a measured deployment just because the current model repository is available. Quantization-aware-trained MXFP4 with MXFP8 activations is a particular kernel path. Keep decode-context parallelism, tensor parallelism and draft options separate.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L26

## How should the settings be applied?

Choose one artifact variant and one hardware profile from the original recipe, then resolve its shared, variant, hardware, strategy and optional-feature settings together. The base arguments listed here are only one layer and are deliberately not presented as a complete runnable command. Check any nightly-image requirement and the model’s parsers before loading weights. A context declaration is a model limit, while batched-token and sequence settings control a serving workload.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L201

## What does the memory number mean?

The GB figure is the recipe’s declared planning minimum for the selected variant. It is not a site measurement, a universal peak allocation or a promise of maximum context. KV cache, auxiliary/draft models, image encoders, temporary workspace and active requests still require memory. Keep GB and GiB distinct, and use the actual artifact size and runtime allocation for a machine-specific budget.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L137

## What is needed to reproduce a deployment?

Save the model and conversion revisions, weight hashes, tokenizer/template, container or runtime digest, device count/topology and complete resolved launch. First check one short public request. For a subsequent measurement, fix input and output counts, concurrency, cache state, warmup, sampling, reasoning effort, repetitions and failures. The declarations on this page are source configuration facts; no throughput or quality outcome follows from them.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L443

## Attributed configuration facts

```json
[
  {
    "name": "Architecture",
    "value": "moe",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L39",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Total parameters as counted by recipe",
    "value": "2.8T",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L40",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Active parameters as counted by recipe",
    "value": "16 experts/token + shared (of 896 routed)",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L41",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Native context declaration (tokens)",
    "value": 1048576,
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L42",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Model-wide minimum vLLM",
    "value": "0.29.0",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L32",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Effective default-variant minimum vLLM",
    "value": "0.29.0",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L32",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Default artifact",
    "value": "moonshotai/Kimi-K3",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Default precision label",
    "value": "mxfp4",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Default planning minimum (GB; recipe declaration)",
    "value": 1680,
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L137",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Default variant hardware allowlist",
    "value": [
      "h100",
      "h200",
      "b200",
      "b300",
      "gb200",
      "gb300",
      "mi300x",
      "mi325x",
      "mi355x"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Shared base arguments (incomplete launch)",
    "value": [
      "--trust-remote-code",
      "--gpu-memory-utilization",
      "0.95"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Declared compatible strategies",
    "value": [
      "single_node_tp",
      "multi_node_tp",
      "multi_node_tep",
      "multi_node_tp_pp",
      "multi_node_dep",
      "pd_cluster"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Optional feature names",
    "value": [
      "tool_calling",
      "reasoning",
      "spec_decoding",
      "text_only",
      "decode_context_parallelism"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  },
  {
    "name": "Features selected by default",
    "value": [
      "spec_decoding",
      "text_only",
      "decode_context_parallelism"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/moonshotai/Kimi-K3.yaml#L132",
    "source_document_sha256": "edd8e731474a0ae2d12612cca5b55b66f398eee13ee15fd87d49b803f177ebf6"
  }
]
```