# NVIDIA Nemotron 3.5 Lightning: vLLM recipe

Nemotron Lightning recipe with distinct NVFP4 and BF16 choices and engine constraints.

- Type: Deployment recipe
- Evidence: source_reported; site reproduced: false
- Model family: Nemotron
- Model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
- Original author: vllm-project
- Original project: https://github.com/vllm-project/recipes
- Pinned document: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml
- Source revision: d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94
- Source commit time (not experiment date): 2026-10-17T09:22:45Z
- Checked: 2026-10-17
- Experiment date: None
- Runtime topics: vLLM
- Hardware topics: nvidia, spark
- Reported setup: {"system": null, "node_count": null, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": null}
- Performance figures: attributed selections with per-point conditions; no cross-source ranking

## Conditions

Unknown is null. Recipe options are not measured benchmark settings.

```json
{
  "conditions": {
    "model_revision": null,
    "weight_hash": null,
    "runtime_execution_revision": null,
    "input_tokens": null,
    "output_tokens": null,
    "concurrency": null,
    "cache_state": null,
    "quantization": null,
    "speculative_decoding": null,
    "temperature": null,
    "top_p": null
  },
  "test_profiles": [],
  "recipe_parameters": {
    "minimum_runtime_version": "0.27.1",
    "nightly_required": false,
    "model_context_limit_tokens": 262144,
    "compatible_strategies": [
      "single_node_tp",
      "single_node_tep"
    ],
    "hardware_claims": [
      {
        "profile": "h100",
        "maintainer_status": "verified"
      },
      {
        "profile": "dgx_spark_gb10",
        "maintainer_status": "verified"
      },
      {
        "profile": "rtx_pro_6000",
        "maintainer_status": "verified"
      },
      {
        "profile": "dgx_station_gb300",
        "maintainer_status": "verified"
      }
    ],
    "recipe_updated_on": "2026-10-08",
    "effective_default_minimum_runtime_version": "0.27.1"
  },
  "deployment_options": [
    {
      "variant": "default",
      "model_id": "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4",
      "precision": "nvfp4",
      "supported_hardware": null,
      "model_revision": null,
      "declared_planning_minimum_gb": 18,
      "minimum_runtime_version": "0.27.1",
      "extra_args": [
        "--kv-cache-dtype",
        "fp8",
        "--moe-backend",
        "marlin"
      ],
      "tensor_parallel_by_hardware": null
    },
    {
      "variant": "bf16",
      "model_id": "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16",
      "precision": "bf16",
      "supported_hardware": null,
      "model_revision": null,
      "declared_planning_minimum_gb": 72,
      "minimum_runtime_version": "0.27.1",
      "extra_args": [
        "--moe-backend",
        "flashinfer_cutlass"
      ],
      "tensor_parallel_by_hardware": null
    }
  ]
}
```

## Reading notes

- Recipe minimum engine version: 0.27.1.
- Default artifact: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4.
- Variant and hardware settings are source declarations; select the matching original profile.

## Limits

- The default is NVFP4 even though the source filename contains BF16.
- Input length, output policy, concurrency and cache are workload-specific; no single benchmark is inferred from the recipe.

## License declaration

{"declared_spdx": "Apache-2.0", "scope": "Recipe repository only; each model and conversion has independent distribution terms."}


## Editorial answer

The pinned NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 recipe defaults to nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 (nvfp4). Its effective default-variant minimum is vLLM 0.27.1. Select the exact variant, hardware override and topology before treating it as a launch plan. No deployment was run by this site.

By Run AI Here editorial · 2026-10-17

AI-assisted source reading and original editorial synthesis; no independent model execution.

## Which artifact and engine does this recipe select?

The recipe identifies nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 as its model family entry and selects nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 by default. The default precision label is nvfp4; model-wide and variant-specific runtime requirements are kept separately below. The document revision fixes what the recipe said, while the weight revision and executed runtime revision remain separate fields. A model-family title alone cannot establish either artifact identity.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69

## What needs special attention for this model?

The title contains BF16, but the default is the NVIDIA NVFP4 conversion with FP8 KV cache. The documented draft models target that NVFP4 path. The Mamba backend and cache mode are part of the hybrid model’s execution configuration; a generic Transformer KV formula is not a verified memory model for it.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L19

## How should the settings be applied?

Choose one artifact variant and one hardware profile from the original recipe, then resolve its shared, variant, hardware, strategy and optional-feature settings together. The base arguments listed here are only one layer and are deliberately not presented as a complete runnable command. Check any nightly-image requirement and the model’s parsers before loading weights. A context declaration is a model limit, while batched-token and sequence settings control a serving workload.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L188

## What does the memory number mean?

The GB figure is the recipe’s declared planning minimum for the selected variant. It is not a site measurement, a universal peak allocation or a promise of maximum context. KV cache, auxiliary/draft models, image encoders, temporary workspace and active requests still require memory. Keep GB and GiB distinct, and use the actual artifact size and runtime allocation for a machine-specific budget.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L117

## What is needed to reproduce a deployment?

Save the model and conversion revisions, weight hashes, tokenizer/template, container or runtime digest, device count/topology and complete resolved launch. First check one short public request. For a subsequent measurement, fix input and output counts, concurrency, cache state, warmup, sampling, reasoning effort, repetitions and failures. The declarations on this page are source configuration facts; no throughput or quality outcome follows from them.

Sources: https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L196

## Attributed configuration facts

```json
[
  {
    "name": "Architecture",
    "value": "moe",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L23",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Total parameters as counted by recipe",
    "value": "30B",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L24",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Active parameters as counted by recipe",
    "value": "3B",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L25",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Native context declaration (tokens)",
    "value": 262144,
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L26",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Model-wide minimum vLLM",
    "value": "0.27.1",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L21",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Effective default-variant minimum vLLM",
    "value": "0.27.1",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L21",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Default artifact",
    "value": "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Default precision label",
    "value": "nvfp4",
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Default planning minimum (GB; recipe declaration)",
    "value": 18,
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L117",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Default variant hardware allowlist",
    "value": null,
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Shared base arguments (incomplete launch)",
    "value": [
      "--mamba-backend",
      "flashinfer",
      "--mamba-cache-mode",
      "align",
      "--enable-prefix-caching",
      "--max-num-batched-tokens",
      "16384"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Declared compatible strategies",
    "value": [
      "single_node_tp",
      "single_node_tep"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Optional feature names",
    "value": [
      "reasoning",
      "tool_calling",
      "mamba_fast_ssm_cache",
      "spec_decoding",
      "long_context"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  },
  {
    "name": "Features selected by default",
    "value": [
      "spec_decoding",
      "long_context"
    ],
    "classification": "recipe_declaration",
    "evidence_url": "https://github.com/vllm-project/recipes/blob/d5fc1f55cee70f4e1944f2dd4f7569cb8276ad94/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml#L69",
    "source_document_sha256": "244058cc70b3d0d131dbf8bbd5dcea627fc81c9c487dacf9c64db870a91ea1ff"
  }
]
```