{
  "id": "vllm",
  "title": "vLLM: model-serving starting point",
  "kind": "deployment",
  "source_type": "maintainer_project",
  "summary": "Follow the serving project and its documentation when planning a compatible model-serving environment. Backend, architecture, package and model support are version-dependent.",
  "runtime": "vLLM",
  "hardware_topics": [
    "nvidia"
  ],
  "platform_topics": [
    "linux"
  ],
  "model_label": null,
  "source": {
    "publisher": "vllm-project",
    "url": "https://github.com/vllm-project/vllm",
    "repository": "vllm-project/vllm",
    "revision": "ce49c47cbb241b59a32ef110079a5261dbd08e49",
    "pinned_url": "https://github.com/vllm-project/vllm/tree/ce49c47cbb241b59a32ef110079a5261dbd08e49",
    "last_commit_at": "2026-10-17T02:26:57Z",
    "checked_on": "2026-10-17",
    "metadata_status": "checked",
    "report_url": null,
    "platform": "github",
    "document_path": "README.md",
    "document_sha256": "4dc7399323b9660117f8cc1226afb7a81cdbb912dbb4ae74b66d89052417d11c"
  },
  "license": {
    "declared_spdx": "Apache-2.0",
    "scope": "Repository declaration only; models, containers, datasets and per-file terms remain separate."
  },
  "evidence_state": "source_reported",
  "site_reproduced": false,
  "record_type": "curated_reference",
  "performance_ranking_allowed": false,
  "metrics": null,
  "metrics_status": "site_metrics_unmeasured; attributed_source_figures_in_editorial.reported_results",
  "conditions": {
    "model_revision": null,
    "weight_hash": null,
    "runtime_execution_revision": null,
    "input_tokens": null,
    "output_tokens": null,
    "concurrency": null,
    "cache_state": null
  },
  "notes": [
    "Pin the runtime and review its support matrix.",
    "Keep context, concurrency, cache and output policy with every test.",
    "Use request-level measurements for service latency."
  ],
  "limitations": [
    "The NVIDIA/Linux tag is this entry’s focus, not an exhaustive support matrix.",
    "No Windows-native or arbitrary ARM container compatibility is inferred."
  ],
  "comparison_status": "not_comparable",
  "editorial_status": "source_derived_detailed_article",
  "public_benchmark_snapshot": false,
  "reported_setup": {
    "system": null,
    "node_count": null,
    "parallelism": null,
    "kv_dtype": null,
    "context_limit_tokens": null,
    "experiment_date": null
  },
  "runtime_topics": [
    "vLLM"
  ],
  "model_family": "General",
  "model_id": null,
  "model_release_date": null,
  "model_artifact": null,
  "test_profiles": [],
  "deployment_options": [],
  "recipe_parameters": null,
  "measurement_scope": "deployment_reference",
  "editorial": {
    "version": 1,
    "published_on": "2026-10-17",
    "modified_on": "2026-10-17",
    "byline": "Run AI Here editorial",
    "creation_method": "AI-assisted source reading and original editorial synthesis; no independent model execution.",
    "answer": "Follow the serving project and its documentation when planning a compatible model-serving environment. Backend, architecture, package and model support are version-dependent.",
    "sections": [
      {
        "heading": "What does vLLM contribute to a serving stack?",
        "paragraphs": [
          "The upstream project provides high-throughput model serving, memory-management and scheduling mechanisms, with an OpenAI-compatible server among its entry points. Model architecture, quantization kernel, accelerator backend and package version remain a joint compatibility question. Use a model-specific recipe alongside this project overview."
        ],
        "citations": [
          "https://github.com/vllm-project/vllm/blob/ce49c47cbb241b59a32ef110079a5261dbd08e49/README.md"
        ]
      },
      {
        "heading": "How should the model and runtime be pinned?",
        "paragraphs": [
          "Record the installed wheel or container digest and executed source revision, then pin the original model and any converted weights independently. A README revision identifies the documentation reviewed here; it does not identify a runtime launched by this site. The collection’s recipe version minima are compatibility declarations, not reproduction receipts."
        ],
        "citations": [
          "https://github.com/vllm-project/vllm/blob/ce49c47cbb241b59a32ef110079a5261dbd08e49/README.md"
        ]
      },
      {
        "heading": "Which controls change service performance?",
        "paragraphs": [
          "Context length, active sequences, batched tokens, KV precision, prefix caching, expert/tensor parallelism and speculative decoding can change latency and memory use. Keep the complete launch and client request with each result. Decode tokens per second, aggregate throughput and end-to-end served rate use different denominators."
        ],
        "citations": [
          "https://github.com/vllm-project/vllm/blob/ce49c47cbb241b59a32ef110079a5261dbd08e49/README.md"
        ]
      },
      {
        "heading": "What should a first performance report contain?",
        "paragraphs": [
          "Begin with a load and correctness check, then preserve the prompt corpus, token counts, output stopping policy, users, cache state, warmup, repeats, failures and latency distribution. Record device count and topology rather than summing nominal memory into a presumed shared pool. This page does not launch a server or claim measurements of its own."
        ],
        "citations": [
          "https://github.com/vllm-project/vllm/blob/ce49c47cbb241b59a32ef110079a5261dbd08e49/README.md"
        ]
      }
    ],
    "facts": [],
    "reported_results": [],
    "source_document_sha256": "4dc7399323b9660117f8cc1226afb7a81cdbb912dbb4ae74b66d89052417d11c"
  }
}