{
  "id": "sglang",
  "title": "SGLang: serving and deployment references",
  "kind": "deployment",
  "source_type": "maintainer_project",
  "summary": "An upstream reference for model-serving deployments. Begin with the project documentation, then verify the exact backend, model and container combination you intend to use.",
  "runtime": "SGLang",
  "hardware_topics": [
    "nvidia"
  ],
  "platform_topics": [
    "linux"
  ],
  "model_label": null,
  "source": {
    "publisher": "sgl-project",
    "url": "https://github.com/sgl-project/sglang",
    "repository": "sgl-project/sglang",
    "revision": "2d3dd3168d4a8e5b69ebd4c9b41a28fa93e6c16e",
    "pinned_url": "https://github.com/sgl-project/sglang/tree/2d3dd3168d4a8e5b69ebd4c9b41a28fa93e6c16e",
    "last_commit_at": "2026-10-17T02:12:41Z",
    "checked_on": "2026-10-17",
    "metadata_status": "checked",
    "report_url": null,
    "platform": "github",
    "document_path": "README.md",
    "document_sha256": "7c99b50fa5ef9f4a666a2f73860f746d3fc259e7e7912d4bff88dffdc1dc4c5e"
  },
  "license": {
    "declared_spdx": "Apache-2.0",
    "scope": "Repository declaration only; models, containers, datasets and per-file terms remain separate."
  },
  "evidence_state": "source_reported",
  "site_reproduced": false,
  "record_type": "curated_reference",
  "performance_ranking_allowed": false,
  "metrics": null,
  "metrics_status": "site_metrics_unmeasured; attributed_source_figures_in_editorial.reported_results",
  "conditions": {
    "model_revision": null,
    "weight_hash": null,
    "runtime_execution_revision": null,
    "input_tokens": null,
    "output_tokens": null,
    "concurrency": null,
    "cache_state": null
  },
  "notes": [
    "Use a version-specific installation guide.",
    "Model architecture and speculative decoding support need their own checks.",
    "Save the complete serving configuration with a performance report."
  ],
  "limitations": [
    "This source tag is a discovery aid; it does not certify a deployment on your device."
  ],
  "comparison_status": "not_comparable",
  "editorial_status": "source_derived_detailed_article",
  "public_benchmark_snapshot": false,
  "reported_setup": {
    "system": null,
    "node_count": null,
    "parallelism": null,
    "kv_dtype": null,
    "context_limit_tokens": null,
    "experiment_date": null
  },
  "runtime_topics": [
    "SGLang"
  ],
  "model_family": "General",
  "model_id": null,
  "model_release_date": null,
  "model_artifact": null,
  "test_profiles": [],
  "deployment_options": [],
  "recipe_parameters": null,
  "measurement_scope": "deployment_reference",
  "editorial": {
    "version": 1,
    "published_on": "2026-10-17",
    "modified_on": "2026-10-17",
    "byline": "Run AI Here editorial",
    "creation_method": "AI-assisted source reading and original editorial synthesis; no independent model execution.",
    "answer": "An upstream reference for model-serving deployments. Begin with the project documentation, then verify the exact backend, model and container combination you intend to use.",
    "sections": [
      {
        "heading": "Where does SGLang fit?",
        "paragraphs": [
          "SGLang is a model-serving framework with its own scheduler, kernel and deployment integrations. Its upstream overview is a starting point; a model-specific cookbook supplies the detailed launch route. A matching model name is insufficient when the backend, quantization or container architecture differs."
        ],
        "citations": [
          "https://github.com/sgl-project/sglang/blob/2d3dd3168d4a8e5b69ebd4c9b41a28fa93e6c16e/README.md"
        ]
      },
      {
        "heading": "Why keep cache and speculative decoding explicit?",
        "paragraphs": [
          "A reused prefix can replace measured prefill with a lookup. A draft model consumes memory and may have a model-specific acceptance pattern. Record whether caching is on, which prompts warm it, and the exact draft artifact and decoding policy. The two-Spark report in this collection shows why these controls materially change reported served rate."
        ],
        "citations": [
          "https://github.com/sgl-project/sglang/blob/2d3dd3168d4a8e5b69ebd4c9b41a28fa93e6c16e/README.md"
        ]
      },
      {
        "heading": "What changes for distributed serving?",
        "paragraphs": [
          "Capture the tensor/data/expert parallelism, interconnect, NCCL transport, per-node image and kernel cache state. A fast standalone link test does not prove that a serving container inherited the same network configuration. The DGXArley transport reference and two-Spark audit describe those distinctions without certifying another cluster."
        ],
        "citations": [
          "https://github.com/sgl-project/sglang/blob/2d3dd3168d4a8e5b69ebd4c9b41a28fa93e6c16e/README.md"
        ]
      },
      {
        "heading": "How should a reproducible trial be scoped?",
        "paragraphs": [
          "First establish architecture and artifact support with a small request. Then record a fixed workload, input/output lengths, concurrency, sampling, warmup, cache, timing boundaries and failures. Keep failed configurations with their conditions. A later successful setting is a new observation rather than proof that the original attempt was valid."
        ],
        "citations": [
          "https://github.com/sgl-project/sglang/blob/2d3dd3168d4a8e5b69ebd4c9b41a28fa93e6c16e/README.md"
        ]
      }
    ],
    "facts": [],
    "reported_results": [],
    "source_document_sha256": "7c99b50fa5ef9f4a666a2f73860f746d3fc259e7e7912d4bff88dffdc1dc4c5e"
  }
}