# llama.cpp: GGUF deployment paths

Start with the upstream build and server documentation for a GGUF deployment. Choose the backend for your own platform and keep the model artifact separate from the runtime revision.

- Type: Deployment recipe
- Evidence: source_reported; site reproduced: false
- Model family: General
- Model: None
- Original author: ggml-org
- Original project: https://github.com/ggml-org/llama.cpp
- Pinned document: https://github.com/ggml-org/llama.cpp/tree/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924
- Source revision: f74e2fa6a31cc33ab4ee4ae516641cfe62df3924
- Source commit time (not experiment date): 2026-10-17T02:03:41Z
- Checked: 2026-10-17
- Experiment date: None
- Runtime topics: llama.cpp
- Hardware topics: cpu, apple, nvidia
- Reported setup: {"system": null, "node_count": null, "parallelism": null, "kv_dtype": null, "context_limit_tokens": null, "experiment_date": null}
- Performance figures: attributed selections with per-point conditions; no cross-source ranking

## Conditions

Unknown is null. Recipe options are not measured benchmark settings.

```json
{
  "conditions": {
    "model_revision": null,
    "weight_hash": null,
    "runtime_execution_revision": null,
    "input_tokens": null,
    "output_tokens": null,
    "concurrency": null,
    "cache_state": null
  },
  "test_profiles": [],
  "recipe_parameters": null,
  "deployment_options": []
}
```

## Reading notes

- CPU, Metal and CUDA are separate build paths.
- A GGUF file still needs an architecture supported by the selected build.
- Record the actual model hash, context and offloaded layers when trying a recipe.

## Limits

- Model revision and artifact: selected by the reader.
- A repository revision is a documentation pin, not a reproduced deployment.

## License declaration

{"declared_spdx": "MIT", "scope": "Repository declaration only; models, containers, datasets and per-file terms remain separate."}


## Editorial answer

Start with the upstream build and server documentation for a GGUF deployment. Choose the backend for your own platform and keep the model artifact separate from the runtime revision.

By Run AI Here editorial · 2026-10-17

AI-assisted source reading and original editorial synthesis; no independent model execution.

## What does the upstream project provide?

llama.cpp supplies C/C++ inference with multiple backend paths, a CLI and an API server. The pinned README separates package installation, prebuilt binaries, Docker and source builds. A source checkout and a downloaded GGUF are different artifacts; pin both when preparing a reproducible setup. The current README also presents the newer llama cli and llama serve launcher forms.

Sources: https://github.com/ggml-org/llama.cpp/blob/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924/README.md#L20

## How should a backend be selected?

Apple silicon uses the native Metal route; NVIDIA CUDA, AMD HIP and other backends have their own build requirements. CPU/GPU hybrid execution can offload part of a model. That support list is a maintainer declaration across the project, not evidence that every model architecture or quantization runs on every backend.

Sources: https://github.com/ggml-org/llama.cpp/blob/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924/README.md#L76

## What should be recorded during a first run?

Record the runtime commit, compiler and CMake configuration, model and conversion revision, complete GGUF hash, selected backend, layer placement, context and KV types. Read the startup log to verify actual placement. A built CUDA binary and a successful CPU request do not establish that layers were placed on the GPU.

Sources: https://github.com/ggml-org/llama.cpp/blob/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924/README.md#L72

## How is a deployment check different from a benchmark?

A short prompt confirms that the selected artifact and executable can complete a request. A speed test additionally needs fixed input and output lengths, concurrency, cache state, sampling, repeated trials and failure counts. Use the linked installation guide for a minimal check and the community reports for dated measurement examples; the project README itself is not a speed result.

Sources: https://github.com/ggml-org/llama.cpp/blob/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924/README.md#L60