Build llama.cpp. Check GPU offload.

Choose one build environment

Use an isolated work directory with Git, CMake and a working C/C++ compiler. This recipe covers POSIX shells on macOS or Linux. It does not claim native Windows validation. On Apple hardware choose a native macOS build; on an NVIDIA host confirm a compatible driver and toolkit before choosing CUDA. CPU is a separate baseline. No model is downloaded by the build instructions.

Pin the exact source commit

The build guide at this commit is a fixed documentation reference. Use a disposable source directory and check the revision before configuring a backend. The source pin does not prove support for a new model architecture; read the build and model documentation for this revision. The upstream project now also offers a llama launcher, while this example calls the built executable directly.

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout --detach f74e2fa6a31cc33ab4ee4ae516641cfe62df3924
git rev-parse HEAD
cmake --version

Configure a single backend

Use only the matching command below. Separate build folders make recovery and inspection simpler. Metal is the native macOS route; the CUDA route needs its own compatible environment. Do not copy a CUDA flag into a Metal build and infer equivalent behavior.

# CPU baseline
cmake -S . -B build-cpu -DGGML_CUDA=OFF -DGGML_METAL=OFF
cmake --build build-cpu --config Release

# OR: native macOS Metal
cmake -S . -B build-metal -DGGML_METAL=ON
cmake --build build-metal --config Release

# OR: NVIDIA CUDA on a compatible Linux host
cmake -S . -B build-cuda -DGGML_CUDA=ON
cmake --build build-cuda --config Release

Use an existing GGUF

Select a locally available GGUF whose architecture is supported by the pinned runtime. Record its full filename, byte size and SHA-256, plus the original model revision and conversion or quantization provenance. A community conversion is a separate artifact from the original safetensors repository. The catalog provides source pointers, not a compatibility guarantee.

Inspect one short run

Replace build-cuda with your selected build directory and YOUR_MODEL.gguf with a real local path. First inspect --help for the pinned executable. Then make one short request and read its startup log for the selected backend, device and offloaded layer count. Full or partial offload is a placement observation, not a speed comparison.

./build-cuda/bin/llama-cli --help
./build-cuda/bin/llama-cli -m /path/to/YOUR_MODEL.gguf \
  -ngl 1 -c 2048 -n 16 -p 'Return the word READY.'

Interpret a failure

Compiler or linker failure belongs to the build environment; do not call it a model OOM. An unsupported architecture belongs to runtime/model compatibility. Allocation failure after loading calls for a smaller context or a separately recorded placement change. If offload remains zero, inspect the build backend and device initialization log. Record each changed condition rather than merging attempts.

Keep timing context with any later speed claim

The dated Spark Q4_K_M report separates prefill at context depths 512 and 16,384 from batch-one decode at 512 and 2,048. It does not specify exact runtime or weight commits. Its Qwen3 30B MoE example reports 89.3 decode tokens/s at depth 512, but that cannot validate this source build, another model, or a maximum-context capacity claim. Preserve those gaps before comparing your own run.

Rollback without touching the model

Stop the test process and return to the previous build directory. Keep the source commit, configuration output and model hash. Removing only the new disposable build directory is sufficient to abandon this experiment; the model store and other services need no change. A future timing run must lock prompt shape, KV type, cache, seed and output policy independently of this installation check.

Sources & change log

1. llama.cpp v0.7.0 build documentation ↗Checked 2026-10-17 · Pinned source tag, CPU, Metal and CUDA build paths; not tested locally.
2. llama.cpp llama-bench documentation ↗Checked 2026-10-17 · Microbenchmark semantics; this reference is a moving branch, not an execution pin.

17 Oct 2026 · Revision 2. Source-based task guide and downloadable resource. Source documentation inspected; environment reproduction not run. Moving documentation may change after review.