How we collect community deployment reports.

Find the original operator

Prefer a maintainer repository, original benchmark report or a traceable operator write-up. An aggregator headline can be a lead, but the collection links back to the person or project responsible for the deployment and measurements.

Keep the source date and revision

Record when we checked the reference and pin the repository revision where available. A commit date is not the experiment date. A later repository update does not silently change the identity of an earlier experiment.

Read a configuration as a whole

Keep model revision, quantization, hardware count/topology, runtime, input/output lengths, concurrency, cache and speculative decoding together. A repository may contain multiple profiles and inherited benchmark sections. A default launch setting is not automatically the configuration behind a result.

Leave incomplete evidence incomplete

Unknown fields are null. Missing failure counts are not zero. A throughput number without its denominator or token shape should not be converted into a precise ranking. Source topics such as Apple silicon or NVIDIA help discovery but do not certify support for every model.

Extract a figure with its configuration

The detailed articles include selected factual results rather than copying complete source tables. Every extracted point keeps its unit, statistic, workload, hardware, explicit missing fields and a link to the pinned source row. Mean ± standard deviation is distinct from a confidence interval, and aggregate throughput is distinct from single-request decode. The original full report remains the source of record.

Make the same material easy for AI

HTML is the main reading surface. catalog.json, catalog.md and per-record JSON/Markdown expose the same sources and limitations. They contain reference data only; there is no benchmark queue, command runner, model-download endpoint or authentication material.

Keep evidence states separate

Current records are source_reported. We have not run independent performance tests. The historical private archive and synthetic developer fixtures remain separate from the curated-reference export. A future site-reproduced result would require its own traceable run and applicable release review.

Preserve disagreement instead of manufacturing precision

For example, the MiniMax card and vLLM recipe report different parameter counts, and a recipe’s variant may raise the required engine version. Record both declarations with their sources. An advertised memory minimum is a planning claim unless the source explicitly supplies a measured setup and workload. Do not fill a missing value with a family default.

How these articles are produced and corrected

The editorial process uses AI assistance to read inspected public documents and write original explanations. Saved document receipts and hashes support the source checks; the public exports retain source identity and per-figure references. No model installation, GPU evaluation or independent performance reproduction has been performed by Run AI Here. A correction should identify the record, pinned source and affected condition so the HTML, JSON and Markdown can be updated together.

Sources & change log

1. llama.cpp llama-bench documentation ↗Checked 2026-10-17 · Microbenchmark semantics; this reference is a moving branch, not an execution pin.
2. vLLM bench serve documentation ↗Checked 2026-10-17 · Request-level metrics; latest documentation is not a tested runtime version.

17 Oct 2026 · Revision 2. Source-based task guide and downloadable resource. Source documentation inspected; environment reproduction not run. Moving documentation may change after review.