{
  "id": "A04",
  "path": "/guides/mac-precision-memory/",
  "title": "Mac Weight Precision: Speed, First Token and Memory",
  "summary": "Compare Qwen3.5 4B and 9B weight formats on one M4 Pro, with separate workloads for generation, first-token delay and peak memory.",
  "answer": "The selected M4 Pro report shows a tradeoff between generation rate, first-token delay and memory. It does not measure quantization accuracy. Keep the model, workload, runtime and missing conditions attached to every value.",
  "published_on": "2026-10-18",
  "modified_on": "2026-10-18",
  "source_checked_on": "2026-10-18",
  "sources": [
    {
      "publisher": "weklund",
      "url": "https://github.com/weklund/mlx-coding-bench",
      "repository": "weklund/mlx-coding-bench",
      "platform": "github",
      "revision": "5ae9b491f7a0539d5abef949eded6bd57785a981",
      "document_path": "measurements/llm_benchmarks/Apple_M4_Pro_10P+4E+20GPU_64GB/2026-03-06__19:47:24/REPORT.md",
      "pinned_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md",
      "report_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md",
      "document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "last_commit_at": "2026-07-01T19:16:36Z",
      "checked_on": "2026-10-18",
      "metadata_status": "checked",
      "record_id": "qwen35-m4pro-20260306"
    }
  ],
  "facts": [],
  "reported_results": [
    {
      "label": "Qwen3.5-4B int4 generation_rate",
      "metric": "generation_rate",
      "value": 68.0,
      "unit": "tokens/s",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.4
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "int4",
        "input_tokens": 256,
        "output_tokens": 100
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L27",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 27
    },
    {
      "label": "Qwen3.5-4B int8 generation_rate",
      "metric": "generation_rate",
      "value": 42.7,
      "unit": "tokens/s",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.3
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "int8",
        "input_tokens": 256,
        "output_tokens": 100
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L27",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 27
    },
    {
      "label": "Qwen3.5-4B bf16 generation_rate",
      "metric": "generation_rate",
      "value": 27.7,
      "unit": "tokens/s",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.2
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "bf16",
        "input_tokens": 256,
        "output_tokens": 100
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L27",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 27
    },
    {
      "label": "Qwen3.5-9B int4 generation_rate",
      "metric": "generation_rate",
      "value": 38.5,
      "unit": "tokens/s",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.4
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "int4",
        "input_tokens": 256,
        "output_tokens": 100
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L28",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 28
    },
    {
      "label": "Qwen3.5-9B int8 generation_rate",
      "metric": "generation_rate",
      "value": 24.1,
      "unit": "tokens/s",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.2
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "int8",
        "input_tokens": 256,
        "output_tokens": 100
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L28",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 28
    },
    {
      "label": "Qwen3.5-9B bf16 generation_rate",
      "metric": "generation_rate",
      "value": 15.0,
      "unit": "tokens/s",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.4
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "bf16",
        "input_tokens": 256,
        "output_tokens": 100
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L28",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 28
    },
    {
      "label": "Qwen3.5-4B int4 ttft",
      "metric": "ttft",
      "value": 6.14,
      "unit": "seconds",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.25
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "int4",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L40",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 40
    },
    {
      "label": "Qwen3.5-4B int8 ttft",
      "metric": "ttft",
      "value": 6.14,
      "unit": "seconds",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.25
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "int8",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L40",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 40
    },
    {
      "label": "Qwen3.5-4B bf16 ttft",
      "metric": "ttft",
      "value": 4.67,
      "unit": "seconds",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.03
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "bf16",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L40",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 40
    },
    {
      "label": "Qwen3.5-9B int4 ttft",
      "metric": "ttft",
      "value": 10.88,
      "unit": "seconds",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.61
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "int4",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L41",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 41
    },
    {
      "label": "Qwen3.5-9B int8 ttft",
      "metric": "ttft",
      "value": 10.63,
      "unit": "seconds",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.4
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "int8",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L41",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 41
    },
    {
      "label": "Qwen3.5-9B bf16 ttft",
      "metric": "ttft",
      "value": 8.2,
      "unit": "seconds",
      "statistic": "mean of 20 measurements",
      "uncertainty": {
        "standard_deviation": 0.06
      },
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "bf16",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L41",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 41
    },
    {
      "label": "Qwen3.5-4B int4 peak_memory",
      "metric": "peak_memory",
      "value": 6.4,
      "unit": "GiB",
      "statistic": "reported peak; no per-cell dispersion supplied",
      "uncertainty": null,
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "int4",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L53",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 53
    },
    {
      "label": "Qwen3.5-4B int8 peak_memory",
      "metric": "peak_memory",
      "value": 8.5,
      "unit": "GiB",
      "statistic": "reported peak; no per-cell dispersion supplied",
      "uncertainty": null,
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "int8",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L53",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 53
    },
    {
      "label": "Qwen3.5-4B bf16 peak_memory",
      "metric": "peak_memory",
      "value": 10.2,
      "unit": "GiB",
      "statistic": "reported peak; no per-cell dispersion supplied",
      "uncertainty": null,
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-4B",
        "weight_precision": "bf16",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L53",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 53
    },
    {
      "label": "Qwen3.5-9B int4 peak_memory",
      "metric": "peak_memory",
      "value": 9.0,
      "unit": "GiB",
      "statistic": "reported peak; no per-cell dispersion supplied",
      "uncertainty": null,
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "int4",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L54",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 54
    },
    {
      "label": "Qwen3.5-9B int8 peak_memory",
      "metric": "peak_memory",
      "value": 13.4,
      "unit": "GiB",
      "statistic": "reported peak; no per-cell dispersion supplied",
      "uncertainty": null,
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "int8",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L54",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 54
    },
    {
      "label": "Qwen3.5-9B bf16 peak_memory",
      "metric": "peak_memory",
      "value": 19.7,
      "unit": "GiB",
      "statistic": "reported peak; no per-cell dispersion supplied",
      "uncertainty": null,
      "conditions": {
        "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
        "node_count": 1,
        "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
        "experiment_date": "2026-03-06",
        "repetitions": 20,
        "warmup_iterations": 1,
        "weight_revision": null,
        "conversion_revision": null,
        "kv_dtype": null,
        "concurrency": null,
        "cache_state": null,
        "sampling": null,
        "runtime_execution_revision": null,
        "model": "Qwen3.5-9B",
        "weight_precision": "bf16",
        "input_tokens": 4096,
        "output_tokens": null
      },
      "evidence_state": "source_reported",
      "site_reproduced": false,
      "evidence_url": "https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L54",
      "source_document_sha256": "ff21a1f8a2597867e191a376cea28af293d47dfff82bb423193009b05c7ef5a3",
      "source_record_id": "qwen35-m4pro-20260306",
      "source_line": 54
    }
  ],
  "conditions": {
    "system": "Apple M4 Pro 10P+4E / 20 GPU cores / 64 GB unified memory",
    "node_count": 1,
    "runtime": "MLX 0.31.0 / mlx-lm 0.30.7",
    "experiment_date": "2026-03-06",
    "repetitions": 20,
    "warmup_iterations": 1,
    "weight_revision": null,
    "conversion_revision": null,
    "kv_dtype": null,
    "concurrency": null,
    "cache_state": null,
    "sampling": null,
    "runtime_execution_revision": null,
    "quality_evaluation": null,
    "comparison_scope": "same machine, same model, same metric workload only",
    "hardware_memory_unit": "GB as reported; not GiB",
    "generation_workload": {
      "input_tokens": 256,
      "output_tokens": 100
    },
    "ttft_and_memory_workload": {
      "input_tokens": 4096,
      "output_tokens": null
    },
    "catalog_point_count_unchanged": 49,
    "selection_status": "additional selection from an existing primary document, not new experiments"
  },
  "generation": {
    "text_draft": "local Spark Qwen",
    "review": "Run AI Here with Codex",
    "image": "deterministic source-backed matplotlib figure",
    "local_first": true
  },
  "image": {
    "url": "/assets/mac-precision-memory-source-chart.webp",
    "alt": "Source-reported Qwen3.5 4B and 9B comparison with separate generation, TTFT and peak-memory panels.",
    "width": 1600,
    "height": 900,
    "sha256": "cd7f32a65e95629573300ebcf1d54585982066733ac0d7714d7cf0d937393610",
    "caption": "Source-backed figure rendered directly from 18 selected values; no image model generated the numbers."
  },
  "evidence_selection": {
    "scope": "18 selected values from the existing Mac source; may overlap catalog selections",
    "primary_catalog_point_count": 49,
    "quality_measured": false
  },
  "schema_version": "1.0.0",
  "record_type": "source_backed_explainer",
  "evidence_state": "source_reported",
  "site_reproduced": false,
  "metrics": null,
  "performance_ranking_allowed": false,
  "body_markdown": "# Mac Weight Precision: Speed, First Token and Memory\n\nWeight precision is a useful deployment choice only when the model and workload stay attached to the result. A smaller memory footprint does not tell you how accurate the answers remain. A fast generation rate does not tell you how long the first token takes.\n\nThis note selects the 4B and 9B rows from [weklund’s pinned Qwen3.5 MLX report](https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md). They provide a useful same-machine example. They are historical Qwen3.5 measurements, not measurements of the latest Qwen checkpoint or every Mac.\n\n![Source-reported Qwen3.5 4B and 9B comparison with separate generation, TTFT and peak-memory panels.](/assets/mac-precision-memory-source-chart.webp)\n\nFigure: deterministically drawn from the 18 selected source values. Rate and TTFT bars show the reported standard deviation; memory bars have no error bars because per-cell dispersion was not supplied. The workloads differ between panels. Full values and conditions are also available in text and JSON.\n\n## The conditions that make this comparison meaningful\n\nThe report identifies an M4 Pro with 10P+4E CPU cores, 20 GPU cores and 64 GB unified memory; MLX 0.31.0 and mlx-lm 0.30.7; experiment date 6 March 2026; one warmup followed by 20 measurements. Its rate table uses 256 input and 100 generated tokens. The TTFT and memory tables use 4,096 input tokens. These are [the report’s selected workloads](https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L19), rather than one common request profile.\n\nFor the two long-input tables, the selected sections do not specify output length. That field stays unknown. The general methodology’s maximum generation length does not establish the exact length used for every section.\n\n## Same model, three weight formats\n\n| Model / weight format | Generation rate, tokens/s | TTFT, seconds | Reported peak, GiB |\n|---|---:|---:|---:|\n| 4B int4 | 68.0 ± 0.4 | 6.14 ± 0.25 | 6.4 |\n| 4B int8 | 42.7 ± 0.3 | 6.14 ± 0.25 | 8.5 |\n| 4B bf16 | 27.7 ± 0.2 | 4.67 ± 0.03 | 10.2 |\n| 9B int4 | 38.5 ± 0.4 | 10.88 ± 0.61 | 9.0 |\n| 9B int8 | 24.1 ± 0.2 | 10.63 ± 0.40 | 13.4 |\n| 9B bf16 | 15.0 ± 0.4 | 8.20 ± 0.06 | 19.7 |\n\nAttribution: weklund; [generation rows](https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L27), [TTFT rows](https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L40), [memory rows](https://github.com/weklund/mlx-coding-bench/blob/5ae9b491f7a0539d5abef949eded6bd57785a981/measurements/llm_benchmarks/Apple_M4_Pro_10P%2B4E%2B20GPU_64GB/2026-03-06__19%3A47%3A24/REPORT.md#L53). ± means standard deviation, not a confidence interval. The memory column is a reported peak, not a model-file size or total-system free-memory reading.\n\n## Read the tradeoff against your task\n\nIn these selected rows, int4 has the higher generation rate and lower reported memory, while bf16 has the shorter first-token wait. That pattern explains why “the fastest precision” needs a stated objective.\n\nFor a long output, investigate sustained delivery and total completion time. For a short answer after a long prompt, investigate first-token wait. For a Mac used alongside other applications, record actual system pressure and swap during a representative request. The table can narrow a test plan; it cannot replace those observations.\n\nAvoid subtracting the GiB peak from the machine’s GB specification to advertise “free RAM.” The units and measurement boundaries differ. Likewise, do not combine the short-input rate with the long-input TTFT to calculate an end-to-end response time: they describe different workloads.\n\n## What is still unknown\n\nThe selected report does not pin exact converted-weight hashes, conversion execution revisions, KV dtype, concurrency, sampling or cache state. Those fields remain null in the export. The reported cooldown policy is useful context, but it is not a thermal trace.\n\nThere is no matched task-quality or perplexity evaluation here. The original document also includes broader recommendations; this note does not turn those recommendations into measured quantization accuracy. Before choosing a smaller format for extraction, coding or reasoning, compare its outputs against the same task set and acceptance criteria.\n\n## How to use the evidence without overextending it\n\nKeep one candidate model fixed, record its conversion and runtime, and choose representative input and output lengths. Check correctness separately from responsiveness and memory. Record warmup, cache, concurrency and repeated-run variation. Only then decide which format serves the task well enough.\n\nRun AI Here has not independently benchmarked these models. The adjacent Markdown and JSON retain the source revision, row anchors, uncertainty and missing fields so an AI reader can preserve the same limits.\n"
}