slobodanmargetic988/weeyuga-benchmarks-public
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

B3 staging seed — 21 runs + catalogue v1.0-draft + methodology + README

Browse Source 
Initial population of the weeyuga-benchmarks-public archive (PRIVATE
staging visibility — flips public after Miljan + Stevan security audit
sign-off per Sloba's 17:34Z dispatch).

Contents:
- README.md       — public-facing intro (warns staging state, schema overview, citation pattern, license split)
- LICENSE         — CC-BY-4.0 default (auto-init from Gitea)
- catalogue.json  — schema_version=1.0-draft (locked once Tomas ratifies); 21 benchmarks indexed, 13 complete + 8 meta-only
- methodology.md  — mirror of WeeyugaWeb docs/BENCHMARKS/HARNESS.md (canonical methodology)
- runs/<id>/run.jsonl|run.log|run.md|metadata.json — packaged copies of every run in WeeyugaWeb docs/BENCHMARKS/runs/*

Run set covers:
- Mission 1 (2026-04-28/29): pavilion-weeyuga-v1 + reconstructed v3 (96 calls, 16 models routed via weeyuga :11435)
- Predator trio (2026-05-04): granite-4.1-8B + gemma-4-E4B-it + qwen3.5-9B
- Predator qwen rerun (2026-05-04): qwen3.5-9B think500/nothink + qwen3-14B feasibility
- A3B campaign (2026-05-04/05): pavilion-a3b + predator-a3b NGL matrix + ctx sweep + NGL+ctx 2D + NGL=6 deep dive
- VPS50 CPU matrix + gemma-e4b CPU lane (2026-05-04/05)

Visibility GATE: this repo stays private until Miljan G1-G4 audit and
Stevan G3 credential audit both green. After sign-off, single API call
flips visibility=public, anonymous read on, push-protection requires
auth, issues moderate by default.

No raw IPs, no SSH user@host strings, no /Users/ paths, no whisper
transcripts in any of these files. Hardware names (pavilion, predator,
vps50) are intentional and fine to share.

Builder: WeeyugaWeb/scripts/benchmarks/build_catalogue.py (deterministic,
idempotent, ~5s wall on 21 runs).
Publish flow: WeeyugaWeb/scripts/benchmarks/publish_bench_run.py
(builds packaged dirs, regenerates catalogue, optional --push to mirror
into this repo, optional --deploy stub for cicd rsync).

Owner: mac/benchmark-tester-ben (Ben).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Slobodan Margetic
2026-05-05 19:46:01 +02:00
parent 5c726cf585
commit a18db6a3da
70 changed files with 16023 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

770
runs/b37836bf-d1a2-4d6b-a732-aff89da1fa07/metadata.json Normal file
View File

@@ -0,0 +1,770 @@
{
"id": "b37836bf-d1a2-4d6b-a732-aff89da1fa07",
"title": "predator-a3b-ngl-matrix-1 \u2014 qwen3 on predator",
"date": "2026-05-04",
"hardware": "predator",
"engine": "llamacpp",
"harness": "predator-a3b-ngl-matrix-1",
"model_family": "qwen3",
"model_sizes": [
"qwen3:30b-a3b-iq2m"
],
"task_kind": null,
"headline": "42 calls across 3 cell(s); ~5.7 tok/s mean; p50 71.5s",
"summary_md_path": null,
"raw_jsonl_path": "runs/b37836bf-d1a2-4d6b-a732-aff89da1fa07.jsonl",
"log_path": "runs/b37836bf-d1a2-4d6b-a732-aff89da1fa07.log",
"synthesis_doc": "A3B_NGL_RETUNE_2026-05-05.md",
"cells": [
{
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 85436.42,
"duration_ms_p50": 92483.0,
"tokens_per_sec_mean": 4.95,
"tokens_per_sec_max": 5.15
},
{
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 75013.58,
"duration_ms_p50": 81347.0,
"tokens_per_sec_mean": 5.59,
"tokens_per_sec_max": 6.63
},
{
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 62607.33,
"duration_ms_p50": 67049.0,
"tokens_per_sec_mean": 6.58,
"tokens_per_sec_max": 6.89
}
],
"n_calls_total": 42,
"n_errors_total": 0,
"started_at": "2026-05-04T22:50:47Z",
"tags": [
"a3b",
"iq2",
"matrix",
"ngl",
"predator",
"qwen3"
],
"status": "complete",
"visibility": "draft",
"cells_full": [
{
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 85436.42,
"duration_ms_p50": 92483.0,
"tokens_per_sec_mean": 4.95,
"tokens_per_sec_max": 5.15,
"calls": [
{
"type": "vram_snapshot",
"model_key": "ngl6",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"ngl": 6,
"raw": "2984, 3046, 6144"
},
{
"type": "llama_bench",
"model_key": "ngl6",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"ngl": 6,
"raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 6 | pp256 | 110.34 \u00b1 3.41 |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 6 | tg64 | 5.23 \u00b1 0.67 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "hello",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 23904,
"prompt_tokens": 9,
"completion_tokens": 114,
"tokens_per_sec": 4.769076305220883,
"finish_reason": "stop",
"reasoning_chars": 433,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "hello",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 19868,
"prompt_tokens": 9,
"completion_tokens": 94,
"tokens_per_sec": 4.731226092208577,
"finish_reason": "stop",
"reasoning_chars": 349,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "hello",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 22272,
"prompt_tokens": 9,
"completion_tokens": 105,
"tokens_per_sec": 4.714439655172414,
"finish_reason": "stop",
"reasoning_chars": 394,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "hello",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 32650,
"prompt_tokens": 9,
"completion_tokens": 168,
"tokens_per_sec": 5.145482388973966,
"finish_reason": "stop",
"reasoning_chars": 656,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-MEDIUM",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 108039,
"prompt_tokens": 22,
"completion_tokens": 512,
"tokens_per_sec": 4.73902942455965,
"finish_reason": "length",
"reasoning_chars": 2049,
"content_chars": 513,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-MEDIUM",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 71270,
"prompt_tokens": 22,
"completion_tokens": 350,
"tokens_per_sec": 4.910902202890417,
"finish_reason": "stop",
"reasoning_chars": 1313,
"content_chars": 403,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-MEDIUM",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 92416,
"prompt_tokens": 22,
"completion_tokens": 449,
"tokens_per_sec": 4.858466066481995,
"finish_reason": "stop",
"reasoning_chars": 1718,
"content_chars": 477,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-MEDIUM",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 100578,
"prompt_tokens": 22,
"completion_tokens": 512,
"tokens_per_sec": 5.0905764680148735,
"finish_reason": "length",
"reasoning_chars": 2008,
"content_chars": 541,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-HARD",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 155414,
"prompt_tokens": 30,
"completion_tokens": 786,
"tokens_per_sec": 5.057459430939298,
"finish_reason": "stop",
"reasoning_chars": 1713,
"content_chars": 1659,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-HARD",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 92550,
"prompt_tokens": 30,
"completion_tokens": 476,
"tokens_per_sec": 5.143165856293895,
"finish_reason": "stop",
"reasoning_chars": 1080,
"content_chars": 1010,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-HARD",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 153066,
"prompt_tokens": 30,
"completion_tokens": 784,
"tokens_per_sec": 5.121973527759267,
"finish_reason": "stop",
"reasoning_chars": 2134,
"content_chars": 1183,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6",
"model_key": "ngl6",
"ngl": 6,
"prompt_id": "P-HARD",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 153210,
"prompt_tokens": 30,
"completion_tokens": 789,
"tokens_per_sec": 5.149794399843352,
"finish_reason": "stop",
"reasoning_chars": 1316,
"content_chars": 1947,
"error": null
}
]
},
{
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 75013.58,
"duration_ms_p50": 81347.0,
"tokens_per_sec_mean": 5.59,
"tokens_per_sec_max": 6.63,
"calls": [
{
"type": "vram_snapshot",
"model_key": "ngl12",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"ngl": 12,
"raw": "4708, 1322, 6144"
},
{
"type": "llama_bench",
"model_key": "ngl12",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"ngl": 12,
"raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 12 | pp256 | 121.86 \u00b1 3.89 |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 12 | tg64 | 6.37 \u00b1 0.63 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "hello",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 41903,
"prompt_tokens": 9,
"completion_tokens": 228,
"tokens_per_sec": 5.4411378660239125,
"finish_reason": "stop",
"reasoning_chars": 849,
"content_chars": 134,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "hello",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 10852,
"prompt_tokens": 9,
"completion_tokens": 72,
"tokens_per_sec": 6.634721710283818,
"finish_reason": "stop",
"reasoning_chars": 251,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "hello",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 9407,
"prompt_tokens": 9,
"completion_tokens": 62,
"tokens_per_sec": 6.590836611034336,
"finish_reason": "stop",
"reasoning_chars": 186,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "hello",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 18374,
"prompt_tokens": 9,
"completion_tokens": 109,
"tokens_per_sec": 5.932295635136606,
"finish_reason": "stop",
"reasoning_chars": 446,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-MEDIUM",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 64718,
"prompt_tokens": 22,
"completion_tokens": 367,
"tokens_per_sec": 5.67075620383819,
"finish_reason": "stop",
"reasoning_chars": 1297,
"content_chars": 506,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-MEDIUM",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 96029,
"prompt_tokens": 22,
"completion_tokens": 512,
"tokens_per_sec": 5.331722708765061,
"finish_reason": "length",
"reasoning_chars": 2166,
"content_chars": 209,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-MEDIUM",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 93926,
"prompt_tokens": 22,
"completion_tokens": 467,
"tokens_per_sec": 4.971999233439091,
"finish_reason": "stop",
"reasoning_chars": 1889,
"content_chars": 433,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-MEDIUM",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 70212,
"prompt_tokens": 22,
"completion_tokens": 371,
"tokens_per_sec": 5.28399703754344,
"finish_reason": "stop",
"reasoning_chars": 1342,
"content_chars": 491,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-HARD",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 149004,
"prompt_tokens": 30,
"completion_tokens": 774,
"tokens_per_sec": 5.194491423049046,
"finish_reason": "stop",
"reasoning_chars": 1519,
"content_chars": 1719,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-HARD",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 92482,
"prompt_tokens": 30,
"completion_tokens": 492,
"tokens_per_sec": 5.319954153240631,
"finish_reason": "stop",
"reasoning_chars": 1005,
"content_chars": 1067,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-HARD",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 108851,
"prompt_tokens": 30,
"completion_tokens": 575,
"tokens_per_sec": 5.282450321999798,
"finish_reason": "stop",
"reasoning_chars": 1368,
"content_chars": 1007,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12",
"model_key": "ngl12",
"ngl": 12,
"prompt_id": "P-HARD",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 144405,
"prompt_tokens": 30,
"completion_tokens": 778,
"tokens_per_sec": 5.387625082233995,
"finish_reason": "stop",
"reasoning_chars": 1233,
"content_chars": 2214,
"error": null
}
]
},
{
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 62607.33,
"duration_ms_p50": 67049.0,
"tokens_per_sec_mean": 6.58,
"tokens_per_sec_max": 6.89,
"calls": [
{
"type": "vram_snapshot",
"model_key": "ngl24",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"ngl": 24,
"raw": "5883, 147, 6144"
},
{
"type": "llama_bench",
"model_key": "ngl24",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"ngl": 24,
"raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 24 | pp256 | 47.42 \u00b1 1.07 |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 24 | tg64 | 6.42 \u00b1 0.02 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "hello",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 11689,
"prompt_tokens": 9,
"completion_tokens": 67,
"tokens_per_sec": 5.731884677902301,
"finish_reason": "stop",
"reasoning_chars": 225,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "hello",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 14951,
"prompt_tokens": 9,
"completion_tokens": 103,
"tokens_per_sec": 6.889171292890108,
"finish_reason": "stop",
"reasoning_chars": 376,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "hello",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 10336,
"prompt_tokens": 9,
"completion_tokens": 71,
"tokens_per_sec": 6.8691950464396285,
"finish_reason": "stop",
"reasoning_chars": 235,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "hello",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 13118,
"prompt_tokens": 9,
"completion_tokens": 90,
"tokens_per_sec": 6.860801951517,
"finish_reason": "stop",
"reasoning_chars": 330,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-MEDIUM",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 67078,
"prompt_tokens": 22,
"completion_tokens": 441,
"tokens_per_sec": 6.57443573153642,
"finish_reason": "stop",
"reasoning_chars": 1696,
"content_chars": 448,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-MEDIUM",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 59196,
"prompt_tokens": 22,
"completion_tokens": 396,
"tokens_per_sec": 6.689641191972431,
"finish_reason": "stop",
"reasoning_chars": 1408,
"content_chars": 491,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-MEDIUM",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 71641,
"prompt_tokens": 22,
"completion_tokens": 479,
"tokens_per_sec": 6.6861154925252295,
"finish_reason": "stop",
"reasoning_chars": 1758,
"content_chars": 560,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-MEDIUM",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 67020,
"prompt_tokens": 22,
"completion_tokens": 450,
"tokens_per_sec": 6.714413607878246,
"finish_reason": "stop",
"reasoning_chars": 1772,
"content_chars": 425,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-HARD",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 119263,
"prompt_tokens": 30,
"completion_tokens": 763,
"tokens_per_sec": 6.397625416097196,
"finish_reason": "stop",
"reasoning_chars": 1316,
"content_chars": 2068,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-HARD",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 72039,
"prompt_tokens": 30,
"completion_tokens": 481,
"tokens_per_sec": 6.676938880328711,
"finish_reason": "stop",
"reasoning_chars": 1157,
"content_chars": 874,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-HARD",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 117297,
"prompt_tokens": 30,
"completion_tokens": 758,
"tokens_per_sec": 6.462228360486628,
"finish_reason": "stop",
"reasoning_chars": 1426,
"content_chars": 1729,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24",
"model_key": "ngl24",
"ngl": 24,
"prompt_id": "P-HARD",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 127660,
"prompt_tokens": 30,
"completion_tokens": 820,
"tokens_per_sec": 6.423311922293593,
"finish_reason": "stop",
"reasoning_chars": 1441,
"content_chars": 2092,
"error": null
}
]
}
],
"meta_record": {
"type": "meta",
"benchmark_run_id": "b37836bf-d1a2-4d6b-a732-aff89da1fa07",
"harness_version": "predator-a3b-ngl-matrix-1",
"started_at_utc": "2026-05-04T22:50:47Z",
"host": "Slobodans-MacBook-Air.local",
"node": "predator",
"engine": "llamacpp",
"purpose": "Sloba 2026-05-05: NGL retune (6/12/24) + 32k ctx for Qwen3-30B-A3B IQ2_M; pinpoint the offload-config sweet spot suspected after the 5fb2913d run showed Pavilion's --n-gpu-layers=12 beating Predator's --n-gpu-layers=99",
"ngl_values": [
6,
12,
24
],
"ctx_size": 32768,
"reasoning_budget": 500,
"gguf": "Qwen3-30B-A3B-UD-IQ2_M.gguf",
"sampling": "T=0.6 top_p=0.95 top_k=20 (Qwen3-A3B README defaults)",
"common_args": "--ctx-size 32768 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-budget 500 --reasoning-format deepseek --jinja --no-warmup --host 0.0.0.0 --port 11436"
},
"packaged_at": "2026-05-05T17:43:34Z"
}

43
runs/b37836bf-d1a2-4d6b-a732-aff89da1fa07/run.jsonl Normal file
View File

@@ -0,0 +1,43 @@
{"type": "meta", "benchmark_run_id": "b37836bf-d1a2-4d6b-a732-aff89da1fa07", "harness_version": "predator-a3b-ngl-matrix-1", "started_at_utc": "2026-05-04T22:50:47Z", "host": "Slobodans-MacBook-Air.local", "node": "predator", "engine": "llamacpp", "purpose": "Sloba 2026-05-05: NGL retune (6/12/24) + 32k ctx for Qwen3-30B-A3B IQ2_M; pinpoint the offload-config sweet spot suspected after the 5fb2913d run showed Pavilion's --n-gpu-layers=12 beating Predator's --n-gpu-layers=99", "ngl_values": [6, 12, 24], "ctx_size": 32768, "reasoning_budget": 500, "gguf": "Qwen3-30B-A3B-UD-IQ2_M.gguf", "sampling": "T=0.6 top_p=0.95 top_k=20 (Qwen3-A3B README defaults)", "common_args": "--ctx-size 32768 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-budget 500 --reasoning-format deepseek --jinja --no-warmup --host 0.0.0.0 --port 11436"}
{"type": "vram_snapshot", "model_key": "ngl6", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "ngl": 6, "raw": "2984, 3046, 6144"}
{"type": "llama_bench", "model_key": "ngl6", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "ngl": 6, "raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 6 | pp256 | 110.34 \u00b1 3.41 |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 6 | tg64 | 5.23 \u00b1 0.67 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "hello", "run_idx": 0, "phase": "cold", "total_duration_ms": 23904, "prompt_tokens": 9, "completion_tokens": 114, "tokens_per_sec": 4.769076305220883, "finish_reason": "stop", "reasoning_chars": 433, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "hello", "run_idx": 1, "phase": "warm", "total_duration_ms": 19868, "prompt_tokens": 9, "completion_tokens": 94, "tokens_per_sec": 4.731226092208577, "finish_reason": "stop", "reasoning_chars": 349, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "hello", "run_idx": 2, "phase": "warm", "total_duration_ms": 22272, "prompt_tokens": 9, "completion_tokens": 105, "tokens_per_sec": 4.714439655172414, "finish_reason": "stop", "reasoning_chars": 394, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "hello", "run_idx": 3, "phase": "warm", "total_duration_ms": 32650, "prompt_tokens": 9, "completion_tokens": 168, "tokens_per_sec": 5.145482388973966, "finish_reason": "stop", "reasoning_chars": 656, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-MEDIUM", "run_idx": 0, "phase": "cold", "total_duration_ms": 108039, "prompt_tokens": 22, "completion_tokens": 512, "tokens_per_sec": 4.73902942455965, "finish_reason": "length", "reasoning_chars": 2049, "content_chars": 513, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-MEDIUM", "run_idx": 1, "phase": "warm", "total_duration_ms": 71270, "prompt_tokens": 22, "completion_tokens": 350, "tokens_per_sec": 4.910902202890417, "finish_reason": "stop", "reasoning_chars": 1313, "content_chars": 403, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-MEDIUM", "run_idx": 2, "phase": "warm", "total_duration_ms": 92416, "prompt_tokens": 22, "completion_tokens": 449, "tokens_per_sec": 4.858466066481995, "finish_reason": "stop", "reasoning_chars": 1718, "content_chars": 477, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-MEDIUM", "run_idx": 3, "phase": "warm", "total_duration_ms": 100578, "prompt_tokens": 22, "completion_tokens": 512, "tokens_per_sec": 5.0905764680148735, "finish_reason": "length", "reasoning_chars": 2008, "content_chars": 541, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-HARD", "run_idx": 0, "phase": "cold", "total_duration_ms": 155414, "prompt_tokens": 30, "completion_tokens": 786, "tokens_per_sec": 5.057459430939298, "finish_reason": "stop", "reasoning_chars": 1713, "content_chars": 1659, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-HARD", "run_idx": 1, "phase": "warm", "total_duration_ms": 92550, "prompt_tokens": 30, "completion_tokens": 476, "tokens_per_sec": 5.143165856293895, "finish_reason": "stop", "reasoning_chars": 1080, "content_chars": 1010, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-HARD", "run_idx": 2, "phase": "warm", "total_duration_ms": 153066, "prompt_tokens": 30, "completion_tokens": 784, "tokens_per_sec": 5.121973527759267, "finish_reason": "stop", "reasoning_chars": 2134, "content_chars": 1183, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl6", "model_key": "ngl6", "ngl": 6, "prompt_id": "P-HARD", "run_idx": 3, "phase": "warm", "total_duration_ms": 153210, "prompt_tokens": 30, "completion_tokens": 789, "tokens_per_sec": 5.149794399843352, "finish_reason": "stop", "reasoning_chars": 1316, "content_chars": 1947, "error": null}
{"type": "vram_snapshot", "model_key": "ngl12", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "ngl": 12, "raw": "4708, 1322, 6144"}
{"type": "llama_bench", "model_key": "ngl12", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "ngl": 12, "raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 12 | pp256 | 121.86 \u00b1 3.89 |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 12 | tg64 | 6.37 \u00b1 0.63 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "hello", "run_idx": 0, "phase": "cold", "total_duration_ms": 41903, "prompt_tokens": 9, "completion_tokens": 228, "tokens_per_sec": 5.4411378660239125, "finish_reason": "stop", "reasoning_chars": 849, "content_chars": 134, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "hello", "run_idx": 1, "phase": "warm", "total_duration_ms": 10852, "prompt_tokens": 9, "completion_tokens": 72, "tokens_per_sec": 6.634721710283818, "finish_reason": "stop", "reasoning_chars": 251, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "hello", "run_idx": 2, "phase": "warm", "total_duration_ms": 9407, "prompt_tokens": 9, "completion_tokens": 62, "tokens_per_sec": 6.590836611034336, "finish_reason": "stop", "reasoning_chars": 186, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "hello", "run_idx": 3, "phase": "warm", "total_duration_ms": 18374, "prompt_tokens": 9, "completion_tokens": 109, "tokens_per_sec": 5.932295635136606, "finish_reason": "stop", "reasoning_chars": 446, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-MEDIUM", "run_idx": 0, "phase": "cold", "total_duration_ms": 64718, "prompt_tokens": 22, "completion_tokens": 367, "tokens_per_sec": 5.67075620383819, "finish_reason": "stop", "reasoning_chars": 1297, "content_chars": 506, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-MEDIUM", "run_idx": 1, "phase": "warm", "total_duration_ms": 96029, "prompt_tokens": 22, "completion_tokens": 512, "tokens_per_sec": 5.331722708765061, "finish_reason": "length", "reasoning_chars": 2166, "content_chars": 209, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-MEDIUM", "run_idx": 2, "phase": "warm", "total_duration_ms": 93926, "prompt_tokens": 22, "completion_tokens": 467, "tokens_per_sec": 4.971999233439091, "finish_reason": "stop", "reasoning_chars": 1889, "content_chars": 433, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-MEDIUM", "run_idx": 3, "phase": "warm", "total_duration_ms": 70212, "prompt_tokens": 22, "completion_tokens": 371, "tokens_per_sec": 5.28399703754344, "finish_reason": "stop", "reasoning_chars": 1342, "content_chars": 491, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-HARD", "run_idx": 0, "phase": "cold", "total_duration_ms": 149004, "prompt_tokens": 30, "completion_tokens": 774, "tokens_per_sec": 5.194491423049046, "finish_reason": "stop", "reasoning_chars": 1519, "content_chars": 1719, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-HARD", "run_idx": 1, "phase": "warm", "total_duration_ms": 92482, "prompt_tokens": 30, "completion_tokens": 492, "tokens_per_sec": 5.319954153240631, "finish_reason": "stop", "reasoning_chars": 1005, "content_chars": 1067, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-HARD", "run_idx": 2, "phase": "warm", "total_duration_ms": 108851, "prompt_tokens": 30, "completion_tokens": 575, "tokens_per_sec": 5.282450321999798, "finish_reason": "stop", "reasoning_chars": 1368, "content_chars": 1007, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl12", "model_key": "ngl12", "ngl": 12, "prompt_id": "P-HARD", "run_idx": 3, "phase": "warm", "total_duration_ms": 144405, "prompt_tokens": 30, "completion_tokens": 778, "tokens_per_sec": 5.387625082233995, "finish_reason": "stop", "reasoning_chars": 1233, "content_chars": 2214, "error": null}
{"type": "vram_snapshot", "model_key": "ngl24", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "ngl": 24, "raw": "5883, 147, 6144"}
{"type": "llama_bench", "model_key": "ngl24", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "ngl": 24, "raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 24 | pp256 | 47.42 \u00b1 1.07 |\r\n| qwen3moe 30B.A3B IQ2_M - 2.7 bpw | 10.11 GiB | 30.53 B | CUDA | 24 | tg64 | 6.42 \u00b1 0.02 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "hello", "run_idx": 0, "phase": "cold", "total_duration_ms": 11689, "prompt_tokens": 9, "completion_tokens": 67, "tokens_per_sec": 5.731884677902301, "finish_reason": "stop", "reasoning_chars": 225, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "hello", "run_idx": 1, "phase": "warm", "total_duration_ms": 14951, "prompt_tokens": 9, "completion_tokens": 103, "tokens_per_sec": 6.889171292890108, "finish_reason": "stop", "reasoning_chars": 376, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "hello", "run_idx": 2, "phase": "warm", "total_duration_ms": 10336, "prompt_tokens": 9, "completion_tokens": 71, "tokens_per_sec": 6.8691950464396285, "finish_reason": "stop", "reasoning_chars": 235, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "hello", "run_idx": 3, "phase": "warm", "total_duration_ms": 13118, "prompt_tokens": 9, "completion_tokens": 90, "tokens_per_sec": 6.860801951517, "finish_reason": "stop", "reasoning_chars": 330, "content_chars": 37, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-MEDIUM", "run_idx": 0, "phase": "cold", "total_duration_ms": 67078, "prompt_tokens": 22, "completion_tokens": 441, "tokens_per_sec": 6.57443573153642, "finish_reason": "stop", "reasoning_chars": 1696, "content_chars": 448, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-MEDIUM", "run_idx": 1, "phase": "warm", "total_duration_ms": 59196, "prompt_tokens": 22, "completion_tokens": 396, "tokens_per_sec": 6.689641191972431, "finish_reason": "stop", "reasoning_chars": 1408, "content_chars": 491, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-MEDIUM", "run_idx": 2, "phase": "warm", "total_duration_ms": 71641, "prompt_tokens": 22, "completion_tokens": 479, "tokens_per_sec": 6.6861154925252295, "finish_reason": "stop", "reasoning_chars": 1758, "content_chars": 560, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-MEDIUM", "run_idx": 3, "phase": "warm", "total_duration_ms": 67020, "prompt_tokens": 22, "completion_tokens": 450, "tokens_per_sec": 6.714413607878246, "finish_reason": "stop", "reasoning_chars": 1772, "content_chars": 425, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-HARD", "run_idx": 0, "phase": "cold", "total_duration_ms": 119263, "prompt_tokens": 30, "completion_tokens": 763, "tokens_per_sec": 6.397625416097196, "finish_reason": "stop", "reasoning_chars": 1316, "content_chars": 2068, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-HARD", "run_idx": 1, "phase": "warm", "total_duration_ms": 72039, "prompt_tokens": 30, "completion_tokens": 481, "tokens_per_sec": 6.676938880328711, "finish_reason": "stop", "reasoning_chars": 1157, "content_chars": 874, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-HARD", "run_idx": 2, "phase": "warm", "total_duration_ms": 117297, "prompt_tokens": 30, "completion_tokens": 758, "tokens_per_sec": 6.462228360486628, "finish_reason": "stop", "reasoning_chars": 1426, "content_chars": 1729, "error": null}
{"type": "call", "cell_id": "predator:llamacpp:qwen3:30b-a3b-iq2m-think500-ctx32k-ngl24", "model_key": "ngl24", "ngl": 24, "prompt_id": "P-HARD", "run_idx": 3, "phase": "warm", "total_duration_ms": 127660, "prompt_tokens": 30, "completion_tokens": 820, "tokens_per_sec": 6.423311922293593, "finish_reason": "stop", "reasoning_chars": 1441, "content_chars": 2092, "error": null}

66
runs/b37836bf-d1a2-4d6b-a732-aff89da1fa07/run.log Normal file
View File

@@ -0,0 +1,66 @@
[22:50:44Z] === predator-a3b-ngl-matrix run_id=b37836bf-d1a2-4d6b-a732-aff89da1fa07 ===
[22:50:44Z] Output: /Users/slobodan/projects/WeeyugaWeb/docs/BENCHMARKS/runs/b37836bf-d1a2-4d6b-a732-aff89da1fa07.jsonl
[22:50:44Z] Mirror: predator:D:\WeeyugaBench\runs\b37836bf-d1a2-4d6b-a732-aff89da1fa07\
[22:50:44Z] Plan: NGL=6 12 24 on ctx=32768
[22:50:44Z] Creating run-mirror dir on Predator: D:\WeeyugaBench\runs\b37836bf-d1a2-4d6b-a732-aff89da1fa07
[22:50:47Z] >>> CELL: NGL=6
[22:50:47Z] Stopping llama-server on Predator...
[22:50:51Z] Starting llama-server with Qwen3-30B-A3B-UD-IQ2_M.gguf, NGL=6, ctx=32768...
[22:50:54Z] Waiting for llama-server (NGL=6, up to 12 min — 32k KV cache + 30B mmap takes a while)...
[22:51:28Z] llama-server up after 25s
[22:51:29Z] Running llama-bench for ngl6...
[22:52:52Z] Running prompts for NGL=6 (cold + 3 warm per prompt)...
[22:52:52Z] prompt=hello cold
[22:53:16Z] prompt=hello warm 1
[22:53:36Z] prompt=hello warm 2
[22:53:58Z] prompt=hello warm 3
[22:54:31Z] prompt=P-MEDIUM cold
[22:56:19Z] prompt=P-MEDIUM warm 1
[22:57:31Z] prompt=P-MEDIUM warm 2
[22:59:03Z] prompt=P-MEDIUM warm 3
[23:00:44Z] prompt=P-HARD cold
[23:03:19Z] prompt=P-HARD warm 1
[23:04:52Z] prompt=P-HARD warm 2
[23:07:25Z] prompt=P-HARD warm 3
[23:09:58Z] >>> CELL: NGL=12
[23:09:58Z] Stopping llama-server on Predator...
[23:10:03Z] Starting llama-server with Qwen3-30B-A3B-UD-IQ2_M.gguf, NGL=12, ctx=32768...
[23:10:06Z] Waiting for llama-server (NGL=12, up to 12 min — 32k KV cache + 30B mmap takes a while)...
[23:10:13Z] llama-server up after 6s
[23:10:14Z] Running llama-bench for ngl12...
[23:11:25Z] Running prompts for NGL=12 (cold + 3 warm per prompt)...
[23:11:25Z] prompt=hello cold
[23:12:07Z] prompt=hello warm 1
[23:12:18Z] prompt=hello warm 2
[23:12:27Z] prompt=hello warm 3
[23:12:46Z] prompt=P-MEDIUM cold
[23:13:51Z] prompt=P-MEDIUM warm 1
[23:15:27Z] prompt=P-MEDIUM warm 2
[23:17:01Z] prompt=P-MEDIUM warm 3
[23:18:11Z] prompt=P-HARD cold
[23:20:40Z] prompt=P-HARD warm 1
[23:22:13Z] prompt=P-HARD warm 2
[23:24:02Z] prompt=P-HARD warm 3
[23:26:26Z] >>> CELL: NGL=24
[23:26:26Z] Stopping llama-server on Predator...
[23:26:31Z] Starting llama-server with Qwen3-30B-A3B-UD-IQ2_M.gguf, NGL=24, ctx=32768...
[23:26:34Z] Waiting for llama-server (NGL=24, up to 12 min — 32k KV cache + 30B mmap takes a while)...
[23:26:44Z] llama-server up after 9s
[23:26:45Z] Running llama-bench for ngl24...
[23:28:17Z] Running prompts for NGL=24 (cold + 3 warm per prompt)...
[23:28:17Z] prompt=hello cold
[23:28:28Z] prompt=hello warm 1
[23:28:44Z] prompt=hello warm 2
[23:28:54Z] prompt=hello warm 3
[23:29:07Z] prompt=P-MEDIUM cold
[23:30:14Z] prompt=P-MEDIUM warm 1
[23:31:14Z] prompt=P-MEDIUM warm 2
[23:32:25Z] prompt=P-MEDIUM warm 3
[23:33:33Z] prompt=P-HARD cold
[23:35:32Z] prompt=P-HARD warm 1
[23:36:44Z] prompt=P-HARD warm 2
[23:38:41Z] prompt=P-HARD warm 3
[23:40:49Z] Stopping llama-server on Predator...
[23:40:53Z] === bench complete ===
[23:40:53Z] Mirroring run-dir to Predator...
[23:41:01Z] On-device mirror at predator:D:\WeeyugaBench\runs\b37836bf-d1a2-4d6b-a732-aff89da1fa07\