slobodanmargetic988/weeyuga-benchmarks-public
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a18db6a3daadb5c3459933c3fa2319fa6de3764c
weeyuga-benchmarks-public/runs/fba9d9b1-cc5d-40bc-9e21-beafbb72c65d/metadata.json
slobodanmargetic988 a18db6a3da B3 staging seed — 21 runs + catalogue v1.0-draft + methodology + README 
Initial population of the weeyuga-benchmarks-public archive (PRIVATE
staging visibility — flips public after Miljan + Stevan security audit
sign-off per Sloba's 17:34Z dispatch).

Contents:
- README.md       — public-facing intro (warns staging state, schema overview, citation pattern, license split)
- LICENSE         — CC-BY-4.0 default (auto-init from Gitea)
- catalogue.json  — schema_version=1.0-draft (locked once Tomas ratifies); 21 benchmarks indexed, 13 complete + 8 meta-only
- methodology.md  — mirror of WeeyugaWeb docs/BENCHMARKS/HARNESS.md (canonical methodology)
- runs/<id>/run.jsonl|run.log|run.md|metadata.json — packaged copies of every run in WeeyugaWeb docs/BENCHMARKS/runs/*

Run set covers:
- Mission 1 (2026-04-28/29): pavilion-weeyuga-v1 + reconstructed v3 (96 calls, 16 models routed via weeyuga :11435)
- Predator trio (2026-05-04): granite-4.1-8B + gemma-4-E4B-it + qwen3.5-9B
- Predator qwen rerun (2026-05-04): qwen3.5-9B think500/nothink + qwen3-14B feasibility
- A3B campaign (2026-05-04/05): pavilion-a3b + predator-a3b NGL matrix + ctx sweep + NGL+ctx 2D + NGL=6 deep dive
- VPS50 CPU matrix + gemma-e4b CPU lane (2026-05-04/05)

Visibility GATE: this repo stays private until Miljan G1-G4 audit and
Stevan G3 credential audit both green. After sign-off, single API call
flips visibility=public, anonymous read on, push-protection requires
auth, issues moderate by default.

No raw IPs, no SSH user@host strings, no /Users/ paths, no whisper
transcripts in any of these files. Hardware names (pavilion, predator,
vps50) are intentional and fine to share.

Builder: WeeyugaWeb/scripts/benchmarks/build_catalogue.py (deterministic,
idempotent, ~5s wall on 21 runs).
Publish flow: WeeyugaWeb/scripts/benchmarks/publish_bench_run.py
(builds packaged dirs, regenerates catalogue, optional --push to mirror
into this repo, optional --deploy stub for cicd rsync).

Owner: mac/benchmark-tester-ben (Ben).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 19:46:01 +02:00

642 lines
24 KiB
JSON
Raw Blame History

{
"id": "fba9d9b1-cc5d-40bc-9e21-beafbb72c65d",
"title": "Predator Qwen rerun \u2014 fba9d9b1-cc5d-40bc-9e21-beafbb72c65d",
"date": "2026-05-04",
"hardware": "predator",
"engine": "llamacpp",
"harness": "predator-qwen-rerun-1",
"model_family": "qwen3/qwen3.5",
"model_sizes": [
"qwen3.5:9b",
"qwen3:14b"
],
"task_kind": null,
"headline": "`Answer chars` is `len(message.content)`. Both averaged over the 3 warm runs.",
"summary_md_path": "runs/fba9d9b1-cc5d-40bc-9e21-beafbb72c65d.md",
"raw_jsonl_path": "runs/fba9d9b1-cc5d-40bc-9e21-beafbb72c65d.jsonl",
"log_path": "runs/fba9d9b1-cc5d-40bc-9e21-beafbb72c65d.log",
"synthesis_doc": "A3B_AND_CPU_OVERNIGHT_2026-05-05.md",
"cells": [
{
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 30474.33,
"duration_ms_p50": 33717.5,
"tokens_per_sec_mean": 14.25,
"tokens_per_sec_max": 15.32
},
{
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 10425.75,
"duration_ms_p50": 7200.0,
"tokens_per_sec_mean": 12.55,
"tokens_per_sec_max": 14.95
},
{
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"n_calls": 8,
"n_errors": 0,
"duration_ms_mean": 130876.17,
"duration_ms_p50": 78946.0,
"tokens_per_sec_mean": 1.06,
"tokens_per_sec_max": 1.33
}
],
"n_calls_total": 36,
"n_errors_total": 0,
"started_at": "2026-05-04T17:49:07Z",
"tags": [
"predator",
"qwen3",
"qwen3.5",
"rerun"
],
"status": "complete",
"visibility": "draft",
"cells_full": [
{
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 30474.33,
"duration_ms_p50": 33717.5,
"tokens_per_sec_mean": 14.25,
"tokens_per_sec_max": 15.32,
"calls": [
{
"type": "vram_snapshot",
"model_key": "qwen-9b-think500",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"raw": "5887, 143, 6144"
},
{
"type": "llama_bench",
"model_key": "qwen-9b-think500",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CUDA | 99 | pp512 | 61.32 \u00b1 1.61 |\r\n| qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CUDA | 99 | tg128 | 7.68 \u00b1 0.06 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "hello",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 6773,
"prompt_tokens": 11,
"completion_tokens": 64,
"tokens_per_sec": 9.449283921452828,
"finish_reason": "length",
"reasoning_chars": 234,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "hello",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 4597,
"prompt_tokens": 11,
"completion_tokens": 64,
"tokens_per_sec": 13.922123123776375,
"finish_reason": "length",
"reasoning_chars": 227,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "hello",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 4632,
"prompt_tokens": 11,
"completion_tokens": 64,
"tokens_per_sec": 13.816925734024181,
"finish_reason": "length",
"reasoning_chars": 217,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "hello",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 4711,
"prompt_tokens": 11,
"completion_tokens": 64,
"tokens_per_sec": 13.585226066652515,
"finish_reason": "length",
"reasoning_chars": 222,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-MEDIUM",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 35949,
"prompt_tokens": 24,
"completion_tokens": 512,
"tokens_per_sec": 14.242398954073828,
"finish_reason": "length",
"reasoning_chars": 2316,
"content_chars": 46,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-MEDIUM",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 33849,
"prompt_tokens": 24,
"completion_tokens": 512,
"tokens_per_sec": 15.126000768117228,
"finish_reason": "length",
"reasoning_chars": 2069,
"content_chars": 52,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-MEDIUM",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 33586,
"prompt_tokens": 24,
"completion_tokens": 512,
"tokens_per_sec": 15.244447091049842,
"finish_reason": "length",
"reasoning_chars": 2201,
"content_chars": 47,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-MEDIUM",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 33484,
"prompt_tokens": 24,
"completion_tokens": 512,
"tokens_per_sec": 15.290885198900966,
"finish_reason": "length",
"reasoning_chars": 2182,
"content_chars": 37,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-HARD",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 53568,
"prompt_tokens": 32,
"completion_tokens": 797,
"tokens_per_sec": 14.878285543608126,
"finish_reason": "stop",
"reasoning_chars": 1998,
"content_chars": 1300,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-HARD",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 51802,
"prompt_tokens": 32,
"completion_tokens": 774,
"tokens_per_sec": 14.941508049882245,
"finish_reason": "stop",
"reasoning_chars": 1880,
"content_chars": 1248,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-HARD",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 51829,
"prompt_tokens": 32,
"completion_tokens": 788,
"tokens_per_sec": 15.203843408130583,
"finish_reason": "stop",
"reasoning_chars": 1960,
"content_chars": 1250,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"model_key": "qwen-9b-think500",
"prompt_id": "P-HARD",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 50912,
"prompt_tokens": 32,
"completion_tokens": 780,
"tokens_per_sec": 15.320553111250787,
"finish_reason": "stop",
"reasoning_chars": 1974,
"content_chars": 1230,
"error": null
}
]
},
{
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"n_calls": 14,
"n_errors": 0,
"duration_ms_mean": 10425.75,
"duration_ms_p50": 7200.0,
"tokens_per_sec_mean": 12.55,
"tokens_per_sec_max": 14.95,
"calls": [
{
"type": "vram_snapshot",
"model_key": "qwen-9b-nothink",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"raw": "5918, 112, 6144"
},
{
"type": "llama_bench",
"model_key": "qwen-9b-nothink",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CUDA | 99 | pp512 | 65.19 \u00b1 0.57 |\r\n| qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CUDA | 99 | tg128 | 7.56 \u00b1 0.02 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "hello",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 2372,
"prompt_tokens": 22,
"completion_tokens": 13,
"tokens_per_sec": 5.480607082630692,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 32,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "hello",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 1350,
"prompt_tokens": 22,
"completion_tokens": 13,
"tokens_per_sec": 9.62962962962963,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 32,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "hello",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 1436,
"prompt_tokens": 22,
"completion_tokens": 13,
"tokens_per_sec": 9.052924791086351,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 32,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "hello",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 1233,
"prompt_tokens": 22,
"completion_tokens": 12,
"tokens_per_sec": 9.7323600973236,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 26,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-MEDIUM",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 7338,
"prompt_tokens": 35,
"completion_tokens": 106,
"tokens_per_sec": 14.445352957209048,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 521,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-MEDIUM",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 8034,
"prompt_tokens": 35,
"completion_tokens": 116,
"tokens_per_sec": 14.438635797859098,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 597,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-MEDIUM",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 7062,
"prompt_tokens": 35,
"completion_tokens": 101,
"tokens_per_sec": 14.301897479467572,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 552,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-MEDIUM",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 6381,
"prompt_tokens": 35,
"completion_tokens": 90,
"tokens_per_sec": 14.104372355430183,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 467,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-HARD",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 20870,
"prompt_tokens": 43,
"completion_tokens": 308,
"tokens_per_sec": 14.758025874460948,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 1405,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-HARD",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 24820,
"prompt_tokens": 43,
"completion_tokens": 371,
"tokens_per_sec": 14.947622884770347,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 1604,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-HARD",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 23327,
"prompt_tokens": 43,
"completion_tokens": 348,
"tokens_per_sec": 14.918334976636514,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 1515,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"model_key": "qwen-9b-nothink",
"prompt_id": "P-HARD",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 20886,
"prompt_tokens": 43,
"completion_tokens": 310,
"tokens_per_sec": 14.842478215072298,
"finish_reason": "stop",
"reasoning_chars": 0,
"content_chars": 1360,
"error": null
}
]
},
{
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"n_calls": 8,
"n_errors": 0,
"duration_ms_mean": 130876.17,
"duration_ms_p50": 78946.0,
"tokens_per_sec_mean": 1.06,
"tokens_per_sec_max": 1.33,
"calls": [
{
"type": "vram_snapshot",
"model_key": "qwen3-14b",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"raw": "5883, 147, 6144"
},
{
"type": "llama_bench",
"model_key": "qwen3-14b",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"raw": "llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):\r\nAt line:1 char:1\r\n+ & 'D:\\WeeyugaTools\\llama.cpp\\llama-bench.exe' -m 'D:\\WeeyugaModels\\ll ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException\r\n + FullyQualifiedErrorId : NativeCommandError\r\n \r\n Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB\r\nload_backend: loaded CUDA backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cuda.dll\r\nload_backend: loaded RPC backend from D:\\WeeyugaTools\\llama.cpp\\ggml-rpc.dll\r\nload_backend: loaded CPU backend from D:\\WeeyugaTools\\llama.cpp\\ggml-cpu-haswell.dll\r\n| model | size | params | backend | ngl | test | t/s |\r\n| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |\r\n| qwen3 14B Q4_K - Medium | 8.38 GiB | 14.77 B | CUDA | 99 | pp512 | 52.96 \u00b1 4.25 |\r\n| qwen3 14B Q4_K - Medium | 8.38 GiB | 14.77 B | CUDA | 99 | tg128 | 0.86 \u00b1 0.00 |\r\n\r\nbuild: 665abc609 (8951)\r\n\r"
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"model_key": "qwen3-14b",
"prompt_id": "hello",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 98037,
"prompt_tokens": 9,
"completion_tokens": 64,
"tokens_per_sec": 0.6528147536134317,
"finish_reason": "length",
"reasoning_chars": 279,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"model_key": "qwen3-14b",
"prompt_id": "hello",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 59855,
"prompt_tokens": 9,
"completion_tokens": 64,
"tokens_per_sec": 1.0692506891654834,
"finish_reason": "length",
"reasoning_chars": 293,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"model_key": "qwen3-14b",
"prompt_id": "hello",
"run_idx": 2,
"phase": "warm",
"total_duration_ms": 58654,
"prompt_tokens": 9,
"completion_tokens": 64,
"tokens_per_sec": 1.0911446789647765,
"finish_reason": "length",
"reasoning_chars": 276,
"content_chars": 0,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"model_key": "qwen3-14b",
"prompt_id": "hello",
"run_idx": 3,
"phase": "warm",
"total_duration_ms": 58891,
"prompt_tokens": 9,
"completion_tokens": 64,
"tokens_per_sec": 1.0867534937426773,
"finish_reason": "length",
"reasoning_chars": 222,
"content_chars": 34,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"model_key": "qwen3-14b",
"prompt_id": "P-MEDIUM",
"run_idx": 0,
"phase": "cold",
"total_duration_ms": 299410,
"prompt_tokens": 22,
"completion_tokens": 342,
"tokens_per_sec": 1.142246417955312,
"finish_reason": "stop",
"reasoning_chars": 1118,
"content_chars": 618,
"error": null
},
{
"type": "call",
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"model_key": "qwen3-14b",
"prompt_id": "P-MEDIUM",
"run_idx": 1,
"phase": "warm",
"total_duration_ms": 210410,
"prompt_tokens": 22,
"completion_tokens": 280,
"tokens_per_sec": 1.3307352312152465,
"finish_reason": "stop",
"reasoning_chars": 839,
"content_chars": 554,
"error": null
}
]
}
],
"meta_record": {
"type": "meta",
"benchmark_run_id": "fba9d9b1-cc5d-40bc-9e21-beafbb72c65d",
"harness_version": "predator-qwen-rerun-1",
"started_at_utc": "2026-05-04T17:49:07Z",
"host": "Slobodans-MacBook-Air.local",
"node": "predator",
"engine": "llamacpp",
"predator_target_url": "http://10.8.0.7:11436",
"purpose": "Sloba's follow-up to trio-bench: budget-capped thinking, thinking-off comparison, 14B-on-6GB-GPU feasibility test",
"models": {
"qwen-9b-think500": {
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-think500",
"gguf": "Qwen3.5-9B-Q4_K_M.gguf",
"reasoning_budget": 500
},
"qwen-9b-nothink": {
"cell_id": "predator:llamacpp:qwen3.5:9b-q4km-nothink",
"gguf": "Qwen3.5-9B-Q4_K_M.gguf",
"reasoning_budget": 0,
"system_msg": "Answer directly without showing reasoning."
},
"qwen3-14b": {
"cell_id": "predator:llamacpp:qwen3:14b-q4km",
"gguf": "Qwen3-14B-Q4_K_M.gguf",
"reasoning_budget": 1024,
"note": "8.4GB Q4_K_M on 6GB GPU; partial CPU offload expected"
}
}
},
"packaged_at": "2026-05-05T17:43:34Z"
}
Reference in New Issue View Git Blame Copy Permalink