slobodanmargetic988/weeyuga-benchmarks-public
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
feature/runner-and-agent-instructions
weeyuga-benchmarks-public/runs/e08a7051-a856-4053-b34b-561d7ce5e8cf/run.log
slobodanmargetic988 a18db6a3da B3 staging seed — 21 runs + catalogue v1.0-draft + methodology + README 
Initial population of the weeyuga-benchmarks-public archive (PRIVATE
staging visibility — flips public after Miljan + Stevan security audit
sign-off per Sloba's 17:34Z dispatch).

Contents:
- README.md       — public-facing intro (warns staging state, schema overview, citation pattern, license split)
- LICENSE         — CC-BY-4.0 default (auto-init from Gitea)
- catalogue.json  — schema_version=1.0-draft (locked once Tomas ratifies); 21 benchmarks indexed, 13 complete + 8 meta-only
- methodology.md  — mirror of WeeyugaWeb docs/BENCHMARKS/HARNESS.md (canonical methodology)
- runs/<id>/run.jsonl|run.log|run.md|metadata.json — packaged copies of every run in WeeyugaWeb docs/BENCHMARKS/runs/*

Run set covers:
- Mission 1 (2026-04-28/29): pavilion-weeyuga-v1 + reconstructed v3 (96 calls, 16 models routed via weeyuga :11435)
- Predator trio (2026-05-04): granite-4.1-8B + gemma-4-E4B-it + qwen3.5-9B
- Predator qwen rerun (2026-05-04): qwen3.5-9B think500/nothink + qwen3-14B feasibility
- A3B campaign (2026-05-04/05): pavilion-a3b + predator-a3b NGL matrix + ctx sweep + NGL+ctx 2D + NGL=6 deep dive
- VPS50 CPU matrix + gemma-e4b CPU lane (2026-05-04/05)

Visibility GATE: this repo stays private until Miljan G1-G4 audit and
Stevan G3 credential audit both green. After sign-off, single API call
flips visibility=public, anonymous read on, push-protection requires
auth, issues moderate by default.

No raw IPs, no SSH user@host strings, no /Users/ paths, no whisper
transcripts in any of these files. Hardware names (pavilion, predator,
vps50) are intentional and fine to share.

Builder: WeeyugaWeb/scripts/benchmarks/build_catalogue.py (deterministic,
idempotent, ~5s wall on 21 runs).
Publish flow: WeeyugaWeb/scripts/benchmarks/publish_bench_run.py
(builds packaged dirs, regenerates catalogue, optional --push to mirror
into this repo, optional --deploy stub for cicd rsync).

Owner: mac/benchmark-tester-ben (Ben).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 19:46:01 +02:00

123 lines
6.2 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
[07:42:54Z] === predator-a3b-ngl-ctx-2d run_id=e08a7051-a856-4053-b34b-561d7ce5e8cf ===
[07:42:54Z] Output: /Users/slobodan/projects/WeeyugaWeb/docs/BENCHMARKS/runs/e08a7051-a856-4053-b34b-561d7ce5e8cf.jsonl
[07:42:54Z] Plan: 6 cells (NGL=12 × ctx=64k/96k/131k, NGL=24 × ctx=48k/64k/96k)
[07:42:54Z] Creating run-mirror dir on Predator: D:\WeeyugaBench\runs\e08a7051-a856-4053-b34b-561d7ce5e8cf
[07:43:00Z] >>> CELL: ngl12-ctx64k (NGL=12 ctx=65536)
[07:43:00Z] Stopping llama-server on Predator...
[07:43:04Z] Starting llama-server (key=ngl12-ctx64k, NGL=12, ctx=65536)...
[07:43:07Z] Waiting for llama-server (key=ngl12-ctx64k, up to 15 min — large KV allocation)...
[07:43:14Z] llama-server up after 6s
[07:43:17Z] Running llama-bench for ngl12-ctx64k (NGL=12)...
[07:44:29Z] Running prompts for ngl12-ctx64k (cold + 3 warm per prompt; max_tokens > reasoning_budget=500)...
[07:44:29Z] prompt=hello cold (max_tokens=512)
[07:44:55Z] prompt=hello warm 1
[07:45:05Z] prompt=hello warm 2
[07:45:21Z] prompt=hello warm 3
[07:45:45Z] prompt=P-MEDIUM cold (max_tokens=1024)
[07:47:36Z] prompt=P-MEDIUM warm 1
[07:48:52Z] prompt=P-MEDIUM warm 2
[07:50:03Z] prompt=P-MEDIUM warm 3
[07:51:25Z] prompt=P-HARD cold (max_tokens=2048)
[07:53:47Z] prompt=P-HARD warm 1
[07:56:17Z] prompt=P-HARD warm 2
[07:58:59Z] prompt=P-HARD warm 3
[08:01:37Z] >>> CELL: ngl12-ctx96k (NGL=12 ctx=98304)
[08:01:37Z] Stopping llama-server on Predator...
[08:01:42Z] Starting llama-server (key=ngl12-ctx96k, NGL=12, ctx=98304)...
[08:01:45Z] Waiting for llama-server (key=ngl12-ctx96k, up to 15 min — large KV allocation)...
[08:01:53Z] llama-server up after 7s
[08:01:57Z] Running llama-bench for ngl12-ctx96k (NGL=12)...
[08:03:13Z] Running prompts for ngl12-ctx96k (cold + 3 warm per prompt; max_tokens > reasoning_budget=500)...
[08:03:13Z] prompt=hello cold (max_tokens=512)
[08:03:27Z] prompt=hello warm 1
[08:04:00Z] prompt=hello warm 2
[08:04:16Z] prompt=hello warm 3
[08:04:37Z] prompt=P-MEDIUM cold (max_tokens=1024)
[08:05:22Z] prompt=P-MEDIUM warm 1
[08:06:22Z] prompt=P-MEDIUM warm 2
[08:07:20Z] prompt=P-MEDIUM warm 3
[08:09:04Z] prompt=P-HARD cold (max_tokens=2048)
[08:11:22Z] prompt=P-HARD warm 1
[08:13:46Z] prompt=P-HARD warm 2
[08:15:28Z] prompt=P-HARD warm 3
[08:17:36Z] >>> CELL: ngl12-ctx131k (NGL=12 ctx=131072)
[08:17:36Z] Stopping llama-server on Predator...
[08:17:41Z] Starting llama-server (key=ngl12-ctx131k, NGL=12, ctx=131072)...
[08:17:45Z] Waiting for llama-server (key=ngl12-ctx131k, up to 15 min — large KV allocation)...
[08:17:54Z] llama-server up after 7s
[08:17:57Z] Running llama-bench for ngl12-ctx131k (NGL=12)...
[08:21:09Z] Running prompts for ngl12-ctx131k (cold + 3 warm per prompt; max_tokens > reasoning_budget=500)...
[08:21:09Z] prompt=hello cold (max_tokens=512)
[08:22:09Z] prompt=hello warm 1
[08:22:19Z] prompt=hello warm 2
[08:22:40Z] prompt=hello warm 3
[08:23:23Z] prompt=P-MEDIUM cold (max_tokens=1024)
[08:25:22Z] prompt=P-MEDIUM warm 1
[08:27:18Z] prompt=P-MEDIUM warm 2
[08:28:54Z] prompt=P-MEDIUM warm 3
[08:30:18Z] prompt=P-HARD cold (max_tokens=2048)
[08:32:49Z] prompt=P-HARD warm 1
[08:35:18Z] prompt=P-HARD warm 2
[08:37:57Z] prompt=P-HARD warm 3
[08:40:27Z] >>> CELL: ngl24-ctx48k (NGL=24 ctx=49152)
[08:40:27Z] Stopping llama-server on Predator...
[08:40:31Z] Starting llama-server (key=ngl24-ctx48k, NGL=24, ctx=49152)...
[08:40:34Z] Waiting for llama-server (key=ngl24-ctx48k, up to 15 min — large KV allocation)...
[08:40:56Z] llama-server up after 18s
[08:40:59Z] Running llama-bench for ngl24-ctx48k (NGL=24)...
[08:42:30Z] Running prompts for ngl24-ctx48k (cold + 3 warm per prompt; max_tokens > reasoning_budget=500)...
[08:42:30Z] prompt=hello cold (max_tokens=512)
[08:42:50Z] prompt=hello warm 1
[08:43:05Z] prompt=hello warm 2
[08:43:21Z] prompt=hello warm 3
[08:43:36Z] prompt=P-MEDIUM cold (max_tokens=1024)
[08:44:34Z] prompt=P-MEDIUM warm 1
[08:46:04Z] prompt=P-MEDIUM warm 2
[08:47:37Z] prompt=P-MEDIUM warm 3
[08:48:37Z] prompt=P-HARD cold (max_tokens=2048)
[08:50:39Z] prompt=P-HARD warm 1
[08:51:48Z] prompt=P-HARD warm 2
[08:53:42Z] prompt=P-HARD warm 3
[08:55:38Z] >>> CELL: ngl24-ctx64k (NGL=24 ctx=65536)
[08:55:38Z] Stopping llama-server on Predator...
[08:55:42Z] Starting llama-server (key=ngl24-ctx64k, NGL=24, ctx=65536)...
[08:55:45Z] Waiting for llama-server (key=ngl24-ctx64k, up to 15 min — large KV allocation)...
[08:55:54Z] llama-server up after 8s
[08:55:57Z] Running llama-bench for ngl24-ctx64k (NGL=24)...
[08:58:08Z] Running prompts for ngl24-ctx64k (cold + 3 warm per prompt; max_tokens > reasoning_budget=500)...
[08:58:08Z] prompt=hello cold (max_tokens=512)
[08:58:27Z] prompt=hello warm 1
[08:58:45Z] prompt=hello warm 2
[08:59:00Z] prompt=hello warm 3
[08:59:15Z] prompt=P-MEDIUM cold (max_tokens=1024)
[09:00:54Z] prompt=P-MEDIUM warm 1
[09:01:50Z] prompt=P-MEDIUM warm 2
[09:03:23Z] prompt=P-MEDIUM warm 3
[09:04:33Z] prompt=P-HARD cold (max_tokens=2048)
[09:06:36Z] prompt=P-HARD warm 1
[09:08:07Z] prompt=P-HARD warm 2
[09:10:19Z] prompt=P-HARD warm 3
[09:12:25Z] >>> CELL: ngl24-ctx96k (NGL=24 ctx=98304)
[09:12:25Z] Stopping llama-server on Predator...
[09:12:29Z] Starting llama-server (key=ngl24-ctx96k, NGL=24, ctx=98304)...
[09:12:31Z] Waiting for llama-server (key=ngl24-ctx96k, up to 15 min — large KV allocation)...
[09:12:48Z] llama-server up after 14s
[09:12:50Z] Running llama-bench for ngl24-ctx96k (NGL=24)...
[09:16:24Z] Running prompts for ngl24-ctx96k (cold + 3 warm per prompt; max_tokens > reasoning_budget=500)...
[09:16:24Z] prompt=hello cold (max_tokens=512)
[09:17:00Z] prompt=hello warm 1
[09:17:27Z] prompt=hello warm 2
[09:17:52Z] prompt=hello warm 3
[09:18:06Z] prompt=P-MEDIUM cold (max_tokens=1024)
[09:19:18Z] prompt=P-MEDIUM warm 1
[09:20:50Z] prompt=P-MEDIUM warm 2
[09:21:54Z] prompt=P-MEDIUM warm 3
[09:22:59Z] prompt=P-HARD cold (max_tokens=2048)
[09:25:06Z] prompt=P-HARD warm 1
[09:27:10Z] prompt=P-HARD warm 2
[09:28:24Z] prompt=P-HARD warm 3
[09:30:17Z] Stopping llama-server on Predator...
[09:30:22Z] === bench complete ===
[09:30:22Z] Mirroring run-dir to Predator...
[09:30:29Z] On-device mirror: predator:D:\WeeyugaBench\runs\e08a7051-a856-4053-b34b-561d7ce5e8cf\
Reference in New Issue View Git Blame Copy Permalink