type: meta

node: predator

engine: llamacpp

purpose: Sloba 2026-05-05: NGL retune (6/12/24) + 32k ctx for Qwen3-30B-A3B IQ2_M; pinpoint the offload-config sweet spot suspected after the 5fb2913d run showed Pavilion's --n-gpu-layers=12 beating Predator's --n-gpu-layers=99

sampling: T=0.6 top_p=0.95 top_k=20 (Qwen3-A3B README defaults)

common_args: --ctx-size 32768 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-budget 500 --reasoning-format deepseek --jinja --no-warmup --host 0.0.0.0 --port 11436


raw:
llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
At line:1 char:1
+ & 'D:\WeeyugaTools\llama.cpp\llama-bench.exe' -m 'D:\WeeyugaModels\ll ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB
load_backend: loaded CUDA backend from D:\WeeyugaTools\llama.cpp\ggml-cuda.dll
load_backend: loaded RPC backend from D:\WeeyugaTools\llama.cpp\ggml-rpc.dll
load_backend: loaded CPU backend from D:\WeeyugaTools\llama.cpp\ggml-cpu-haswell.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ2_M - 2.7 bpw |  10.11 GiB |    30.53 B | CUDA       |   6 |           pp256 |        110.34 ± 3.41 |
| qwen3moe 30B.A3B IQ2_M - 2.7 bpw |  10.11 GiB |    30.53 B | CUDA       |   6 |            tg64 |          5.23 ± 0.67 |

build: 665abc609 (8951)


raw:
llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
At line:1 char:1
+ & 'D:\WeeyugaTools\llama.cpp\llama-bench.exe' -m 'D:\WeeyugaModels\ll ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB
load_backend: loaded CUDA backend from D:\WeeyugaTools\llama.cpp\ggml-cuda.dll
load_backend: loaded RPC backend from D:\WeeyugaTools\llama.cpp\ggml-rpc.dll
load_backend: loaded CPU backend from D:\WeeyugaTools\llama.cpp\ggml-cpu-haswell.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ2_M - 2.7 bpw |  10.11 GiB |    30.53 B | CUDA       |  12 |           pp256 |        121.86 ± 3.89 |
| qwen3moe 30B.A3B IQ2_M - 2.7 bpw |  10.11 GiB |    30.53 B | CUDA       |  12 |            tg64 |          6.37 ± 0.63 |

build: 665abc609 (8951)


raw:
llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
At line:1 char:1
+ & 'D:\WeeyugaTools\llama.cpp\llama-bench.exe' -m 'D:\WeeyugaModels\ll ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB
load_backend: loaded CUDA backend from D:\WeeyugaTools\llama.cpp\ggml-cuda.dll
load_backend: loaded RPC backend from D:\WeeyugaTools\llama.cpp\ggml-rpc.dll
load_backend: loaded CPU backend from D:\WeeyugaTools\llama.cpp\ggml-cpu-haswell.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ2_M - 2.7 bpw |  10.11 GiB |    30.53 B | CUDA       |  24 |           pp256 |         47.42 ± 1.07 |
| qwen3moe 30B.A3B IQ2_M - 2.7 bpw |  10.11 GiB |    30.53 B | CUDA       |  24 |            tg64 |          6.42 ± 0.02 |

build: 665abc609 (8951)