model_key: granite

raw:
llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
At line:1 char:1
+ & 'D:\WeeyugaTools\llama.cpp\llama-bench.exe' -m 'D:\WeeyugaModels\ll ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB
load_backend: loaded CUDA backend from D:\WeeyugaTools\llama.cpp\ggml-cuda.dll
load_backend: loaded RPC backend from D:\WeeyugaTools\llama.cpp\ggml-rpc.dll
load_backend: loaded CPU backend from D:\WeeyugaTools\llama.cpp\ggml-cpu-haswell.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| granite 3B Q4_K - Medium       |   4.98 GiB |     8.79 B | CUDA       |  99 |           pp512 |        142.89 ± 0.51 |
| granite 3B Q4_K - Medium       |   4.98 GiB |     8.79 B | CUDA       |  99 |           tg128 |         12.19 ± 0.04 |

build: 665abc609 (8951)


model_key: gemma

raw:
llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
At line:1 char:1
+ & 'D:\WeeyugaTools\llama.cpp\llama-bench.exe' -m 'D:\WeeyugaModels\ll ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB
load_backend: loaded CUDA backend from D:\WeeyugaTools\llama.cpp\ggml-cuda.dll
load_backend: loaded RPC backend from D:\WeeyugaTools\llama.cpp\ggml-rpc.dll
load_backend: loaded CPU backend from D:\WeeyugaTools\llama.cpp\ggml-cpu-haswell.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma4 E4B Q4_K - Medium       |   4.62 GiB |     7.52 B | CUDA       |  99 |           pp512 |        498.70 ± 0.89 |
| gemma4 E4B Q4_K - Medium       |   4.62 GiB |     7.52 B | CUDA       |  99 |           tg128 |         24.53 ± 0.06 |

build: 665abc609 (8951)


model_key: qwen

raw:
llama-bench.exe : ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
At line:1 char:1
+ & 'D:\WeeyugaTools\llama.cpp\llama-bench.exe' -m 'D:\WeeyugaModels\ll ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ggml_cuda_init:...RAM: 6143 MiB)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  Device 0: NVIDIA GeForce GTX 1060, compute capability 6.1, VMM: yes, VRAM: 6143 MiB
load_backend: loaded CUDA backend from D:\WeeyugaTools\llama.cpp\ggml-cuda.dll
load_backend: loaded RPC backend from D:\WeeyugaTools\llama.cpp\ggml-rpc.dll
load_backend: loaded CPU backend from D:\WeeyugaTools\llama.cpp\ggml-cpu-haswell.dll
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 9B Q4_K - Medium        |   5.28 GiB |     8.95 B | CUDA       |  99 |           pp512 |         63.64 ± 0.14 |
| qwen35 9B Q4_K - Medium        |   5.28 GiB |     8.95 B | CUDA       |  99 |           tg128 |          7.43 ± 0.01 |

build: 665abc609 (8951)