Qwen3.8-27B benchmark on 3x RTX 5060 Ti 16GB on Ollama 0.32.15

I managed to get the model running with 256k context on my 3x RTX 5060 Ti 16GB setup.  Following Parameters were used:

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
Environment="OLLAMA_SCHED_SPREAD=1"
Environment="OLLAMA_CONTEXT_LENGTH=262144"

And the results using llama-benchy:

llama-benchy --base-url http://localhost:11434/v1 --model qwen3.8:27b --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model       |   test |           t/s |     peak t/s |       ttfr (ms) |    est_ppt (ms) |   e2e_ttft (ms) |
|:------------|-------:|--------------:|-------------:|----------------:|----------------:|----------------:|
| qwen3.8:27b | pp2048 | 645.39 ± 5.82 |              | 2979.82 ± 95.73 | 2738.63 ± 95.73 | 2979.82 ± 95.73 |
| qwen3.8:27b |  tg512 |  40.68 ± 1.89 | 41.33 ± 2.05 |                 |                 |                 |


llama-benchy --base-url http://localhost:11434/v1 --model qwen3.8:27b --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model       |        test |   t/s (total) |       t/s (req) |     peak t/s |   peak t/s (req) |           ttfr (ms) |        est_ppt (ms) |       e2e_ttft (ms) |
|:------------|------------:|--------------:|----------------:|-------------:|-----------------:|--------------------:|--------------------:|--------------------:|
| qwen3.8:27b | pp2048 (c4) | 139.64 ± 2.14 | 209.82 ± 259.08 |              |                  | 28291.80 ± 18803.48 | 28062.33 ± 18803.48 | 28291.80 ± 18803.48 |
| qwen3.8:27b |  tg512 (c4) |  32.37 ± 1.13 |    38.26 ± 3.17 | 42.67 ± 2.62 |     38.75 ± 3.11 |                     |                     |                     |


llama-benchy --base-url http://localhost:11434/v1 --model qwen3.8:27b --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model       |        test |   t/s (total) |       t/s (req) |     peak t/s |   peak t/s (req) |           ttfr (ms) |        est_ppt (ms) |
|:------------|------------:|--------------:|----------------:|-------------:|-----------------:|--------------------:|--------------------:|-------
| qwen3.8:27b | pp2048 (c8) | 125.29 ± 3.90 | 118.09 ± 211.66 |              |                  | 61020.08 ± 38195.67 | 60794.71 ± 38195.67 | 61020.
| qwen3.8:27b |  tg512 (c8) |  31.23 ± 1.19 |    37.53 ± 2.75 | 41.33 ± 1.25 |     37.96 ± 2.70 |                     |                     |

Next goal is to get the model running in vLLM and see how it compares.