Deepseek-v4.1-Flash-1017 benchmark on 3x RTX 5060 Ti 16GB with Dual Intel Xeon E5-2683 v4

Using Project Maya (https://github.com/mw00/project-maya) using deepseek-maya-xs With 1M context

llama-benchy --base-url http://localhost:8080/v1 --model deepseek-v4.1-flash --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model               |   test |                 t/s |     peak t/s |    ttfr (ms) |   est_ppt (ms) |     e2e_ttft (ms) |
|:--------------------|-------:|--------------------:|-------------:|-------------:|---------------:|------------------:|
| deepseek-v4.1-flash | pp2048 | 161102.16 ± 9350.12 |              | 16.45 ± 0.88 |   11.82 ± 0.88 | 11712.44 ± 247.77 |
| deepseek-v4.1-flash |  tg512 |        29.56 ± 1.54 | 30.33 ± 1.70 |              |                |                   |

llama-benchy --base-url http://localhost:8080/v1 --model deepseek-v4.1-flash --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model               |        test |   t/s (total) |           t/s (req) |     peak t/s |   peak t/s (req) |     ttfr (ms) |   est_ppt (ms) |       e2e_ttft (ms) |
|:--------------------|------------:|--------------:|--------------------:|-------------:|-----------------:|--------------:|---------------:|--------------------:|
| deepseek-v4.1-flash | pp2048 (c4) |  88.81 ± 1.43 | 47116.81 ± 14605.95 |              |                  | 46.70 ± 12.56 |  42.59 ± 12.56 | 47575.89 ± 26684.74 |
| deepseek-v4.1-flash |  tg512 (c4) |  18.29 ± 0.67 |        31.79 ± 2.47 | 34.33 ± 0.94 |     32.17 ± 2.58 |               |                |                     |


llama-benchy --base-url http://localhost:8080/v1 --model deepseek-v4.1-flash --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model               |        test |   t/s (total) |          t/s (req) |     peak t/s |   peak t/s (req) |     ttfr (ms) |   est_ppt (ms) |       e2e_ttft (ms) |
|:--------------------|------------:|--------------:|-------------------:|-------------:|-----------------:|--------------:|---------------:|--------------------:|
| deepseek-v4.1-flash | pp2048 (c8) |  80.61 ± 0.93 | 29333.59 ± 9435.68 |              |                  | 72.39 ± 18.65 |  68.07 ± 18.65 | 96932.54 ± 55296.54 |
| deepseek-v4.1-flash |  tg512 (c8) |  17.73 ± 0.53 |       31.21 ± 1.73 | 34.00 ± 0.82 |     31.71 ± 1.72 |               |                |                     |

Detect and read DIMM memory module info in linux

 sudo dmidecode -t memory

Ornith-1.5-35B-A3B-NVFP4 benchmark on RTX 5060 Ti 16GB

Using 2 GPUs on vLLM 0.27.1 (tensor parallelism = 2)

sudo docker run -t -d --name vllm --restart unless-stopped --runtime nvidia --gpus '"device=1,2"' -v ~/.cache/huggingface:/root/.cache/huggingface --env "HF_TOKEN=XXXXXXXXXXXXXX" -e HOST='0.0.0.0' -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --trust-remote-code --enable-auto-tool-choice --tensor-parallel-size 2 --tool-call-parser qwen3_xml --reasoning-parser qwen3 --enable-prefix-caching --max-model-len 262144 --max_num_seqs 128 --gpu-memory-utilization 0.96


llama-benchy --base-url http://localhost:8000/v1 --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                              |   test |              t/s |      peak t/s |     ttfr (ms) |   est_ppt (ms) |   e2e_ttft (ms) |
|:-----------------------------------|-------:|-----------------:|--------------:|--------------:|---------------:|----------------:|
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 | pp2048 | 6670.65 ± 103.98 |               | 387.83 ± 4.72 |  307.24 ± 4.72 |   387.83 ± 4.72 |
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 |  tg512 |    145.04 ± 4.56 | 152.33 ± 0.47 |               |                |                 |


llama-benchy --base-url http://localhost:8000/v1 --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                              |        test |     t/s (total) |         t/s (req) |      peak t/s |   peak t/s (req) |        ttfr (ms) |     est_ppt (ms) |    e2e_ttft (ms) |
|:-----------------------------------|------------:|----------------:|------------------:|--------------:|-----------------:|-----------------:|-----------------:|-----------------:|
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 | pp2048 (c4) | 5668.69 ± 32.41 | 2785.43 ± 1925.05 |               |                  | 1095.21 ± 417.30 | 1015.99 ± 417.30 | 1095.21 ± 417.30 |
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 |  tg512 (c4) |   333.05 ± 5.04 |      96.41 ± 7.26 | 420.33 ± 2.87 |    105.17 ± 0.99 |                  |                  |                  |


llama-benchy --base-url http://localhost:8000/v1 --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                              |        test |     t/s (total) |         t/s (req) |      peak t/s |   peak t/s (req) |        ttfr (ms) |     est_ppt (ms) |    e2e_ttft (ms) |
|:-----------------------------------|------------:|----------------:|------------------:|--------------:|-----------------:|-----------------:|-----------------:|-----------------:|
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 | pp2048 (c8) | 5830.13 ± 29.08 | 1766.29 ± 1657.68 |               |                  | 1859.85 ± 792.08 | 1781.12 ± 792.08 | 1859.85 ± 792.08 |
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 |  tg512 (c8) |  450.09 ± 10.60 |      70.37 ± 7.33 | 658.67 ± 3.77 |     82.38 ± 0.48 |                  |                  |                  |

Using 3 GPUs on vLLM 0.27.1 (pipeline parallelism = 3)

sudo docker run -t -d --name vllm --restart unless-stopped --runtime nvidia --gpus '"device=0,1,2"' -v ~/.cache/huggingface:/root/.cache/huggingface --env "HF_TOKEN=XXXXXXXXXXXXXXXXX" -e HOST='0.0.0.0' -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --trust-remote-code --enable-auto-tool-choice --tensor-parallel-size 1 --pipeline-parallel-size 3 --tool-call-parser qwen3_xml --reasoning-parser qwen3 --enable-prefix-caching --max-model-len 262144 --gpu-memory-utilization 0.96

llama-benchy --base-url http://localhost:8000/v1 --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                              |   test |              t/s |       peak t/s |      ttfr (ms) |   est_ppt (ms) |   e2e_ttft (ms) |
|:-----------------------------------|-------:|-----------------:|---------------:|---------------:|---------------:|----------------:|
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 | pp2048 | 9425.73 ± 549.01 |                | 346.51 ± 12.38 | 218.03 ± 12.38 |  346.51 ± 12.38 |
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 |  tg512 |     97.73 ± 5.37 | 110.00 ± 11.52 |                |                |                 |


llama-benchy --base-url http://localhost:8000/v1 --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                              |        test |     t/s (total) |         t/s (req) |      peak t/s |   peak t/s (req) |       ttfr (ms) |    est_ppt (ms) |   e2e_ttft (ms) |
|:-----------------------------------|------------:|----------------:|------------------:|--------------:|-----------------:|----------------:|----------------:|----------------:|
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 | pp2048 (c4) | 6566.67 ± 36.49 | 3748.84 ± 2944.48 |               |                  | 941.75 ± 361.16 | 821.21 ± 361.16 | 941.75 ± 361.16 |
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 |  tg512 (c4) |   254.66 ± 6.30 |      71.56 ± 3.69 | 308.00 ± 0.00 |     77.17 ± 0.55 |                 |                 |                 |


llama-benchy --base-url http://localhost:8000/v1 --model ornith-ai/Ornith-1.5-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                              |        test |     t/s (total) |         t/s (req) |      peak t/s |   peak t/s (req) |        ttfr (ms) |     est_ppt (ms) |    e2e_ttft (ms) |
|:-----------------------------------|------------:|----------------:|------------------:|--------------:|-----------------:|-----------------:|-----------------:|-----------------:|
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 | pp2048 (c8) | 6732.56 ± 65.44 | 2240.45 ± 2322.53 |               |                  | 1611.31 ± 680.65 | 1491.84 ± 680.65 | 1611.31 ± 680.65 |
| ornith-ai/Ornith-1.5-35B-A3B-NVFP4 |  tg512 (c8) |   353.03 ± 9.06 |      53.08 ± 3.75 | 485.33 ± 3.77 |     60.67 ± 0.47 |                  |                  |                  |

Using 3 GPUs on Ollama 0.32.15

llama-benchy --base-url http://localhost:11434/v1 --model ornith-1.5:35b --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model          |   test |              t/s |      peak t/s |      ttfr (ms) |   est_ppt (ms) |   e2e_ttft (ms) |
|:---------------|-------:|-----------------:|--------------:|---------------:|---------------:|----------------:|
| ornith-1.5:35b | pp2048 | 2302.64 ± 212.34 |               | 920.73 ± 72.15 | 826.61 ± 72.15 |  920.73 ± 72.15 |
| ornith-1.5:35b |  tg512 |    116.40 ± 2.91 | 117.00 ± 2.83 |                |                |                 |


llama-benchy --base-url http://localhost:11434/v1 --model ornith-1.5:35b --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model          |        test |    t/s (total) |       t/s (req) |      peak t/s |   peak t/s (req) |         ttfr (ms) |      est_ppt (ms) |     e2e_ttft (ms) |
|:---------------|------------:|---------------:|----------------:|--------------:|-----------------:|------------------:|------------------:|------------------:|
| ornith-1.5:35b | pp2048 (c4) | 462.18 ± 21.53 | 730.62 ± 931.06 |               |                  | 8726.79 ± 5759.04 | 8624.00 ± 5759.04 | 8726.79 ± 5759.04 |
| ornith-1.5:35b |  tg512 (c4) |  100.62 ± 0.69 |   116.26 ± 2.88 | 119.67 ± 0.47 |    116.67 ± 2.95 |                   |                   |      


llama-benchy --base-url http://localhost:11434/v1 --model ornith-1.5:35b --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model          |        test |   t/s (total) |       t/s (req) |      peak t/s |   peak t/s (req) |           ttfr (ms) |        est_ppt (ms) |       e2e_ttft (ms) |
|:---------------|------------:|--------------:|----------------:|--------------:|-----------------:|--------------------:|--------------------:|--------------------:|
| ornith-1.5:35b | pp2048 (c8) | 396.01 ± 8.87 | 390.72 ± 724.14 |               |                  | 19186.55 ± 11898.78 | 19084.22 ± 11898.78 | 19186.55 ± 11898.78 |
| ornith-1.5:35b |  tg512 (c8) |  97.21 ± 1.60 |   114.69 ± 4.75 | 119.67 ± 0.47 |    115.12 ± 4.75 |                     |                     |                     |

Qwen3.8-27B benchmark on 3x RTX 5060 Ti 16GB

I managed to get the model running with 256k context on my 3x RTX 5060 Ti 16GB setup.  Following Parameters were used on Ollama 0.32.15:

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
Environment="OLLAMA_SCHED_SPREAD=1"
Environment="OLLAMA_CONTEXT_LENGTH=262144"

And the results using llama-benchy:

llama-benchy --base-url http://localhost:11434/v1 --model qwen3.8:27b --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model       |   test |           t/s |     peak t/s |       ttfr (ms) |    est_ppt (ms) |   e2e_ttft (ms) |
|:------------|-------:|--------------:|-------------:|----------------:|----------------:|----------------:|
| qwen3.8:27b | pp2048 | 645.39 ± 5.82 |              | 2979.82 ± 95.73 | 2738.63 ± 95.73 | 2979.82 ± 95.73 |
| qwen3.8:27b |  tg512 |  40.68 ± 1.89 | 41.33 ± 2.05 |                 |                 |                 |


llama-benchy --base-url http://localhost:11434/v1 --model qwen3.8:27b --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model       |        test |   t/s (total) |       t/s (req) |     peak t/s |   peak t/s (req) |           ttfr (ms) |        est_ppt (ms) |       e2e_ttft (ms) |
|:------------|------------:|--------------:|----------------:|-------------:|-----------------:|--------------------:|--------------------:|--------------------:|
| qwen3.8:27b | pp2048 (c4) | 139.64 ± 2.14 | 209.82 ± 259.08 |              |                  | 28291.80 ± 18803.48 | 28062.33 ± 18803.48 | 28291.80 ± 18803.48 |
| qwen3.8:27b |  tg512 (c4) |  32.37 ± 1.13 |    38.26 ± 3.17 | 42.67 ± 2.62 |     38.75 ± 3.11 |                     |                     |                     |


llama-benchy --base-url http://localhost:11434/v1 --model qwen3.8:27b --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model       |        test |   t/s (total) |       t/s (req) |     peak t/s |   peak t/s (req) |           ttfr (ms) |        est_ppt (ms) |
|:------------|------------:|--------------:|----------------:|-------------:|-----------------:|--------------------:|--------------------:|-------
| qwen3.8:27b | pp2048 (c8) | 125.29 ± 3.90 | 118.09 ± 211.66 |              |                  | 61020.08 ± 38195.67 | 60794.71 ± 38195.67 | 61020.
| qwen3.8:27b |  tg512 (c8) |  31.23 ± 1.19 |    37.53 ± 2.75 | 41.33 ± 1.25 |     37.96 ± 2.70 |                     |                     |

I also managed to get the NVFP4 model running vLLM 0.28.0 using 2x GPUs, albeit with much smaller context length:

sudo docker run -t -d --name vllm --restart unless-stopped --runtime nvidia --gpus '"device=1,2"' -v ~/.cache/huggingface:/root/.cache/huggingface --env "HF_TOKEN=XXXXXXXXXXXX" -e HOST='0.0.0.0' -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model unsloth/Qwen3.8-27B-NVFP4 --tensor-parallel-size 2 --gpu-memory-utilization 0.96 --max-model-len 64K --max_num_seqs 8

and the benchmark:

llama-benchy --base-url http://localhost:8000/v1 --model unsloth/Qwen3.8-27B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                     |   test |             t/s |     peak t/s |       ttfr (ms) |    est_ppt (ms) |   e2e_ttft (ms) |
|:--------------------------|-------:|----------------:|-------------:|----------------:|----------------:|----------------:|
| unsloth/Qwen3.8-27B-NVFP4 | pp2048 | 2004.90 ± 21.32 |              | 1208.68 ± 11.06 | 1022.28 ± 11.06 | 1208.68 ± 11.06 |
| unsloth/Qwen3.8-27B-NVFP4 |  tg512 |    33.79 ± 0.00 | 35.00 ± 0.00 |                 |                 |                 |

Unable to boot from USB thumb drive on Supermicro motherboard (X10DRC-T4+)

There's a weird bug in Supermicro motherboards, where you cannot boot from (a known good bootable) USB thumb drive.  To get around this, make sure that the Video EPROM setting is set to Legacy (not UEFI)

Local LLM on three nVidia RTX 5060 Ti 16GB using nvidia/Qwen3.6-35B-A3B-NVFP4

Finally managed to get the nvidia/Qwen3.6-35B-A3B-NVFP4 to run across three RTX 5060 Ti 16GB with 256k context using vLLM's pipeline parallelism:

sudo docker run -d --name vllm --restart unless-stopped --runtime nvidia --gpus '"device=0,1,2"' -v ~/.cache/huggingface:/root/.cache/huggingface --env "HF_TOKEN=xxxxxxxx" -e HOST='0.0.0.0' -p 8000:8000 --ipc=host vllm/vllm-openai:nightly --model nvidia/Qwen3.6-35B-A3B-NVFP4 --trust-remote-code --enable-auto-tool-choice --tensor-parallel-size 1 --pipeline-parallel-size 3 --tool-call-parser qwen3_xml --reasoning-parser qwen3 --mm-encoder-tp-mode data --enable-prefix-caching --gpu-memory-utilization 0.96

utilization from nvtop:

 Device 0 [NVIDIA GeForce RTX 5060 Ti] PCIe GEN 3@ 8x RX: 59.55 MiB/s TX: 25.50 MiB/s
 GPU 2797MHz MEM 13801MH TEMP  53°C  FAN  32%   POW  52 / 180 W
 GPU[||||||||||||||||||||||||||||||||100%] MEM[||||||||||||||||||||14.023Gi/15.929Gi]

 Device 1 [NVIDIA GeForce RTX 5060 Ti] PCIe GEN 3@ 8x RX: 1.785 GiB/s TX: 320.9 MiB/s
 GPU 2790MHz MEM 13801MH TEMP  54°C  FAN  35%   POW  56 / 180 W
 GPU[||||||||||||||||||||||||||||||||100%] MEM[||||||||||||||||||||14.005Gi/15.929Gi]

 Device 2 [NVIDIA GeForce RTX 5060 Ti] PCIe GEN 3@ 8x RX: 2.148 GiB/s TX: 338.1 MiB/s
 GPU 2790MHz MEM 13801MH TEMP  54°C  FAN  32%   POW  60 / 180 W
 GPU[||||||||||||||||||||||||||       71%] MEM[||||||||||||||||||||14.539Gi/15.929Gi]
   ┌──────────────────────────────────────────┐   ┌──────────────────────────────────────────┐   ┌──────────────────────────────────────────┐
100│GPU0 %──────┐ ┌─────┐ ┌───────────────────│100│GPU1 %──────┐ ┌─────┐ ┌───────────────────│100│GPU2 %                                    │
   │GPU0 mem%───┼─┼─────┴─┴───────────────────│   │GPU1 mem%───┼─┼─────┼─┼───────────────────│   │GPU2 mem%───────────┬─┬───────────────────│
 75│            │ │                           │ 75│            │ │     └─┘                   │ 75│  ┌─────────┐ ┌─┐   │ └───────┐   ┌─┐     │
   │            └─┘                           │   │            └─┘                           │   │──┘         │ │ └───┘         │   │ │ ┌───│
   │                                          │   │                                          │   │            └─┘               └───┘ └─┘   │
 50│                                          │ 50│                                          │ 50│                                          │
   │                                          │   │                                          │   │                                          │
 25│                                          │ 25│                                          │ 25│                                          │
   │                                          │   │                                          │   │                                          │
  0│                                          │  0│                                          │  0│                                          │
   └21s──────15s────────10s───────5s────────0s┘   └21s──────15s────────10s───────5s────────0s┘   └21s──────15s────────10s───────5s────────0s┘
    PID USER DEV     TYPE  GPU        GPU MEM    CPU  HOST MEM Command
 933979 root   2  Compute  71%  14456MiB  89%    82%   5565MiB VLLM::Worker_PP2
 933977 root   0  Compute  99%  13928MiB  85%    72%   5925MiB VLLM::Worker_PP0
 933978 root   1  Compute  99%  13910MiB  85%    75%   5455MiB VLLM::Worker_PP1

and the benchmark result:

llama-benchy --base-url http://localhost:8000/v1 --model nvidia/Qwen3.6-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 1 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                        |   test |               t/s |      peak t/s |      ttfr (ms) |   est_ppt (ms) |   e2e_ttft (ms) |
|:-----------------------------|-------:|------------------:|--------------:|---------------:|---------------:|----------------:|
| nvidia/Qwen3.6-35B-A3B-NVFP4 | pp2048 | 7621.64 ± 1816.91 |               | 409.04 ± 57.99 | 282.69 ± 57.99 |  409.04 ± 57.99 |
| nvidia/Qwen3.6-35B-A3B-NVFP4 |  tg512 |     115.19 ± 8.86 | 118.33 ± 8.01 |                |                |                 |


llama-benchy --base-url http://localhost:8000/v1 --model nvidia/Qwen3.6-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 4 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                        |        test |      t/s (total) |         t/s (req) |      peak t/s |   peak t/s (req) |       ttfr (ms) |    est_ppt (ms) |   e2e_ttft (ms) |
|:-----------------------------|------------:|-----------------:|------------------:|--------------:|-----------------:|----------------:|----------------:|----------------:|
| nvidia/Qwen3.6-35B-A3B-NVFP4 | pp2048 (c4) | 5814.28 ± 168.19 | 3834.94 ± 2748.07 |               |                  | 910.26 ± 387.12 | 785.86 ± 387.12 | 910.26 ± 387.12 |
| nvidia/Qwen3.6-35B-A3B-NVFP4 |  tg512 (c4) |    257.64 ± 1.27 |      69.65 ± 3.35 | 306.67 ± 1.89 |     76.67 ± 0.47 |                 |                 |                 |


llama-benchy --base-url http://localhost:8000/v1 --model nvidia/Qwen3.6-35B-A3B-NVFP4 --depth 0 --pp 2048 --tg 512 --concurrency 8 --runs 3 --latency-mode generation --no-cache --save-total-throughput-timeseries

| model                        |        test |     t/s (total) |         t/s (req) |      peak t/s |   peak t/s (req) |        ttfr (ms) |     est_ppt (ms) |    e2e_ttft (ms) |
|:-----------------------------|------------:|----------------:|------------------:|--------------:|-----------------:|-----------------:|-----------------:|-----------------:|
| nvidia/Qwen3.6-35B-A3B-NVFP4 | pp2048 (c8) | 6026.16 ± 38.26 | 2415.53 ± 2407.68 |               |                  | 1573.88 ± 755.91 | 1446.76 ± 755.91 | 1573.88 ± 755.91 |
| nvidia/Qwen3.6-35B-A3B-NVFP4 |  tg512 (c8) |   364.16 ± 1.92 |      51.75 ± 3.59 | 491.33 ± 2.49 |     61.42 ± 0.49 |                  |                  |                  |

To disable Windows 11 RDP client from displaying warnings every time you're using a known-good RDP file, run this as an Administrator:

reg add "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services\Client" /v RedirectionWarningDialogVersion /t REG_DWORD /d 1 /f