Latest Posts ⚡

Qwen3.8-Flash-Next: What a Single DGX Spark or RTX PRO 6000 Can Do

Generative AI|

Qwen3.8-Flash-Next is a 176B-parameter Mixture-of-Experts model with a 262K-token context window. It scores 39.8 on the Artificial Analysis Intelligence Index and 53.6 on the Agentic Index.* Its NVIDIA NVFP4 checkpoint is 132.7 GB. A DGX Spark has 128 GB of memory, and an RTX PRO 6000 has 96 GB, so on paper the model fits on neither.

In our measurements it runs on both: at 28.5 tok/s on one DGX Spark and 155.8 tok/s on one RTX PRO 6000. This post explains how, and what each device delivers.

The trick: a table the model looks up, not computes with

To generate each token, a language model runs through all its layers and multiplies by its weight matrices. Those weights, and the KV cache that remembers the conversation, are read at every step. They must sit in fast GPU memory, or everything slows down.

Qwen3.8-Flash-Next also carries a different kind of component: a 51B-parameter lookup table of learned word combinations, known as an Engram-style table. The model does not compute with this table; it looks things up in it. For each token, it reads only the handful of rows that match the last few words: about 2.5 KB out of a 51 GB table. DeepSeek, which introduced the idea, calls this conditional memory: a row is read only when the input calls for it.

Think of it as a large dictionary on a shelf. You do not need to carry the whole book around; you open only the page you need.

So the table does not need to be in GPU memory. It can sit in a slower, larger place while the GPU keeps the part of the model it computes with:

  • DGX Spark: CPU and GPU share the same 128 GB, so the table goes to the local NVMe drive.
  • RTX PRO 6000: the table goes to the system RAM of the workstation.

The table’s values are unchanged, so the model’s answers are the same.

On a single DGX Spark

Speed per request and time to first token at each concurrency:

  • 1 simultaneous request: 28.5 tok/s, time to first token 302 ms
  • 2 simultaneous requests: 23.6 tok/s, time to first token 393 ms
  • 4 simultaneous requests: 17.1 tok/s, time to first token 566 ms
  • 8 simultaneous requests: 11.8 tok/s, time to first token 763 ms

A desktop device runs a model of this class at a comfortable reading speed. Up to two simultaneous requests stay above 20 tok/s, and the first token arrives in under a second at every level tested. Based on this speed and the KV cache memory left for users’ conversations, our LLM Inference Benchmark Explorer estimates this is enough for 8 chat users or 3 agentic users: a good fit for a personal assistant, a small team or development work.

On a single RTX PRO 6000

Speed per request and time to first token at each concurrency:

  • 1 simultaneous request: 155.8 tok/s, time to first token 139 ms
  • 4 simultaneous requests: 90.8 tok/s, time to first token 221 ms
  • 8 simultaneous requests: 63.7 tok/s, time to first token 252 ms
  • 16 simultaneous requests: 43.2 tok/s, time to first token 257 ms

The workstation card runs the same model more than five times faster. Even with 16 simultaneous requests, each one gets 43.2 tok/s and its first token in about a quarter of a second. We estimate this is enough for about 20 chat users or 8 agentic users (run details).

Good to know

  • The user counts are estimates based on measurements with short prompts of about 128 tokens.
  • The RTX PRO 6000 workstation needs at least 64 GB of free system RAM for the table.

Read more

  • The white paper: Engram Offloading in LLM Inference explains the mechanism, the memory differences between DGX Spark, RTX PRO 6000 and DGX B300, and all measured results.
  • The measurements: every run, with its full concurrency sweep, is in the LLM Inference Benchmark Explorer. Adjust the targets and see how the capacity estimates change.

Want to know which models your hardware can run, or which hardware your models need? Contact the OpenZeka team; we will size it with you.

*Intelligence Index v4.3 and Agentic Index, published by Artificial Analysis, retrieved 28 September 2026 and reproduced with attribution. Speeds are OpenZeka measurements made with the CordatusAI LLM Benchmark Tool (about 128 input and 128 output tokens, mean values).

Latest Posts ⚡