{"id":1885,"date":"2026-08-20T08:05:32","date_gmt":"2026-08-20T08:05:32","guid":{"rendered":"https:\/\/blog.openzeka.com\/en\/?p=1885"},"modified":"2026-08-20T11:36:39","modified_gmt":"2026-08-20T11:36:39","slug":"serving-local-llm-on-dgx-spark","status":"publish","type":"post","link":"https:\/\/blog.openzeka.com\/en\/serving-local-llm-on-dgx-spark\/","title":{"rendered":"Serving a Local LLM with vLLM on DGX Spark"},"content":{"rendered":"

In this tutorial, you will run a large language model locally on a DGX Spark and chat with it from another computer’s browser.<\/p>\n

The Spark can be used as a standalone computer by connecting a monitor and keyboard, or as a remote server accessed from another computer. In this tutorial, we will connect to the Spark remotely and install the necessary software to run the model.<\/p>\n

We will use Qwen3.6-35B-A3B-FP8 (a 35-billion parameter Mixture-of-Experts model, 3 billion parameters active, FP8 quantization) as the language model, vLLM as the inference engine, and Open WebUI as the web interface. vLLM will load the model’s trained weights into GPU memory and expose an API that accepts external requests. Open WebUI will connect to this API and display a chat interface in the browser. Both will run on the Spark, inside Docker containers.<\/p>\n

\n

Prerequisites: Finding the Spark’s IP Address<\/h2>\n

If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command in the terminal:<\/p>\n<\/div>