{"id":1739,"date":"2026-08-19T13:06:31","date_gmt":"2026-08-19T13:06:31","guid":{"rendered":"https:\/\/blog-en.openzeka.com\/?p=1739"},"modified":"2026-08-19T13:06:31","modified_gmt":"2026-08-19T13:06:31","slug":"running-glm-5-2-on-four-dgx-sparks","status":"publish","type":"post","link":"https:\/\/blog.openzeka.com\/en\/running-glm-5-2-on-four-dgx-sparks\/","title":{"rendered":"Running GLM-5.2 on Four DGX Sparks"},"content":{"rendered":"

In this tutorial, we will run the GLM-5.2<\/strong> large language model distributed across 4 DGX Spark<\/strong> units. We will use sparkrun<\/strong> for this purpose.<\/p>\n

A Spark can be used directly as a computer by connecting a monitor and keyboard, or it can serve as a remote server accessed from another machine. In this tutorial, we will connect to the Spark remotely, install the necessary software, and run the model.<\/p>\n

We will use GLM-5.2<\/strong> as the language model (a Mixture-of-Experts model with 744 billion parameters, ~40 billion active parameters, int4\/int8 quantization), vLLM<\/strong> as the inference engine, and sparkrun<\/strong> as the management tool. vLLM will load the model\u2019s trained weights into GPU memory and serve an API that accepts requests from outside. sparkrun will manage Docker containers and model distribution across the Sparks via the command line. This automates image synchronization, model transfer, and cluster configuration. Both will run on the Main Spark<\/strong> inside Docker containers.<\/p>\n

This tutorial consists of three parts:<\/strong><\/p>\n

    \n
  • Setup:<\/strong> Downloading the Docker image and preparing the recipe<\/li>\n
  • Running:<\/strong> Starting, monitoring, testing, and shutting down the model on 4 Sparks<\/li>\n
  • Benchmark:<\/strong> Performance measurement at different concurrency levels<\/li>\n<\/ul>\n

    Throughout this tutorial, the main device is called the Main Spark<\/strong>, and the other devices are called Worker Sparks<\/strong>.<\/p>\n<\/div>

    Setup<\/b><\/h2>\n

    Prerequisites<\/b><\/h3>\n

    Previous tutorials covered connecting to the Spark<\/b>, installing sparkrun<\/b>, and configuring a multi-Spark cluster<\/b> step by step. In this tutorial, we assume all these steps have been completed<\/b> and your setup is ready<\/b>.<\/p>\n

    This tutorial requires 4 DGX Sparks<\/b>. We will refer to the main device as <main-spark-ip><\/b>, and the other three devices as <worker-spark-ip-1><\/b>, <worker-spark-ip-2><\/b>, and <worker-spark-ip-3><\/b>. Note each device\u2019s IP address<\/i> beforehand.<\/p>\n

    1. Downloading the Docker Image<\/b><\/h3>\n

    GLM-5.2\u2019s DeepSeek Sparse Attention (DSA)<\/b> architecture requires kernels compiled for GB10 (SM 12.1)<\/b>. Standard vLLM images<\/i> do not include these kernels. Therefore, we will use a custom Docker image<\/b> that contains the necessary patches.<\/p>\n

    Run the following command to pull the Docker image prepared by OpenZeka<\/b>:<\/p>\n<\/div>