Resources, Tools, and Community

NVIDIA DGX Spark™ is a compact desktop AI system designed for working with artificial intelligence models. It brings development, testing, inference, fine-tuning, and prototyping workflows together on a single platform.

The software and knowledge ecosystem surrounding the device is extensive. Models, Docker images, NVIDIA playbooks, and forums are only a few of its components. DGX Spark has also developed a broad developer community. Community-built images, multi-device recipes, tools, software improvements, and performance rankings have expanded the ecosystem well beyond NVIDIA’s official resources.

Models

Models intended to run on DGX Spark are hosted on Hugging Face, and many are available in multiple quantization formats. A single model may have FP8, NVFP4, MXFP4, or INT4 variants on Hugging Face. DGX Spark provides hardware support for all of these formats, allowing compatible models to be run after download without requiring an additional conversion step.

Among these quantization formats, NVFP4 is an FP4 format specifically designed for the Blackwell architecture used by DGX Spark. By representing model weights at a lower bit width, it substantially reduces memory requirements. For example, a 200-billion-parameter model that would require approximately 400 GB of memory in a standard 16-bit format can be reduced to roughly one-quarter of that size with NVFP4. This makes it possible to run significantly larger models within the system’s 128 GB of unified memory.

Hugging Face hosts a collection of nearly 100 NVFP4 models converted by NVIDIA. The community has also published a wide range of models specifically prepared for Spark.

Inference Engines

Inference engines are software systems that load model weights into GPU memory and expose APIs for accepting external requests. These APIs can then be used by Open WebUI, Odysseus, or any other compatible client. Common inference engines include vLLM, SGLang, TensorRT-LLM, and llama.cpp.

The most widely used inference engine on DGX Spark is vLLM. One of the main reasons is its strong support for distributed execution. In particular, its robust multi-node tensor parallelism capabilities make it well suited to multi-Spark deployments. Other important features include PagedAttention for efficient GPU memory utilization, prefix caching for reusing computations across repeated prompts, and support for lower-precision KV cache formats such as FP8 and NVFP4.

vLLM is commonly run inside Docker containers. Docker images are used to package the runtime together with its GPU dependencies. Official vLLM images are published on Docker Hub under vllm/vllm-openai.

Community

The GB10 chip used in DGX Spark has an SM 12.1 compute capability, which differs from that of conventional NVIDIA GPUs. Official Docker images for inference engines such as vLLM may not include native kernels for this architecture. In other words, models may still run with official vLLM images, but some workloads may not fully utilize the performance capabilities of the Spark GPU. For this reason, specialized images and deployment recipes are sometimes required.

Developers

Community developers build and publish Docker images containing native SM 12.1a kernels for GB10. As a result, community-maintained images are frequently used alongside official vLLM images when working with DGX Spark.

Some of the most prominent developers and their contributions to the Spark ecosystem are summarized below. New contributors continue to appear regularly:

• eugr: spark-vllm-docker — General-purpose inference images, MXFP4 builds, and the B12X kernel family
• christopherowen: vllm (fork) — Source code for CUTLASS and FlashInfer kernels
• local-inference-lab: vllm (fork) — Source of the B12X kernel family and experimental high-performance kernels for the sm12x architecture
• tonyd2wild: tonyd2wild — Model-specific images, multi-Spark deployment recipes for DeepSeek V4 Flash, GLM-5.2, MiMo V2.5, and MiniMax-M3, as well as DSpark speculative decoding patches
• mpfaffenberger: mpfaffenberger — Runtime modifications addressing model compatibility issues
• MiaAI-Lab: MiaAI-Lab — Community deployment recipes for DeepSeek V4 Flash, GLM-5.2, and Laguna S, as well as the sparkDash monitoring tool

In addition, Spark Arena maintains an LLM performance leaderboard ranking the performance of different models and inference engines on DGX Spark.

Tools

Docker alone is sufficient for running models on a single DGX Spark. With multiple Sparks, however, manually managing Docker containers, creating the cluster, and configuring the CX-7 network, SSH mesh, and NCCL communication can become considerably more complex. The community has developed several tools to simplify this process.

eugr/spark-vllm-docker, developed by eugr, is one of the most comprehensive toolsets for building Docker images with native SM 12.1a kernels for GB10. It includes build scripts, cluster initialization, recipe deployment, model downloading, node discovery, runtime modifications, and ready-to-use recipes. These tools are widely used by the community.

sparkrun, developed by Spark Arena, automates cluster management. Its setup wizard configures the CX-7 network, SSH mesh, and NCCL environment with a single command. A YAML recipe file can then be used to define the model, Docker image, inference parameters, and memory configuration. sparkrun also provides comprehensive real-time cluster monitoring and has become a widely used cluster-management tool within the DGX Spark ecosystem.

OpenZeka Resources

At OpenZeka, we publish Turkish and English tutorials covering our work with DGX Spark. English-language articles are available at blog.openzeka.com/en/. The blog primarily focuses on user education, including tutorials, step-by-step model and tool installation instructions, and practical usage guides. Some of our DGX Spark articles are listed below:

• Serving a Local LLM with vLLM on DGX Spark: Single Spark, vLLM, Open WebUI
• Local GPT-OSS 120B Serving on DGX Spark with sparkrun: Single/dual Spark, GPT-OSS 120B, vLLM, sparkrun
• DeepSeek-V4-Flash-0731 on 2x DGX Spark: 2× Spark, DeepSeek-V4-Flash-0731, vLLM, sparkrun
• Local AI Workspace with Odysseus on DGX Spark: Local AI workspace, agents, RAG, tool use
• Local Model Fine-Tuning with Unsloth on DGX Spark: Fine-tuning, Unsloth, QLoRA
• GPU-Accelerated Data Science and Machine Learning on DGX Spark: CUDA-X, RAPIDS, cuDF, cuML
• Building a High-Speed AI Cluster with 4 DGX Spark Systems: 4× Spark cluster, 200GbE, RDMA, RoCEv2

Our white papers are published at whitepapers.openzeka.com. Unlike the blog’s user-education-oriented content, the white papers target a more technical and professional audience. They provide in-depth analyses intended to support decision-making in areas such as hardware selection, benchmark interpretation, cluster architecture comparison, and scaling strategies. Some of our white papers are listed below:

• Local LLM Usage Guide: Hardware, model, and software selection guide
• Qwen3.6-27B DGX Spark Benchmark: FP8, AWQ, and NVFP4 performance analysis
• Qwen3.6-27B DGX Spark Cluster Scaling: 1×/2×/4× Spark scaling analysis
• DGX Spark 2-Node Cluster Setup Guide: Two-node cluster setup
• DGX Spark 3-Node Cluster Setup Guide: Three-node ring topology
• DGX Spark 4-Node Cluster Setup Guide: Four-node switch topology

We also publish a variety of tutorials, tests, and product-analysis videos on the OpenZeka and CordatusAI YouTube channels.

Forums and Documentation

The DGX Spark / GB10 category of the NVIDIA Developer Forums is a community discussion space in which NVIDIA engineers also actively participate. It contains hundreds of discussions covering model deployment recipes, performance optimization, newly released models, and many other topics. It is one of the first resources worth consulting when encountering a DGX Spark-related issue. The NVIDIA Developer page brings together links to this forum, official documentation, and playbooks in one place.

Official DGX Spark documentation is available at docs.nvidia.com/dgx/dgx-spark. The User Guide covers hardware specifications, DGX OS, ConnectX-7 clustering, system recovery, and known issues, with dedicated pages for hardware specifications, OS and component updates, and release notes. For multi-Spark deployments, NVIDIA provides documentation for the NVIDIA Sync Cluster Assistant and cluster network inspection. For performance analysis, documentation is available for both Nsight Systems and Nsight Compute.

Spark Playbook Catalog

NVIDIA provides a collection of Spark Playbooks for DGX Spark. Each playbook contains step-by-step installation and execution instructions for a particular tool or workflow. Some of these tools directly run models, while others provide the surrounding infrastructure required to make those models usable in practical applications.

Inference Engines

Models are typically developed using machine learning frameworks such as PyTorch or JAX. Inference engines load or convert the resulting model weights and optimize GPU kernels, KV cache management, batching, and memory utilization to execute the model efficiently. Some also provide an OpenAI-compatible API through either a built-in or separate model-server component.

• TensorRT-LLM: trt-llm — TensorRT-LLM is an open-source inference library optimized for LLM execution on NVIDIA GPUs. It uses optimized kernels and supports tensor, pipeline, and sequence parallelism. It integrates with Hugging Face and PyTorch.
• vLLM: vllm — vLLM is a high-throughput inference engine that uses PagedAttention and continuous batching to improve memory efficiency. It also provides an OpenAI-compatible model server.
• llama.cpp: llama-cpp — llama.cpp is a lightweight C/C++ inference engine for running GGUF models across CPUs and multiple GPU backends. Its llama-server component can expose an OpenAI-compatible API.
• SGLang: sglang — SGLang is a high-performance inference and model-serving framework for LLMs and VLMs. It jointly optimizes the backend runtime and model interaction layer.
• NVIDIA NIM: nim-llm — NIM is NVIDIA’s solution for packaging inference engines such as those listed above into containerized HTTP microservices. Rather than being an inference engine itself, it acts as a packaging and deployment layer that prepares these engines for production and enterprise use.

Applications and Tools

These tools allow users and developers to interact directly with models. Live VLM WebUI and Continue connect to an existing backend service, whereas Ollama and LM Studio also include their own inference runtimes.

• Ollama + Open WebUI: open-webui — Ollama is a local model-serving layer built on llama.cpp that provides a CLI, model library, and OpenAI-compatible API. Open WebUI is a self-hosted web interface that connects to this service.
• LM Studio: lm-studio — LM Studio is a local AI application that combines model discovery, execution, and API-based serving.
• Live VLM WebUI: live-vlm-webui — Live VLM WebUI is a web interface that sends webcam streams to Ollama, vLLM, SGLang, or cloud-hosted VLM services. It allows the outputs and performance of different backend models to be compared.
• Continue + VS Code: vibe-coding — Continue is a coding client that runs inside VS Code. In the playbook, Ollama serves the model while Continue connects to that service for chat and code generation.

Models and Inference Optimizations

Nemotron is NVIDIA’s model family. Speculative decoding and NVFP4, meanwhile, are techniques intended to improve the performance or memory efficiency of compatible inference engines.

• NVIDIA Nemotron: nemotron — Nemotron is NVIDIA’s open model family designed for reasoning, tool use, and long-context tasks. It can be served with llama.cpp, vLLM, SGLang, or TensorRT-LLM.
• Speculative decoding: speculative-decoding — Speculative decoding accelerates the decoding process by having a smaller, faster draft model propose tokens that are then verified in batches by the target model. The playbook demonstrates this technique with EAGLE-3 and Draft–Target methods in TensorRT-LLM.
• NVFP4 quantization: nvfp4-quantization — NVFP4 is a Blackwell-compatible 4-bit quantization format for model weights and activations. The playbook converts the model with TensorRT Model Optimizer and then runs it in a TensorRT-LLM environment.

Fine-Tuning Frameworks and Tools

Fine-tuning tools use existing model weights to produce updated model weights. The resulting model can subsequently be run using an inference engine.

PyTorch is the underlying machine learning framework. NeMo, LLaMA-Factory, and Unsloth provide more streamlined and optimized training workflows built on top of PyTorch.

• NVIDIA NeMo: nemo-fine-tune — NeMo AutoModel is a framework for training Hugging Face models with native PyTorch support and scaling them to multi-node systems. It supports SFT, PEFT, and distributed training workflows.
• LLaMA-Factory: llama-factory — LLaMA-Factory is a fine-tuning framework that provides a common interface for optimization methods such as SFT, LoRA, and QLoRA across different LLMs and VLMs.
• Unsloth: unsloth — Unsloth is a fine-tuning toolkit that uses custom GPU kernels to accelerate LoRA and QLoRA training while reducing memory consumption.
• PyTorch: pytorch-fine-tune — PyTorch is a foundational machine learning framework that provides tensor operations, automatic differentiation, and GPU acceleration. This playbook implements SFT, LoRA, and QLoRA workflows directly in PyTorch at a lower level of abstraction.
• FLUX.1 DreamBooth LoRA: flux-finetuning — This playbook uses DreamBooth LoRA to teach new concepts and styles to the FLUX.1-dev diffusion image-generation model.

Agentic AI Systems

AI agents connect to an LLM service for decision-making while interacting with files, terminal commands, APIs, and other tools. The model itself is run by a service such as vLLM or Ollama, while the agent application manages task planning, memory, and tool use.

OpenClaw and Hermes are agent applications. OpenShell is a sandbox runtime that surrounds the agent process with filesystem, network, permission, and security policies. NemoClaw combines these layers into an integrated deployment.

• OpenClaw: openclaw — OpenClaw is a persistent local agent application combining memory, filesystem access, tool use, and skills. In the playbook, its decision-making model is served through vLLM.
• Hermes Agent: hermes-agent — Hermes is an autonomous agent capable of generating reusable skills from its experiences, retaining memory across sessions, and running scheduled tasks. In the playbook, vLLM is used as the local model service.
• OpenShell: openshell — OpenShell is a sandbox runtime that constrains agents at the kernel level. The playbook runs OpenClaw inside the sandbox while the model service is accessed through an endpoint outside the sandbox.
• NemoClaw: nemoclaw — NemoClaw combines the OpenClaw agent, OpenShell sandbox, and local vLLM inference. Filesystem, network, process, and inference access are configured within a unified deployment.
• NemoClaw example agents: nemoclaw-applications — This playbook deploys news summarization, software development, document review, and calendar negotiation agents on top of an existing NemoClaw sandbox, demonstrating how the same infrastructure can be adapted to different applications.
• CLI Coding Agents: cli-coding-agent — Claude Code, OpenCode, and Codex CLI are terminal-based agents that manage file editing, command execution, and coding tasks. In the playbook, model inference is provided by Ollama.

Complete AI Applications and Solutions

These applications combine models, inference engines, data processing, user interfaces, and agent components to solve specific problems. Instead of interacting with individual infrastructure layers, users interact with a complete end-to-end system.

• RAG Application in AI Workbench: rag-ai-workbench — An agentic RAG application combining query routing, information retrieval, answer generation, and hallucination evaluation. It can use either NVIDIA-hosted APIs or self-hosted model services.
• Video Search and Summarization: vss — VSS uses VLM, LLM, and RAG components to generate video summaries, question-answering capabilities, and real-time alerts.
• Multi-Agent Chatbot: multi-agent-chatbot — A supervisor agent coordinates coding, RAG, and visual-understanding agents. llama.cpp and TensorRT-LLM servers are used for model serving, while MCP is used for tool connectivity.
• Image Generation with TensorRT: multi-modal-inference — This playbook optimizes the FLUX.1 and SDXL diffusion models with TensorRT for text-to-image generation.
• Text to Knowledge Graph: txt2kg — Extracts subject–predicate–object relationships from text using Ollama, stores the resulting data in ArangoDB, and visualizes it with Three.js WebGPU. Ollama performs inference while ArangoDB handles relationship queries.
• Spark & Reachy Photo Booth: spark-reachy-photo-booth — A multimodal application combining NeMo Agent Toolkit, TensorRT-LLM, speech recognition, speech synthesis, FLUX image generation, and object tracking with the Reachy Mini robot. Services communicate through a message bus.

Other GPU-Accelerated Workloads

These playbooks are not part of AI inference or model-serving workflows. Instead, they demonstrate how DGX Spark’s computational acceleration capabilities can be applied to areas such as data science, optimization, bioinformatics, and robotics.

• CUDA-X Data Science: cuda-x-data-science — CUDA-X Data Science uses RAPIDS libraries such as cuDF and cuML to accelerate pandas and scikit-learn workflows on the GPU.
• Single-cell RNA Sequencing: single-cell — RAPIDS-singlecell provides a Scanpy-like API for running scRNA-seq preprocessing, quality control, clustering, and visualization on the GPU. It uses RAPIDS components such as cuML and cuGraph.
• Portfolio Optimization: portfolio-optimization — cuOpt solves portfolio constraints as LP/MILP optimization problems, while cuML accelerates the generation of risk scenarios. The playbook combines these tools in an end-to-end Mean-CVaR-based financial workflow.
• Isaac Sim and Isaac Lab: isaac — Isaac Sim is a GPU-accelerated robotics simulation platform. Isaac Lab is a framework built on top of this environment for developing reinforcement-learning policies.

Multi-DGX Spark Deployments

These playbooks prepare multi-device infrastructure independently of the model or application layer. First, the QSFP network and inter-node SSH access are configured. NCCL then provides high-performance collective GPU communication over this connection for distributed training and inference.

Higher-level software such as vLLM, TensorRT-LLM, PyTorch, and NeMo can use this infrastructure when operating in distributed configurations.

• 2× DGX Spark direct connection: connect-two-sparks — Connects two DGX Spark systems directly through a 200 GbE QSFP link and passwordless SSH, creating a two-node distributed system.
• 3× DGX Spark ring connection: connect-three-sparks — Connects three DGX Spark systems in a ring topology using three QSFP cables.
• 4+ DGX Spark switch connection: multi-sparks-through-switch — Connects four or more DGX Spark systems through a QSFP switch to create a scalable cluster.
• NCCL: nccl — NCCL is NVIDIA’s library for high-performance collective communication between GPUs located on different nodes.

All of these playbooks are available at build.nvidia.com/spark.

Conclusion

DGX Spark has a strong and rapidly evolving ecosystem built on NVIDIA’s foundations and continuously expanded through a growing range of community projects. OpenZeka contributes to this ecosystem by making its technical knowledge more accessible to end users through Turkish- and English-language tutorials, technical analysis documents, practical guides, and YouTube videos.

In this tutorial, you will run a large language model locally on a DGX Spark and chat with it from another computer’s browser.

The Spark can be used as a standalone computer by connecting a monitor and keyboard, or as a remote server accessed from another computer. In this tutorial, we will connect to the Spark remotely and install the necessary software to run the model.

We will use Qwen3.6-35B-A3B-FP8 (a 35-billion parameter Mixture-of-Experts model, 3 billion parameters active, FP8 quantization) as the language model, vLLM as the inference engine, and Open WebUI as the web interface. vLLM will load the model’s trained weights into GPU memory and expose an API that accepts external requests. Open WebUI will connect to this API and display a chat interface in the browser. Both will run on the Spark, inside Docker containers.

Prerequisites: Finding the Spark’s IP Address

If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command in the terminal:

The command outputs the IP address of the Spark’s default network interface:

Note this address; you will use it in place of throughout this tutorial. Alternatively, you can find the IP address by checking the NVIDIA Sync application.

1. Connecting to the Spark

Make sure your computer is connected to the same network as the Spark. Then, open a terminal on your computer and connect to the Spark via SSH:

On the first connection, you will see a fingerprint warning. Type yes and press Enter. Then, when prompted for a password, enter the Spark’s password:

Once connected, the Spark will start accepting commands sent from this terminal.

2. Clearing the Cache

Before starting vLLM, flush the filesystem cache:

The main reason for doing this is the DGX Spark’s unified memory architecture: the operating system caches model files read from disk in RAM. vLLM loads the model weights from here into GPU memory. After loading is complete, the cached data remains in RAM even though it will not be used again. On systems with separate memory, this is not significant — since inference runs in GPU memory, RAM utilization does not affect performance. On the Spark, however, the CPU and GPU share the same RAM, so the cache reduces the memory available to the GPU. This command frees the cache, providing maximum memory for vLLM.

When prompted for a password, enter the Spark’s password. The command produces no output and completes silently:

3. Setting Up the vLLM Server

Pull the vLLM Docker image:

During the download, layers will be pulled sequentially and you will see progress percentages on the screen. If the image is already on disk, Docker skips the download. Once the download is complete, you will see the following output:

This image contains everything needed to run vLLM — PyTorch, CUDA kernels, Python libraries…

Now, let’s start the vLLM server:

This command starts the downloaded image as a container. If the model is not already installed on the Spark, it will be automatically downloaded from Hugging Face. As the download progresses, you will see progress bars, and once complete, the files will be placed in the ~/.cache/huggingface/hub/ directory. If the model has been previously downloaded on the Spark (if you have followed this tutorial before), vLLM skips the download step and loads the downloaded weights directly into GPU memory.

When the command finishes, you will see a container ID; this indicates that the server has started in the background and the model is now being served on port 8000:

If a container with the same name (from your previous attempts) already exists, the command will return the following error:

In this case, check the container list — vllm-qwen36-35b-fp8 should be listed:

If the status column shows “Up”, the container is already running and ready to use — you can proceed to the next step:

If the status column shows “Exited”, the container has stopped:

You can restart the stopped container and proceed to the next step:

4. Monitoring the vLLM Startup Process

After the model download is complete, it may take a few minutes for vLLM to become ready for serving. During this process, vLLM loads the model weights into GPU memory, compiles GPU kernels, and allocates memory for inference. To monitor this process:

You will see the following in the logs:

Once you see the Application startup complete. line, the model server is ready. Press Ctrl+C to stop watching the logs. The server will continue running in the background.

5. Testing the vLLM Server

First, let’s verify that the server is running:

If you don’t receive any errors, the server is healthy:

Now test the model:

The simplified response is shown below. In the content field (the model’s response), you will see the number 204. This confirms that the model performed the multiplication correctly. Additionally, the response includes a reasoning field. This field contains the model’s thought process. As you can see, the model performed the calculation using two separate methods and verified the solution before arriving at the answer:

vLLM is now running and we can ask questions to Qwen3.6 and receive responses. However, sending each question as a terminal command like the one above is not practical. We need a web interface to chat from the browser.

6. Setting Up Open WebUI

Open WebUI is an interface that connects to vLLM and can be used from your browser. You can chat with your language model and upload images to ask questions about them. Responses start appearing on the screen as they are being generated — you don’t have to wait for the entire response to finish before you start reading. For each chat, you can adjust settings such as temperature, top_p, and max_tokens. On first setup, it downloads a small embedding model, which enables you to upload documents and ask questions about them (RAG — Retrieval-Augmented Generation). Your chat history, settings, and account information are stored on the Spark’s disk; even if the device or container is shut down, your data is not lost, and you can continue from where you left off when you restart.

First, pull the Open WebUI Docker image on the Spark:

If the image is already on disk, Docker skips the download. Once the download is complete, you will see the following output:

Once the download is complete, start the container:

The command returns a container ID, indicating that the container has started:

As mentioned in the vLLM setup section, if a container with the same name (from your previous attempts) already exists, the command will return the following error:

In this case, check the container list — open-webui should be listed:

If the status column shows “Up”, the container is already running and ready to use — you can proceed to the next step:

If the status column shows “Exited”, the container has stopped:

You can restart the stopped container and proceed to the next step:

Your Open WebUI server will then be ready for use.

7. Monitoring the Open WebUI Startup Process

To watch the logs:

You will see output similar to the following in the logs:

After you see the Scheduler worker started line in the logs, wait a few seconds. Press Ctrl+C to exit log monitoring.

8. Testing Open WebUI

To verify that the server is ready and healthy, run the following command:

If you receive an HTTP 200 response, Open WebUI is ready:

9. Accessing from the Browser

On your computer’s browser, which is on the same network as the Spark, navigate to:

On first launch, a login screen will appear. Click the “Sign up” link and enter your name, email, and password. The first account created automatically becomes the administrator account. The purpose of creating an account is to store chat history and settings. This information is stored on the Spark.

The model is automatically detected by Open WebUI and listed in the model selection menu. You will see the Qwen/Qwen3.6-35B-A3B-FP8 model in the menu. Select the model. If you are running multiple models at the same time, you can instantly switch between models from here. Additionally, the chat history tab on the left allows you to create multiple chat sessions and switch between them.

Type a question in the message box and press Enter. As the model responds, a collapsible reasoning section will appear at the top, and the response will appear below.

Here are a few experiments we conducted to demonstrate what the model and web interface can do:

First, to test the model’s Turkish comprehension skills, we asked the model to list Turkey’s three largest cities and provide brief information about each.

Next, we asked the model to explain the benefits of artificial intelligence in the healthcare sector in three paragraphs.

After that, to test the model’s programming skills, we asked it to create a Python function that checks whether a text is a palindrome.

Using the file upload button in the message area, we sent the model an image of nature. The model described elements in the uploaded image such as flowers, leaves, water drops, and the background.

From the same file upload section, this time we gave the model the DGX Spark datasheet and asked it to analyze it.

Your local LLM setup on the Spark is ready! You can now chat with Qwen3.6 from all your devices on the same network.

10. Shutdown

When you are done, stop both containers:

Although the stopped containers cease running, their data and configuration are preserved on disk. To restart the containers from where they left off:

To completely remove the containers:

Even if you remove the containers, the Docker images and model files remain on disk. Therefore, you do not need to re-download to restart — simply run the docker run commands from steps 3 and 6 again. Chat history and account information are stored in the open-webui-data volume; even if the container is removed, this data is preserved on the Spark. If you want to delete this data as well, you can use the docker volume rm open-webui-data command.

In this tutorial, you will run a large language model on a single or multiple DGX Sparks using sparkrun.

Spark can be used directly as a computer by connecting a monitor and keyboard, or as a server accessed remotely from another computer. In this tutorial, we will connect to the Spark remotely, install the necessary software, and run the model.

We will use GPT-OSS 120B as the language model (a Mixture-of-Experts model with 117 billion parameters, 5.1 billion active parameters, mxfp4 quantization), vLLM as the inference engine, and sparkrun as the management tool. vLLM will load the model’s trained weights into GPU memory and run them, serving an API that accepts external requests. sparkrun will manage Docker containers and model distribution across Sparks via the command line. This automates image synchronization, model transfer, and cluster configuration. Both will run on the Main Spark, inside Docker containers.

This tutorial consists of three parts:

• Setup: sparkrun installation, Docker image build, cluster configuration, and recipe creation
• Running with a Single Spark: Starting, monitoring, and testing the model on a single Spark
• Running with Two Sparks: Running the model split across multiple Sparks (tensor parallelism) and comparing performance

If you have a single Spark, review the “Setup” and “Running with a Single Spark” sections. If you are using multiple Sparks, you can proceed directly to the “Running with Two Sparks” section after “Setup”.

If you are using multiple Sparks, note that throughout the tutorial the primary device will be referred to as the Main Spark, and the other devices as Worker Sparks.

Setup

1. Connecting to the Spark

If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command from the terminal:

The command returns the IP address of the Spark’s default network interface:

Note this address; throughout the tutorial you will use it in place of . Alternatively, you can find the IP address by checking the NVIDIA Sync application.

If you are using multiple Sparks, repeat this step for every other device and note the addresses. You will use them in place of throughout the tutorial.

If you are using multiple Sparks, ensure the same username is used on all Sparks. sparkrun requires matching usernames when setting up passwordless SSH between devices. The default DGX OS username is nvidia. Check your username on each Spark:

If the username is not nvidia, create this user on all Sparks:

These commands: create the user, add them to the sudo group, set a password, and switch to the new user. Ensure the same username exists on all Sparks.

If you are using multiple Sparks, connect the devices to each other with QSFP cables. The cable can be plugged into any of the CX-7 ports on each Spark. To verify the connection, log in to each device with a monitor and keyboard and run the following command:

Those shown as (Up) are the ports where the cable is connected. Those shown as (Down) are unused ports. If no interface shows as (Up), check the QSFP cable and restart the Sparks.

Ensure your computer is connected to the same network as the Spark. Then, open a terminal on your computer and connect to the Spark via SSH:

On the first connection you will see a fingerprint warning. Type yes and press Enter. Then, when prompted for a password, enter the Spark’s password:

Once connected, the Spark will begin accepting commands sent from this terminal. Throughout the tutorial, you will enter all commands you encounter into this terminal on your computer.

2. Installing sparkrun

First, download the uv package manager:

Add uv to your PATH:

PATH updated. The ~/.local/bin directory (where uv and sparkrun are installed) is now available and has been added to the PATH for all future terminal sessions. Now install sparkrun:

Let’s verify the installation:

sparkrun v0.2.40 is installed and accessible.

3. Building the Docker Image

To get the best performance from GPT-OSS 120B, we will build a Docker image that contains CUTLASS MoE and FlashInfer attention kernels compiled specifically for the Spark’s GPU architecture. This image also includes the --mxfp4-backend, --mxfp4-layers flags and GPT-OSS’s tiktoken encoding files.

We will build this image using the build-and-copy.sh script from the eugr community project. First, clone the eugr repository:

The Dockerfile.mxfp4 file in the repository determines the version of the source code that build-and-copy.sh will compile. Before starting the build, we must ensure this version is up to date. Therefore, let’s query the latest commit hash of the mxfp4_v2 branch in christopherowen’s community project vLLM fork:

This hash returned by the command will direct build-and-copy.sh to the latest version of the vLLM source code we want. Copy or note it. Then open the Dockerfile.mxfp4 file with nano:

Once the file is open, press Ctrl+W to search within the file, type VLLM_SHA and press Enter. The cursor will land on a line similar to:

Delete the old hash value and paste the updated hash you noted above. Then press Ctrl+O to save the file, confirm with Enter, and exit nano with Ctrl+X.

Let’s verify the change we made to the Dockerfile.mxfp4 file:

The returned hash value should be the updated hash you copied in the previous step.

Fallback: If you experience issues with the latest commit, you can use the hash 04f641e537e80a67db464c6e65b928dbfab5d647 (January 28, 2026), which we have tested and confirmed. You can also check for newer commits in the commit history .

Now build the image:

The first build takes approximately 50 minutes; subsequent builds complete in about 5 minutes thanks to caching. When the build completes, you will see the following output:

Verify the image:

Verify the image’s vLLM version:

If you are using multiple Sparks, building the image only on the Main Spark is sufficient. sparkrun will automatically synchronize the image to the Workers over the CX-7 network on first launch. No additional step is needed.

4. Setup Wizard

The sparkrun setup wizard configures all the infrastructure needed to run the Spark with a single interactive command:

Six phases run sequentially, and the phases are interactive. In the first phase, enter in the ‘Enter host IPs/hostnames’ field and press Enter. For the subsequent prompts, you can simply press Enter (confirming the default options) to continue:

If you are using multiple Sparks, enter all the Sparks’ IP addresses separated by commas in the ‘Enter host IPs/hostnames’ field .

The second phase sets up passwordless SSH. Type Y and press Enter:

If you are using multiple Sparks, the wizard connects to the Worker Sparks via SSH during this phase and asks for the password. Enter the password — this step is done once; after the wizard distributes the passwordless SSH keys, no password will be asked for in subsequent connections.

The third phase configures the CX-7 high-speed network interfaces. It is automatically skipped if a single Spark is used. Type Y and press Enter:

If you are using multiple Sparks, the wizard automatically detects CX-7 interfaces, selects non-overlapping subnets, assigns a static IP to each Spark, sets MTU 9000 (jumbo frame), and writes the /etc/netplan/40-cx7.yaml file and applies it with netplan apply. This operation requires root privileges — enter the sudo password.

After CX-7 configuration, the wizard automatically refreshes the SSH network with the new IPs. This phase requires no additional input.

The fourth phase verifies that the user is in the docker group. Type Y and press Enter:

The fifth phase installs restricted sudoers rules. The installed rules are for passwordlessly fixing HuggingFace cache ownership and passwordlessly clearing the Linux page cache. Broad sudo privileges are not granted. Type Y and press Enter:

The sixth phase installs earlyoom OOM protection. earlyoom terminates inference processes instead of locking up the system when memory limits are approached in the DGX Spark’s unified memory architecture. Type Y and press Enter:

5. Creating the Recipe

A recipe is a YAML file that defines how sparkrun runs the model. The model, Docker image, vLLM flags, and memory settings are consolidated in a single file. This recipe can be used for both a single Spark (TP=1) and multiple Sparks (TP>1) — the only difference is that the --tp value in the launch command is adjusted according to the number of Sparks.

Verify the file:

Key fields in the recipe and the reasons they were selected:

• container
  • Value: vllm-node-mxfp4
  • Why: Image built in step 3 — includes the CUTLASS/FlashInfer forks
• tensor_parallel
  • Value: 1 (default)
  • Why: Default for a single Spark; overridden with the --tp value when using multiple Sparks
• gpu_memory_utilization
  • Value: 0.7
  • Why: Safe value for DGX Spark UMA
• --distributed-executor-backend ray
  • Value: Ray
  • Why: sparkrun automatically detects Ray and sets up the Ray cluster. For TP=1, Ray runs on a single node.
• --mxfp4-backend CUTLASS
  • Value: CUTLASS
  • Why: Custom CUTLASS MXFP4 MoE GEMM kernel
• --mxfp4-layers moe,qkv,o,lm_head
  • Value: Full
  • Why: All layers are quantized to FP4
• --attention-backend FLASHINFER
  • Value: FlashInfer
  • Why: Custom FlashInfer kernel supporting GPT-OSS attention architecture
• --kv-cache-dtype fp8
  • Value: FP8
  • Why: Reduced memory usage for the KV cache
• --load-format fastsafetensors
  • Value: Fast loader
  • Why: ~41-second model loading time
• --reasoning-parser openai_gptoss
  • Value: Harmony
  • Why: Separates the reasoning process into the reasoning field
• --tool-call-parser openai
  • Value: Tool usage
  • Why: Required for --enable-auto-tool-choice

Important:
The --rootful flag is not specified in the recipe; it is passed in the launch command.
FlashInfer compiles CUTLASS attention kernels at runtime, and this compilation requires root privileges.
The launch commands are shown in the “Launch the Model” step of the relevant section.

6. Pre-Launch Checks

Verify that `sparkrun` correctly parses the recipe and that the memory budget is appropriate:

sparkrun correctly parsed the recipe, selected vllm-ray, and gave the ‘DGX Spark fit: YES’ confirmation.

Now preview the launch plan:

Dry run successful. The recipe is valid, the serve command includes the flags we added, and it is ready to launch. If the model is not installed on the Spark, sparkrun will automatically download it from Hugging Face on first launch. The GPT-OSS 120B model is approximately 183 GB; download time depends on your internet speed. The dry run does not trigger this download — downloading only happens on the actual launch.

If you are using multiple Sparks, set the --tp value according to the number of Sparks for the TP dry-run (for example, --tp 2 for two Sparks). You will see --tensor-parallel-size 2 in the serve command, and the mode will appear as cluster (2 nodes) instead of solo:

Running with a Single Spark

1. Clearing the Cache

Before starting vLLM, clear the filesystem cache:

The main reason for doing this is the DGX Spark’s unified memory architecture: The operating system caches the model files it reads from disk in RAM. vLLM loads the model weights from here into GPU memory. After loading is complete, the data remaining in the cache is not used again but is not cleaned up immediately. In systems with separate memory, this is not significant. Since inference runs in GPU memory, RAM fullness does not affect performance. On the Spark, however, since the CPU and GPU share the same RAM, the cache reduces the space available to the GPU. The command clears this cache, providing maximum memory for vLLM.

When prompted for a password, enter the Spark’s password. The command produces no output; it completes silently:

2. Launching the Model

Now let’s launch the model:

The --rootful flag runs the container with root privileges. FlashInfer compiles CUTLASS attention kernels at runtime and requires root privileges for this compilation. The --no-follow flag causes sparkrun to return to the command line after launching the containers.

sparkrun successfully completed all 6 steps. Image building was skipped, and the image we built during setup was used. Since the model files were already on disk, distribution completed quickly. All flags were correctly resolved within the serve command.

3. Monitoring the Startup Process

After the model download completes, it may take a few minutes for vLLM to become ready to serve. During this process, vLLM loads the model weights into GPU memory, compiles GPU kernels, and allocates memory for inference.

To monitor the vLLM logs:

You will see the following in the logs:

If you see the Application startup complete. line, the model server is ready. Press Ctrl+C to stop watching the logs. The server will continue running in the background.

4. Testing the Model

First, verify that the server is running:

An ‘HTTP 200’ response indicates the server is healthy. Now check the sparkrun container status:

List the registered models:

The model openai/gpt-oss-120b is registered.

Now test the model with the prompt “What is 12*17?”:

A simplified version of the response is below. In the content field (the model’s answer) you will see 204. From this, you can tell the model performed the multiplication correctly. Additionally, the response has a reasoning field. This field contains the model’s reasoning process. GPT-OSS’s Harmony format is working: the reasoning process and the final answer are presented in separate fields:

vLLM is now running and serving on the Spark’s port 8000. We can ask GPT-OSS 120B questions and get answers.

5. Shutdown

When you are done, stop the model:

This command stops the container and frees up memory. However, the container, the Docker image, and the model files remain on disk. This means you don’t need to re-download to start again. Simply re-run the sparkrun run command from step 2.

Running with Two Sparks

1. Clearing the Cache

Before starting vLLM, clear the filesystem cache on both Sparks. The main reason for doing this is the DGX Spark’s unified memory architecture (UMA): The operating system caches the model files it reads from disk in RAM. vLLM loads the model weights from here into GPU memory. After loading is complete, the data remaining in the cache is not used again but is not cleaned up immediately. In systems with separate memory, this is not significant. Since inference runs in GPU memory, RAM fullness does not affect performance. On the Spark, however, since the CPU and GPU share the same RAM, the cache reduces the space available to the GPU. The command clears this cache, providing maximum memory for vLLM.

Clear the cache on the Main Spark:

Apply the same cleanup on the Worker Spark via SSH from the Main Spark:

2. Launching the Model

Now, let’s launch the model on two Sparks:

The --tp 2 flag tells sparkrun to run the model split (tensor parallel) across two Sparks. The --rootful flag runs the container with root privileges — FlashInfer compiles CUTLASS attention kernels at runtime and requires root privileges for this compilation. The --no-follow flag causes sparkrun to return to the command line after launching the containers. sparkrun automatically synchronizes the image to the Worker (skips if the same ID), copies the model to the Worker (skips if already present), sets up the Ray cluster, configures NCCL for the CX-7 interfaces, and launches containers on both Sparks.

sparkrun successfully completed all 6 steps. The ‘Mode: cluster (2 nodes)’ section was seen. The image was synchronized to the Worker over CX-7. --tensor-parallel-size 2 appears in the serve command. sparkrun detected the --distributed-executor-backend ray flag, selected the vllm-ray runtime, and automatically set up the Ray cluster.

At TP=2, the model weights are split in two — each Spark loads ~30 GB (vs. 61 GB at TP=1). This leaves 35.97 GiB of memory for KV cache on each Spark (vs. 8.46 GiB at TP=1) — meaning more concurrent requests.

3. Monitoring the Startup Process

After the model download completes, it may take a few minutes for vLLM to become ready to serve. During this process, vLLM loads the model weights into GPU memory, compiles GPU kernels, and allocates memory for inference.

Monitor the logs:

You will see the following in the logs:

If you see the Application startup complete. line, the model server is ready. Press Ctrl+C to stop watching the logs. The server will continue running in the background.

4. Testing the Model

Check the container status. Containers should be running on both Sparks:

Both containers are running.

Let’s run a health check on the model:

The model is healthy. Now let’s test the model with a prompt:

The model answered 12 × 17 = 204. Additionally, the reasoning and reasoning_content fields contain the model’s reasoning processes.

The API is served only through the head node (Main Spark). That is, we communicate with the model via the Main Spark. The Worker Spark only participates in the inference computation.

5. Shutdown

When you are done, stop the model. Since you used --tp 2 when launching, you need to specify --tp 2 when stopping as well:

This command stops the container and frees up memory. However, the container, the Docker image, and the model files remain on disk. This means you don’t need to re-download to start again. Simply re-run the sparkrun run command from step 2.

Using with Open WebUI

In the previous tutorial (Local LLM Serving with vLLM on DGX Spark), we set up Open WebUI and connected it to vLLM’s port 8000. The GPT-OSS 120B model is also served from the same port 8000. If Open WebUI is running, it automatically detects the model and lists it as openai/gpt-oss-120b in the model selection menu on the chat screen. You can select the model and start using it from the browser.

Single vs Dual Spark Performance Comparison

We compared the single Spark (TP=1) and dual Spark (TP=2) configurations at different concurrency levels. The measurements recorded average TTFT (Time to First Token — the time until the first token starts being generated) and TPS (Tokens Per Second — the number of tokens generated per second) values.

The dual Spark setup provides 26% higher token throughput at concurrency 1 (single user), and as concurrency increases, this gap grows, exceeding 60% at concurrency 16, as shown.

The measurements were taken using the CordatusAI LLM Benchmark Tool. This tool is a benchmarking application developed by CordatusAI that tests LLM servers with OpenAI-compatible APIs. Below are screenshots of the benchmark results the application produced for our model. As can be seen, the application can test at concurrency levels from 1 to 64, and at each level it measures TTFT, ITL, TPS, latency, and throughput metrics, presenting the results as tables and graphs. It also calculates the recommended number of users the system can support. This calculation is a general estimate based on certain assumptions and does not reflect all usage scenarios. As seen in the image, the tool calculated a recommended user count of 55 for a single Spark and 120 for dual Sparks.

Single Spark benchmark results:

Double Spark benchmark results:

In this tutorial, you will accelerate data science and machine learning workflows on a DGX Spark using its GPU.

The Spark can be used as a standalone computer by connecting a monitor and keyboard, or as a remote server accessed from another computer. In this tutorial, we will connect to the Spark remotely and install the necessary software to run our workloads.

We will use pandas as our data processing library and scikit-learn as our machine learning library. These two libraries are foundational to the data science and machine learning ecosystem. In this tutorial, we will accelerate their workflows on the GPU using CUDA-X.

CUDA-X is the umbrella name for NVIDIA’s collection of GPU acceleration libraries. Within this collection, the set containing data science libraries is called RAPIDS. In this tutorial, we will use cuDF from the RAPIDS library suite to accelerate pandas, and cuML to accelerate scikit-learn.

This tutorial consists of three parts:

• Setup: Installing Miniconda, creating conda environments, and cloning the playbook repository
• Running the Notebooks: Running two notebooks in both CPU and GPU environments
• Comparison: Comparing CPU and GPU performance cell by cell

Setup

Connecting to the Spark

If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command in the terminal:

The command outputs the IP address of the Spark’s default network interface:

Note this address; you will use it in place of throughout this tutorial. Alternatively, you can find the IP address by checking the NVIDIA Sync application.

Make sure your computer is connected to the same network as the Spark. Then, open a terminal on your computer and connect to the Spark via SSH:

On the first connection, you will see a fingerprint warning. Type yes and press Enter. Then, when prompted for a password, enter the Spark’s password:

Once connected, the Spark will start accepting commands from this terminal. Throughout this tutorial, you will enter all commands in this terminal on your computer.

Installing Miniconda

Conda is a package manager that manages Python packages and environments. If conda is not installed on the Spark, you need to install Miniconda. Run the following commands:

Verify the installation:

Add conda to your PATH:

Creating Conda Environments

In this tutorial, we will use two separate conda environments on the Spark. The first contains standard data science packages — reflecting a typical setup you may already have. Let’s call this the CPU environment. The second contains the RAPIDS libraries. Let’s call this the GPU environment.

Create the CPU environment (standard data science packages, no RAPIDS):

Create the GPU environment (includes RAPIDS libraries):

This command installs the RAPIDS 26.06 package from the rapidsai channel, CUDA 13.0 runtime from the nvidia channel, and other packages from the conda-forge channel.

After installation completes, verify the package versions:

Cloning the Playbook Repository

NVIDIA shares its DGX Spark playbooks on its website and GitHub repository . The CUDA-X Data Science playbook there contains the two notebooks we will use.

Let’s look at the directory containing the notebooks:

Two notebooks are available:

• cudf_pandas_demo.ipynb — Accelerates pandas workflows with cuDF
• cuml_sklearn_demo.ipynb — Accelerates scikit-learn workflows with cuML

Kaggle API Key

The first notebook downloads a 4.76 GB dataset from Kaggle. To do this, you need a Kaggle API key (kaggle.json). If you don’t have a Kaggle account, sign up at kaggle.com and create an API key.

Copy the kaggle.json file from your computer to the Spark. Open a separate terminal on your computer and run the following command:

Then return to your SSH terminal and set the file permissions:

Preparing CPU Copies

To enable comparison, we will create a CPU copy of each notebook. The only difference in these copies is that the acceleration commands are commented out. All other code remains the same.

Now comment out the %load_ext line in each copy:

What Are cuDF and cuML?

In this section, before diving into the practical work, we will explain cuDF and cuML.

cuDF — pandas Acceleration

cuDF offers a “pandas accelerator mode” called cudf.pandas. This mode is activated by adding a single acceleration command to a Jupyter notebook:

When this command is run, the import pandas as pd statement loads a “proxy module” instead of the normal pandas. This proxy attempts to run every pandas operation on the GPU (via cuDF) first; if the operation is not supported on the GPU, it automatically falls back to the CPU (pandas) and synchronizes the data in the background.

Important: This mechanism requires no changes to user code. All pandas commands such as pd.read_csv(), df.merge(), and df.groupby() work as-is — they just run on the GPU.

cuML — scikit-learn Acceleration

cuML offers a “scikit-learn accelerator mode” called cuml.accel. This mode is also activated with a single acceleration command:

This command intercepts the import operations of the sklearn, umap-learn, and hdbscan modules. When you create an estimator (e.g., LinearSVC), a “proxy estimator” is created instead of the actual estimator. If cuML has a GPU implementation for that algorithm, the operation is dispatched to the GPU; otherwise, it automatically falls back to the CPU (scikit-learn).

Notebook 1: Accelerating pandas with cuDF

This notebook contains pandas data processing operations: loading, string operations, and grouping.

The cuDF accelerator mode (cudf.pandas) routes these operations to the GPU without requiring any code changes.

Running the Notebooks

We will run the notebooks in headless mode using jupyter nbconvert –execute. This method automatically runs all cells in the notebook and saves the outputs into the notebook file.

Running on GPU (RAPIDS environment, acceleration command active):

Running on CPU (standard environment, acceleration command disabled):

We are running the CPU copy we prepared in the “Preparing CPU Copies” section:

Notebook 2: Accelerating scikit-learn with cuML

This notebook contains various machine learning operations with scikit-learn: classification, clustering, and dimensionality reduction.

The cuML accelerator mode (cuml.accel) routes these operations to the GPU without requiring any code changes.

Running

Running on GPU:

Running on CPU:

Viewing Executed Notebooks with Jupyter

Executed notebooks have all cell outputs (timings, tables, charts) embedded within them. You can view them in your browser through a Jupyter server.

Start a Jupyter server on the Spark:

Open the URL shown in the output (including the token) in your computer’s browser. In the file list on the left, you will see four executed notebooks:

• cudf_pandas_demo_gpu_executed.ipynb — GPU run (cuDF)
• cudf_pandas_demo_cpu_executed.ipynb — CPU run (cuDF)
• cuml_sklearn_demo_gpu_executed.ipynb — GPU run (cuML)
• cuml_sklearn_demo_cpu_executed.ipynb — CPU run (cuML)

Click on any of them to view cell outputs, %%time timings, tables, and charts. You can open the CPU and GPU notebooks side by side in two browser tabs for comparison.

Comparison

In this section, we will compare important cells side by side and evaluate the results. In the screenshots, the CPU run is on the left and the GPU run is on the right.

Notebook 1: cuDF / pandas

In[4] — Acceleration Command

This cell is the only difference between the CPU and GPU runs. In the GPU notebook, the %load_ext cudf.pandas command replaces pandas with a proxy module. In the CPU notebook, this line is commented out (# %load_ext cudf.pandas), so normal pandas is used.

In[6] — Data Loading (4.76 GB)

Loads the 4.76 GB job_summary.csv file. This file contains the text summaries of 1.1 million LinkedIn job postings.

CPU pandas reads the dataset with a single thread, which is why it is slow. cuDF’s GPU CSV reader, on the other hand, loads the data in parallel into GPU memory. For a 4.76 GB file, the time difference is very significant.

In[7-8] — Data Loading + String Operation

Loads two smaller CSV files (job_skills.csv 642 MB, linkedin_job_postings.csv 397 MB, totaling ~1 GB) and then calculates the length of 1.1 million strings. Calculating string length is a common operation when working with text data.

In[7] — Data loading (2 CSVs, ~1 GB):

In[8] — String operation (str.len() on 1.1M strings):

In string operations, the GPU processes thousands of strings simultaneously rather than calculating each string’s length one by one.

In[11] — Aggregation (groupby + sort)

Groups 1.1 million rows by job_title and job_location columns, calculates the mean, and sorts by three columns.

The GPU is significantly faster in this operation.

Notebook 2: cuML / scikit-learn

In[2] — Acceleration Command

As in the previous notebook, this cell is the only difference between the CPU and GPU runs. In the GPU notebook, the %load_ext cuml.accel command intercepts scikit-learn/umap-learn/hdbscan imports.

In[7] — Classification: LinearSVC Training

Trains a Linear Support Vector Classifier on the UCI Covertype dataset. The dataset contains 581 thousand samples and 55 features.

Model training is the most time-consuming step in machine learning. The GPU performs LinearSVC’s matrix operations in parallel, significantly reducing training time. This is of great importance in scenarios where you need to train multiple models, such as hyperparameter search, which involve heavy computational workloads.

In[12] — CPU Fallback: KernelDensity

 

Note: This screenshot is taken from the GPU notebook only (no side-by-side comparison).

This cell is used to demonstrate cuML’s CPU fallback mechanism. The KernelDensity algorithm does not have a GPU implementation in cuML. The notebook deliberately calls this algorithm to show that unsupported operations gracefully fall back to the CPU.

In[16] — Clustering: HDBSCAN

HDBSCAN is a density-based clustering algorithm. Here it runs on synthetic data (20 thousand samples, 100 features, 5 clusters).

This is the largest speedup recorded during this tutorial. HDBSCAN involves computationally intensive steps such as density calculations and minimum spanning tree construction. These steps can be accelerated dozens of times by leveraging the GPU’s parallel processing power.

In[25] — Dimensionality Reduction: UMAP

UMAP reduces high-dimensional data to a lower dimension (here, 561D → 2D). The UCI HAR dataset (7,352 samples, 561 features) contains human activities recorded from smartphone sensors.

UMAP involves building a k-NN graph and various optimization steps. These steps provide significant parallelization advantages on the GPU.

Conclusion

In this tutorial, we accelerated pandas and scikit-learn workflows on the DGX Spark using CUDA-X libraries (cuDF, cuML) on the GPU. We ran two notebooks in both CPU and GPU environments to make a real performance comparison.

The results obtained:

The most striking results were achieved with the HDBSCAN (73.6x) and UMAP (37.9x) algorithms. This is because these algorithms involve intensive mathematical computations, which is where the GPU’s parallel processing power provides a significant advantage.

These results show that by adding a single command to your pandas and scikit-learn code running on the Spark, you can achieve significant speedups. We also recommend trying these notebooks on your own computer. This is because whether GPU acceleration is present or not, the performance of the DGX Spark’s 20-core ARM CPU may surprise you!

Shutdown

When you are done, stop the Jupyter server with Ctrl+C. The conda environments you created on the Spark (cpu-baseline and rapids-test), the executed notebooks, and the downloaded datasets remain on disk — nothing is deleted.

To view the notebooks again, restart the Jupyter server:

When the server starts, open the URL containing the token shown in the terminal in your browser.