NVIDIA DGX Spark Ecosystem

Resources, Tools, and Community

NVIDIA DGX Spark™ is a compact desktop AI system designed for working with artificial intelligence models. It brings development, testing, inference, fine-tuning, and prototyping workflows together on a single platform.

The software and knowledge ecosystem surrounding the device is extensive. Models, Docker images, NVIDIA playbooks, and forums are only a few of its components. DGX Spark has also developed a broad developer community. Community-built images, multi-device recipes, tools, software improvements, and performance rankings have expanded the ecosystem well beyond NVIDIA’s official resources.

Models

Models intended to run on DGX Spark are hosted on Hugging Face, and many are available in multiple quantization formats. A single model may have FP8, NVFP4, MXFP4, or INT4 variants on Hugging Face. DGX Spark provides hardware support for all of these formats, allowing compatible models to be run after download without requiring an additional conversion step.

Among these quantization formats, NVFP4 is an FP4 format specifically designed for the Blackwell architecture used by DGX Spark. By representing model weights at a lower bit width, it substantially reduces memory requirements. For example, a 200-billion-parameter model that would require approximately 400 GB of memory in a standard 16-bit format can be reduced to roughly one-quarter of that size with NVFP4. This makes it possible to run significantly larger models within the system’s 128 GB of unified memory.

Hugging Face hosts a collection of nearly 100 NVFP4 models converted by NVIDIA. The community has also published a wide range of models specifically prepared for Spark.

Inference Engines

Inference engines are software systems that load model weights into GPU memory and expose APIs for accepting external requests. These APIs can then be used by Open WebUI, Odysseus, or any other compatible client. Common inference engines include vLLM, SGLang, TensorRT-LLM, and llama.cpp.

The most widely used inference engine on DGX Spark is vLLM. One of the main reasons is its strong support for distributed execution. In particular, its robust multi-node tensor parallelism capabilities make it well suited to multi-Spark deployments. Other important features include PagedAttention for efficient GPU memory utilization, prefix caching for reusing computations across repeated prompts, and support for lower-precision KV cache formats such as FP8 and NVFP4.

vLLM is commonly run inside Docker containers. Docker images are used to package the runtime together with its GPU dependencies. Official vLLM images are published on Docker Hub under vllm/vllm-openai.

Community

The GB10 chip used in DGX Spark has an SM 12.1 compute capability, which differs from that of conventional NVIDIA GPUs. Official Docker images for inference engines such as vLLM may not include native kernels for this architecture. In other words, models may still run with official vLLM images, but some workloads may not fully utilize the performance capabilities of the Spark GPU. For this reason, specialized images and deployment recipes are sometimes required.

Developers

Community developers build and publish Docker images containing native SM 12.1a kernels for GB10. As a result, community-maintained images are frequently used alongside official vLLM images when working with DGX Spark.

Some of the most prominent developers and their contributions to the Spark ecosystem are summarized below. New contributors continue to appear regularly:

• eugr: spark-vllm-docker — General-purpose inference images, MXFP4 builds, and the B12X kernel family
• christopherowen: vllm (fork) — Source code for CUTLASS and FlashInfer kernels
• local-inference-lab: vllm (fork) — Source of the B12X kernel family and experimental high-performance kernels for the sm12x architecture
• tonyd2wild: tonyd2wild — Model-specific images, multi-Spark deployment recipes for DeepSeek V4 Flash, GLM-5.2, MiMo V2.5, and MiniMax-M3, as well as DSpark speculative decoding patches
• mpfaffenberger: mpfaffenberger — Runtime modifications addressing model compatibility issues
• MiaAI-Lab: MiaAI-Lab — Community deployment recipes for DeepSeek V4 Flash, GLM-5.2, and Laguna S, as well as the sparkDash monitoring tool

In addition, Spark Arena maintains an LLM performance leaderboard ranking the performance of different models and inference engines on DGX Spark.

Tools

Docker alone is sufficient for running models on a single DGX Spark. With multiple Sparks, however, manually managing Docker containers, creating the cluster, and configuring the CX-7 network, SSH mesh, and NCCL communication can become considerably more complex. The community has developed several tools to simplify this process.

eugr/spark-vllm-docker, developed by eugr, is one of the most comprehensive toolsets for building Docker images with native SM 12.1a kernels for GB10. It includes build scripts, cluster initialization, recipe deployment, model downloading, node discovery, runtime modifications, and ready-to-use recipes. These tools are widely used by the community.

sparkrun, developed by Spark Arena, automates cluster management. Its setup wizard configures the CX-7 network, SSH mesh, and NCCL environment with a single command. A YAML recipe file can then be used to define the model, Docker image, inference parameters, and memory configuration. sparkrun also provides comprehensive real-time cluster monitoring and has become a widely used cluster-management tool within the DGX Spark ecosystem.

OpenZeka Resources

At OpenZeka, we publish Turkish and English tutorials covering our work with DGX Spark. English-language articles are available at blog.openzeka.com/en/. The blog primarily focuses on user education, including tutorials, step-by-step model and tool installation instructions, and practical usage guides. Some of our DGX Spark articles are listed below:

• Serving a Local LLM with vLLM on DGX Spark: Single Spark, vLLM, Open WebUI
• Local GPT-OSS 120B Serving on DGX Spark with sparkrun: Single/dual Spark, GPT-OSS 120B, vLLM, sparkrun
• DeepSeek-V4-Flash-0731 on 2x DGX Spark: 2× Spark, DeepSeek-V4-Flash-0731, vLLM, sparkrun
• Local AI Workspace with Odysseus on DGX Spark: Local AI workspace, agents, RAG, tool use
• Local Model Fine-Tuning with Unsloth on DGX Spark: Fine-tuning, Unsloth, QLoRA
• GPU-Accelerated Data Science and Machine Learning on DGX Spark: CUDA-X, RAPIDS, cuDF, cuML
• Building a High-Speed AI Cluster with 4 DGX Spark Systems: 4× Spark cluster, 200GbE, RDMA, RoCEv2

Our white papers are published at whitepapers.openzeka.com. Unlike the blog’s user-education-oriented content, the white papers target a more technical and professional audience. They provide in-depth analyses intended to support decision-making in areas such as hardware selection, benchmark interpretation, cluster architecture comparison, and scaling strategies. Some of our white papers are listed below:

• Local LLM Usage Guide: Hardware, model, and software selection guide
• Qwen3.6-27B DGX Spark Benchmark: FP8, AWQ, and NVFP4 performance analysis
• Qwen3.6-27B DGX Spark Cluster Scaling: 1×/2×/4× Spark scaling analysis
• DGX Spark 2-Node Cluster Setup Guide: Two-node cluster setup
• DGX Spark 3-Node Cluster Setup Guide: Three-node ring topology
• DGX Spark 4-Node Cluster Setup Guide: Four-node switch topology

We also publish a variety of tutorials, tests, and product-analysis videos on the OpenZeka and CordatusAI YouTube channels.

Forums and Documentation

The DGX Spark / GB10 category of the NVIDIA Developer Forums is a community discussion space in which NVIDIA engineers also actively participate. It contains hundreds of discussions covering model deployment recipes, performance optimization, newly released models, and many other topics. It is one of the first resources worth consulting when encountering a DGX Spark-related issue. The NVIDIA Developer page brings together links to this forum, official documentation, and playbooks in one place.

Official DGX Spark documentation is available at docs.nvidia.com/dgx/dgx-spark. The User Guide covers hardware specifications, DGX OS, ConnectX-7 clustering, system recovery, and known issues, with dedicated pages for hardware specifications, OS and component updates, and release notes. For multi-Spark deployments, NVIDIA provides documentation for the NVIDIA Sync Cluster Assistant and cluster network inspection. For performance analysis, documentation is available for both Nsight Systems and Nsight Compute.

Spark Playbook Catalog

NVIDIA provides a collection of Spark Playbooks for DGX Spark. Each playbook contains step-by-step installation and execution instructions for a particular tool or workflow. Some of these tools directly run models, while others provide the surrounding infrastructure required to make those models usable in practical applications.

Inference Engines

Models are typically developed using machine learning frameworks such as PyTorch or JAX. Inference engines load or convert the resulting model weights and optimize GPU kernels, KV cache management, batching, and memory utilization to execute the model efficiently. Some also provide an OpenAI-compatible API through either a built-in or separate model-server component.

• TensorRT-LLM: trt-llm — TensorRT-LLM is an open-source inference library optimized for LLM execution on NVIDIA GPUs. It uses optimized kernels and supports tensor, pipeline, and sequence parallelism. It integrates with Hugging Face and PyTorch.
• vLLM: vllm — vLLM is a high-throughput inference engine that uses PagedAttention and continuous batching to improve memory efficiency. It also provides an OpenAI-compatible model server.
• llama.cpp: llama-cpp — llama.cpp is a lightweight C/C++ inference engine for running GGUF models across CPUs and multiple GPU backends. Its llama-server component can expose an OpenAI-compatible API.
• SGLang: sglang — SGLang is a high-performance inference and model-serving framework for LLMs and VLMs. It jointly optimizes the backend runtime and model interaction layer.
• NVIDIA NIM: nim-llm — NIM is NVIDIA’s solution for packaging inference engines such as those listed above into containerized HTTP microservices. Rather than being an inference engine itself, it acts as a packaging and deployment layer that prepares these engines for production and enterprise use.

Applications and Tools

These tools allow users and developers to interact directly with models. Live VLM WebUI and Continue connect to an existing backend service, whereas Ollama and LM Studio also include their own inference runtimes.

• Ollama + Open WebUI: open-webui — Ollama is a local model-serving layer built on llama.cpp that provides a CLI, model library, and OpenAI-compatible API. Open WebUI is a self-hosted web interface that connects to this service.
• LM Studio: lm-studio — LM Studio is a local AI application that combines model discovery, execution, and API-based serving.
• Live VLM WebUI: live-vlm-webui — Live VLM WebUI is a web interface that sends webcam streams to Ollama, vLLM, SGLang, or cloud-hosted VLM services. It allows the outputs and performance of different backend models to be compared.
• Continue + VS Code: vibe-coding — Continue is a coding client that runs inside VS Code. In the playbook, Ollama serves the model while Continue connects to that service for chat and code generation.

Models and Inference Optimizations

Nemotron is NVIDIA’s model family. Speculative decoding and NVFP4, meanwhile, are techniques intended to improve the performance or memory efficiency of compatible inference engines.

• NVIDIA Nemotron: nemotron — Nemotron is NVIDIA’s open model family designed for reasoning, tool use, and long-context tasks. It can be served with llama.cpp, vLLM, SGLang, or TensorRT-LLM.
• Speculative decoding: speculative-decoding — Speculative decoding accelerates the decoding process by having a smaller, faster draft model propose tokens that are then verified in batches by the target model. The playbook demonstrates this technique with EAGLE-3 and Draft–Target methods in TensorRT-LLM.
• NVFP4 quantization: nvfp4-quantization — NVFP4 is a Blackwell-compatible 4-bit quantization format for model weights and activations. The playbook converts the model with TensorRT Model Optimizer and then runs it in a TensorRT-LLM environment.

Fine-Tuning Frameworks and Tools

Fine-tuning tools use existing model weights to produce updated model weights. The resulting model can subsequently be run using an inference engine.

PyTorch is the underlying machine learning framework. NeMo, LLaMA-Factory, and Unsloth provide more streamlined and optimized training workflows built on top of PyTorch.

• NVIDIA NeMo: nemo-fine-tune — NeMo AutoModel is a framework for training Hugging Face models with native PyTorch support and scaling them to multi-node systems. It supports SFT, PEFT, and distributed training workflows.
• LLaMA-Factory: llama-factory — LLaMA-Factory is a fine-tuning framework that provides a common interface for optimization methods such as SFT, LoRA, and QLoRA across different LLMs and VLMs.
• Unsloth: unsloth — Unsloth is a fine-tuning toolkit that uses custom GPU kernels to accelerate LoRA and QLoRA training while reducing memory consumption.
• PyTorch: pytorch-fine-tune — PyTorch is a foundational machine learning framework that provides tensor operations, automatic differentiation, and GPU acceleration. This playbook implements SFT, LoRA, and QLoRA workflows directly in PyTorch at a lower level of abstraction.
• FLUX.1 DreamBooth LoRA: flux-finetuning — This playbook uses DreamBooth LoRA to teach new concepts and styles to the FLUX.1-dev diffusion image-generation model.

Agentic AI Systems

AI agents connect to an LLM service for decision-making while interacting with files, terminal commands, APIs, and other tools. The model itself is run by a service such as vLLM or Ollama, while the agent application manages task planning, memory, and tool use.

OpenClaw and Hermes are agent applications. OpenShell is a sandbox runtime that surrounds the agent process with filesystem, network, permission, and security policies. NemoClaw combines these layers into an integrated deployment.

• OpenClaw: openclaw — OpenClaw is a persistent local agent application combining memory, filesystem access, tool use, and skills. In the playbook, its decision-making model is served through vLLM.
• Hermes Agent: hermes-agent — Hermes is an autonomous agent capable of generating reusable skills from its experiences, retaining memory across sessions, and running scheduled tasks. In the playbook, vLLM is used as the local model service.
• OpenShell: openshell — OpenShell is a sandbox runtime that constrains agents at the kernel level. The playbook runs OpenClaw inside the sandbox while the model service is accessed through an endpoint outside the sandbox.
• NemoClaw: nemoclaw — NemoClaw combines the OpenClaw agent, OpenShell sandbox, and local vLLM inference. Filesystem, network, process, and inference access are configured within a unified deployment.
• NemoClaw example agents: nemoclaw-applications — This playbook deploys news summarization, software development, document review, and calendar negotiation agents on top of an existing NemoClaw sandbox, demonstrating how the same infrastructure can be adapted to different applications.
• CLI Coding Agents: cli-coding-agent — Claude Code, OpenCode, and Codex CLI are terminal-based agents that manage file editing, command execution, and coding tasks. In the playbook, model inference is provided by Ollama.

Complete AI Applications and Solutions

These applications combine models, inference engines, data processing, user interfaces, and agent components to solve specific problems. Instead of interacting with individual infrastructure layers, users interact with a complete end-to-end system.

• RAG Application in AI Workbench: rag-ai-workbench — An agentic RAG application combining query routing, information retrieval, answer generation, and hallucination evaluation. It can use either NVIDIA-hosted APIs or self-hosted model services.
• Video Search and Summarization: vss — VSS uses VLM, LLM, and RAG components to generate video summaries, question-answering capabilities, and real-time alerts.
• Multi-Agent Chatbot: multi-agent-chatbot — A supervisor agent coordinates coding, RAG, and visual-understanding agents. llama.cpp and TensorRT-LLM servers are used for model serving, while MCP is used for tool connectivity.
• Image Generation with TensorRT: multi-modal-inference — This playbook optimizes the FLUX.1 and SDXL diffusion models with TensorRT for text-to-image generation.
• Text to Knowledge Graph: txt2kg — Extracts subject–predicate–object relationships from text using Ollama, stores the resulting data in ArangoDB, and visualizes it with Three.js WebGPU. Ollama performs inference while ArangoDB handles relationship queries.
• Spark & Reachy Photo Booth: spark-reachy-photo-booth — A multimodal application combining NeMo Agent Toolkit, TensorRT-LLM, speech recognition, speech synthesis, FLUX image generation, and object tracking with the Reachy Mini robot. Services communicate through a message bus.

Other GPU-Accelerated Workloads

These playbooks are not part of AI inference or model-serving workflows. Instead, they demonstrate how DGX Spark’s computational acceleration capabilities can be applied to areas such as data science, optimization, bioinformatics, and robotics.

• CUDA-X Data Science: cuda-x-data-science — CUDA-X Data Science uses RAPIDS libraries such as cuDF and cuML to accelerate pandas and scikit-learn workflows on the GPU.
• Single-cell RNA Sequencing: single-cell — RAPIDS-singlecell provides a Scanpy-like API for running scRNA-seq preprocessing, quality control, clustering, and visualization on the GPU. It uses RAPIDS components such as cuML and cuGraph.
• Portfolio Optimization: portfolio-optimization — cuOpt solves portfolio constraints as LP/MILP optimization problems, while cuML accelerates the generation of risk scenarios. The playbook combines these tools in an end-to-end Mean-CVaR-based financial workflow.
• Isaac Sim and Isaac Lab: isaac — Isaac Sim is a GPU-accelerated robotics simulation platform. Isaac Lab is a framework built on top of this environment for developing reinforcement-learning policies.

Multi-DGX Spark Deployments

These playbooks prepare multi-device infrastructure independently of the model or application layer. First, the QSFP network and inter-node SSH access are configured. NCCL then provides high-performance collective GPU communication over this connection for distributed training and inference.

Higher-level software such as vLLM, TensorRT-LLM, PyTorch, and NeMo can use this infrastructure when operating in distributed configurations.

• 2× DGX Spark direct connection: connect-two-sparks — Connects two DGX Spark systems directly through a 200 GbE QSFP link and passwordless SSH, creating a two-node distributed system.
• 3× DGX Spark ring connection: connect-three-sparks — Connects three DGX Spark systems in a ring topology using three QSFP cables.
• 4+ DGX Spark switch connection: multi-sparks-through-switch — Connects four or more DGX Spark systems through a QSFP switch to create a scalable cluster.
• NCCL: nccl — NCCL is NVIDIA’s library for high-performance collective communication between GPUs located on different nodes.

All of these playbooks are available at build.nvidia.com/spark.

Conclusion

DGX Spark has a strong and rapidly evolving ecosystem built on NVIDIA’s foundations and continuously expanded through a growing range of community projects. OpenZeka contributes to this ecosystem by making its technical knowledge more accessible to end users through Turkish- and English-language tutorials, technical analysis documents, practical guides, and YouTube videos.