<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>OpenZeka EN Blog</title>
	<atom:link href="https://blog.openzeka.com/en/feed/" rel="self" type="application/rss+xml" />
	<link>https://blog.openzeka.com/en/</link>
	<description>NVIDIA Jetson Developer Kits &#38;Edge Devices</description>
	<lastBuildDate>Tue, 06 Oct 2026 11:06:18 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://blog.openzeka.com/en/wp-content/uploads/2026/06/cropped-favicon-32x32-3-66x66.webp</url>
	<title>OpenZeka EN Blog</title>
	<link>https://blog.openzeka.com/en/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Who Spoke, and When? Speaker Diarization on Jetson with NVIDIA Nemotron-3-Diarization</title>
		<link>https://blog.openzeka.com/en/speech-studio-transcribing-recordings-on-a-jetson/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Tue, 06 Oct 2026 10:30:05 +0000</pubDate>
				<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=2012</guid>

					<description><![CDATA[<p>Finding one particular exchange in a meeting, intervie ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/speech-studio-transcribing-recordings-on-a-jetson/">Who Spoke, and When? Speaker Diarization on Jetson with NVIDIA Nemotron-3-Diarization</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-1 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-0 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-blend:overlay;--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-1"><p>Finding one particular exchange in a meeting, interview or podcast can mean listening through the entire recording. The answer to “who said what?” is in the file, but reaching it takes time. Cloud services make this easier, but they require you to upload the recording to another server.</p>
<p><strong>Speech Studio</strong> is an open-source tool that does this work <strong>locally on an NVIDIA Jetson Orin Nano Developer Kit</strong>. It transcribes recordings, separates speakers, summarizes the content and answers questions about the recording. The audio file stays on the device throughout the process.</p>
<p>At the heart of the app is NVIDIA’s speaker diarization model, <strong><a style="color: green;" href="https://huggingface.co/nvidia/Nemotron-3-Diarization">Nemotron-3-Diarization</a></strong>. It answers the question “who spoke when?”, and it gets its own section below.</p>
<p>The source code is available on GitHub: <strong><a style="color: green;" href="https://github.com/openzeka/speech-studio">github.com/openzeka/speech-studio</a></strong></p>
<p>The measurements and screenshots in this post were taken on an <strong>8 GB NVIDIA Jetson Orin Nano Developer Kit</strong>.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-1 hover-type-none"><img fetchpriority="high" decoding="async" width="1500" height="1050" alt="Speech Studio with a recording open" title="1-konsol-ekrani-en" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1.webp" class="img-responsive wp-image-2020" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-200x140.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-300x210.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-400x280.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-600x420.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-768x538.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-800x560.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-1024x717.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1-1200x840.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-konsol-ekrani-en-1.webp 1500w" sizes="(max-width: 640px) 100vw, 1500px" /></span></div><div class="fusion-text fusion-text-2"><h2></h2>
<h2>A real recording</h2>
<p>The main test used a 20 min 30 s recording taken from <a style="color: green;" href="https://www.youtube.com/watch?v=KT89PEe9d7U">a conversation between NVIDIA and OpenAI about AI infrastructure</a>.</p>
<p>Speech Studio found <strong>3 speakers and 187 speaker turns</strong>. It matched each part of the transcript with a speaker and a timestamp:</p>
<blockquote>
<p><strong>[00:09]</strong> <em>Speaker 3:</em> So one of the things we wanted to talk about today is the concept of building AI infrastructure at scale.</p>
</blockquote>
<p>Labels such as “Speaker 3” can be replaced with a person’s name. The app also produces a summary, key points and action items. When you ask a question about the recording, it points to the relevant passages with timestamps.</p>
<p>Transcripts and summaries are generated by AI and can contain mistakes. Proper names are especially prone to errors. Timestamps make it easier to check important details against the original recording.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-2 hover-type-none"><img decoding="async" width="1500" height="1850" alt="Transcript, summary and Q&amp;A panel" title="2-konusma-detay-en" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1.webp" class="img-responsive wp-image-2021" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-200x247.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-243x300.webp 243w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-400x493.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-600x740.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-768x947.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-800x987.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-830x1024.webp 830w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-1200x1480.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1-1245x1536.webp 1245w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-konusma-detay-en-1.webp 1500w" sizes="(max-width: 640px) 100vw, 1500px" /></span></div><div class="fusion-text fusion-text-3"><h2></h2>
<h2>The model that separates speakers: <span style="color: #3ebc00;">Nemotron-3-Diarization</span></h2>
<h3>Three different questions you can ask a recording</h3>
<p>It helps to sort speech AI models by the question they answer:</p>
<ul>
<li><strong>Speech recognition (ASR)</strong> — Input: Audio, Output: Text — “What was said?”</li>
<li><strong>Text-to-speech (TTS)</strong> — Input: Text, Output: Audio — “How should this text sound?”</li>
<li><strong>Speaker diarization</strong> — Input: Audio, Output: Speaker timeline — <strong>“Who spoke when?”</strong></li>
</ul>
<p>ASR works like a court stenographer: it writes down every word it hears, but it doesn’t tell you who said them. TTS goes the other way, taking text and giving it a voice. Diarization produces neither text nor audio. What it produces is a map: “The first voice spoke from 00:00 to 00:12, the second voice cut in at 00:12, the first voice came back at 00:30.”</p>
<p>To do this, it doesn’t look at what the words mean; it tells the voices themselves apart: timbre, pitch, manner of speaking. It also doesn’t know who anyone is. It never says “Alice” or “Bob”, only “this voice is different from that one.” You add the names afterwards.</p>
<h3>What problem does it solve?</h3>
<p>With ASR alone, the transcript of a 20-minute meeting arrives as one long block of text. Somewhere in it is the sentence “I can have this done by Friday.” But who made that promise? Who raised that objection? The transcript doesn’t say.</p>
<p>Diarization turns that text into a conversation. Every sentence is tied to the person who said it, so you can see how much each person talked, read one person’s contributions in a row, and attribute decisions and commitments to the right people. It is also what lets the summary and Q&amp;A answer questions like “who suggested this?”</p>
<h3>Why is it hard?</h3>
<p>It sounds simple, but it is one of the hard problems in speech processing:</p>
<ul>
<li><strong>The speakers aren’t known in advance.</strong> The model starts without knowing how many people are in the recording or who they are.</li>
<li><strong>People talk over each other.</strong> In a real conversation, people interrupt and chime in with short “yeah”s and “mm-hm”s.</li>
<li><strong>Voices can be similar.</strong> Telling apart two people of the same age and gender is much harder than separating clearly different voices.</li>
<li><strong>It has to stay consistent.</strong> Whoever is “Speaker 2” in minute one must still be “Speaker 2” when they speak again in minute twenty.</li>
</ul>
<h3>How the model works inside</h3>
<p>Classic systems tackle this as a chain of steps: first find where speech occurs, then extract a “voice fingerprint” from each segment, and finally group the fingerprints by similarity. Each step inherits the previous step’s mistakes, and overlapping speech fits this design poorly, since each segment usually gets a single speaker.</p>
<p>Nemotron-3-Diarization belongs to NVIDIA’s <strong>Sortformer</strong> family and does the whole job in one pass. It listens to the recording in very short time slices and, for each slice, answers eight questions at once: “Is speaker one talking right now? Speaker two? …” The result looks like an eight-track recorder with a separate track for every speaker. Because several tracks can be active at the same moment, overlapping speech is marked naturally. The colored timeline in Speech Studio is those tracks made visible.</p>
<p>A known problem with this kind of model is that labels have no inherent order. Even when the model separates two voices correctly, which one it calls “1” and which “2” is arbitrary, which makes both training and consistency harder. Sortformer fixes this with a simple rule: <strong>whoever speaks first in the recording is “1”, the next person to join is “2”</strong>. (The numbers Speech Studio shows don’t always follow that order: in the sample recording, the first person to speak was labelled “Speaker 3”. Either way, labels can be renamed to people’s names in the interface.)</p>
<p>The model also works on live audio. In that mode it keeps the speakers it has already heard in a small memory (the <em>Arrival-Order Speaker Cache</em>), so when someone speaks again minutes later it recognizes them and gives them the same label. Latency can go as low as 0.32 seconds.</p>
<h3>How was it trained, and how well does it do?</h3>
<p>The model was trained on about <strong>10,000 hours of real conversations</strong> and <strong>82,611 hours of simulated multi-speaker audio</strong>. The simulated mixtures matter: by overlaying different recordings, the model was shown plenty of interruptions and simultaneous speech. The training data covers many languages, led by English, Mandarin and several Indian languages.</p>
<p>Accuracy is measured with the <em>Diarization Error Rate</em> (DER): missed speech, speech detected where there is none, and speech assigned to the wrong person, as a percentage of total time. Lower is better. Results NVIDIA publishes on the <a style="color: green;" href="https://huggingface.co/nvidia/Nemotron-3-Diarization">model card</a>:</p>
<ul>
<li><strong>NOTSOFAR1</strong> — meeting recordings — DER 6.77%</li>
<li><strong>CALLHOME (Part 2)</strong> — telephone calls — DER 9.10%</li>
<li><strong>DIHARD III</strong> — hard recordings from many domains — DER 12.73%</li>
</ul>
<p>These numbers come from the mode that processes the whole recording. In live streaming, even at the lowest latency (0.32 s), the error on DIHARD III stays at 13.55%.</p>
<p>It does all this with about <strong>100 million parameters</strong>, tiny next to today’s language models. That is what lets it run on the 8 GB Orin Nano alongside the speech recognition and summarization models. It distinguishes up to 8 speakers and is licensed under <a style="color: green;" href="https://huggingface.co/nvidia/Nemotron-3-Diarization">OpenMDW 1.1</a>, which also permits commercial use.</p>
<h3>How does Speech Studio use it?</h3>
<p>Speech Studio runs two models side by side. ASR writes out every word with a timestamp; Nemotron-3-Diarization says who was speaking at that moment. Combining the two ties each word to its speaker. Speech Studio then gathers consecutive words from the same person into speaker turns, and the timeline, transcript, summary and Q&amp;A are all built on those turns. Both models run on the Jetson’s GPU inside NVIDIA’s local inference engine, <a style="color: green;" href="https://github.com/NVIDIA/NeMo-Speech.cpp">NeMo-Speech.cpp</a>.</p>
<p>For more on speaker diarization, see NVIDIA’s <a style="color: green;" href="https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/asr/speaker_diarization/intro.html">NeMo documentation</a>.</p>
<h2>What can you do with it?</h2>
<p>Drop a WAV, MP3, FLAC, OGG, Opus or M4A file into the interface to:</p>
<ul>
<li><strong>Separate speakers.</strong> Each speaker appears in a different color and timeline lane.</li>
<li><strong>Review the transcript.</strong> Click a timestamp to listen to that part of the recording.</li>
<li><strong>Get a concise summary.</strong> Key points, decisions and action items are collected in one place.</li>
<li><strong>Ask questions about the recording.</strong> Follow timestamps to the passages behind the answers.</li>
<li><strong>Work in Turkish or English.</strong> The interface, summaries and answers follow the selected language.</li>
<li><strong>Export the result as Markdown.</strong> The transcript and summary can be downloaded as one file.</li>
</ul>
<h2>How does it work?</h2>
<p>Speech Studio brings speech processing and language-model features together in one interface. The audio is processed on the Jetson; the recording does not need to be sent elsewhere for transcription, speaker separation, summarization or Q&amp;A.</p>
<p>The required models are downloaded from the internet during setup. After installation, recordings are processed on the device. By default, the web interface is available only on the device itself; an SSH tunnel can be used to connect from another computer.</p>
<h2>Test results and limitations</h2>
<p>In tests with different recordings, the system found:</p>
<ul>
<li>3 speakers and 187 turns in a 20 min 30 s recording,</li>
<li>7 speakers in a 1 min 38 s synthetic dialogue,</li>
<li>3 speakers in roughly one-minute clips from a three-person podcast,</li>
<li>a Turkish transcript and summary from a Turkish recording.</li>
</ul>
<p>Results vary with the model and the recording. Summary quality depends on the selected language model, especially when using a smaller model or summarizing into another language. The models support live audio, but the current interface works with uploaded files.</p>
<p>Recordings are stored in the device’s <code>library/</code> folder. Deleting those files removes the local copies kept by the app.</p>
<h2>Conclusion</h2>
<p>Speech Studio brings transcription, speaker separation, summarization and recording search together in one interface. For people who want to process recordings on their own hardware, the Jetson Orin Nano provides a compact platform for running these tasks locally.</p>
<p>To try it, explore the code or contribute: <strong><a style="color: green;" href="https://github.com/openzeka/speech-studio">github.com/openzeka/speech-studio</a></strong></p>
</div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/speech-studio-transcribing-recordings-on-a-jetson/">Who Spoke, and When? Speaker Diarization on Jetson with NVIDIA Nemotron-3-Diarization</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>An Abdominal CT Expert and a 32B Medical AI in One DGX Spark: MedPortal</title>
		<link>https://blog.openzeka.com/en/an-abdominal-ct-expert-and-a-32b-medical-ai-in-one-dgx-spark-medportal/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Mon, 05 Oct 2026 10:10:34 +0000</pubDate>
				<category><![CDATA[Uncategorized]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=2000</guid>

					<description><![CDATA[<p>﻿     A contrast-enhanced abdominal CT can contain hund ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/an-abdominal-ct-expert-and-a-32b-medical-ai-in-one-dgx-spark-medportal/">An Abdominal CT Expert and a 32B Medical AI in One DGX Spark: MedPortal</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p><div class="fusion-fullwidth fullwidth-box fusion-builder-row-2 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-1 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-4"><p><span data-mce-type="bookmark" style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" class="mce_SELRES_start">﻿</span></p>
</div></div></div></div></div><div class="fusion-fullwidth fullwidth-box fusion-builder-row-3 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-2 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-blend:overlay;--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-5"><p>A contrast-enhanced abdominal CT can contain hundreds of slices, dozens of organs, and a long list of potential findings that must not be missed. That is the everyday reality of radiology. Now imagine reviewing that information on one desktop system, while patient data never leaves the premises.</p>
<p>That is the idea behind <strong>MedPortal</strong>, built at OpenZeka. It unifies two large medical AI models in one web interface, both running on a single <strong>NVIDIA DGX Spark</strong>. This article looks at the system, its output on a live case, and the reasons DGX Spark is a strong fit for this workflow.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-3 hover-type-none"><img decoding="async" width="1280" height="577" alt="Completed RADAR analysis: CT slice viewer on the left and thresholded findings on the right" title="1-radar-sonuclar" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar.webp" class="img-responsive wp-image-1994" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-200x90.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-300x135.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-400x180.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-600x270.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-768x346.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-800x361.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-1024x462.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar-1200x541.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/1-radar-sonuclar.webp 1280w" sizes="(max-width: 640px) 100vw, 1280px" /></span></div><div class="fusion-text fusion-text-6"><p><em>Live-system screenshot: RADAR&#8217;s official demo case (AC423ccbe) was run end to end. At slice 184/368 and a 0.50 threshold, three findings were surfaced: cardiomegaly 0.83, calcification 0.70, and atherosclerosis 0.58.</em></p>
<h2>Two demanding models, one system</h2>
<p>MedPortal keeps two high-capacity models available on the same DGX Spark:</p>
<ul>
<li><strong>RADAR</strong> provides expert-level analysis for contrast-enhanced abdominal CT. Upload a <code>.nii.gz</code> volume and it returns a score for <strong>each of 146 findings</strong>, along with an interactive slice viewer and CSV export. Published in <em>Science</em> (2026), the model was trained on more than 400,000 contrast-enhanced abdominal CT examinations and 15 million anatomy-aware image-text pairs.</li>
<li><strong>ClinFusion-32B</strong> is a multimodal medical language model with strong visual capabilities, based on Qwen3-VL-32B. It accepts natural-language questions, 2D images, and 3D CT/MRI volumes, then preserves conversational context while the case is discussed.</li>
</ul>
<p>Running both together is the key challenge. RADAR uses a 3D visual encoder, while ClinFusion carries roughly <strong>180 GB of weights</strong> across its 32B base model, DINOv2 and CLIP-ConvNeXt visual encoders, and 3D positional encoding. For responsive interaction, those weights need to remain available in memory.</p>
<h2>Live test: 146 scored findings in seconds</h2>
<p>For a practical test, we loaded RADAR&#8217;s official demo case, AC423ccbe: a 368-slice contrast-enhanced abdominal CT. Uploading, viewing, and analysis all took place directly in the live portal.</p>
<table>
<thead>
<tr>
<th>Finding</th>
<th>Organ</th>
<th style="text-align: right;">Score</th>
</tr>
</thead>
<tbody>
<tr>
<td>Cardiomegaly</td>
<td>HEART</td>
<td style="text-align: right;">0.833</td>
</tr>
<tr>
<td>Calcification</td>
<td>AORTA</td>
<td style="text-align: right;">0.697</td>
</tr>
<tr>
<td>Atherosclerosis</td>
<td>AORTA</td>
<td style="text-align: right;">0.578</td>
</tr>
</tbody>
</table>
<p>Reducing the threshold to 0.30 expanded the table to eight findings, including lipoma (0.41), rib fracture (0.39), aortic dissection (0.38), pericardial effusion (0.36), and mesenteric panniculitis (0.32). Once the model is loaded, analysis completes in the <strong>10–30 second</strong> range. In this run, the table populated in seconds rather than minutes.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-4 hover-type-none"><img decoding="async" width="1280" height="577" alt="Findings table at a 0.30 threshold, showing eight findings" title="2-radar-esik-030" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030.webp" class="img-responsive wp-image-1995" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-200x90.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-300x135.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-400x180.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-600x270.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-768x346.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-800x361.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-1024x462.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030-1200x541.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/2-radar-esik-030.webp 1280w" sizes="(max-width: 640px) 100vw, 1280px" /></span></div><div class="fusion-text fusion-text-7"><p><em>The same case at a 0.30 threshold for a more sensitive review. Slice navigation and soft-tissue, lung, and bone window controls appear at lower left.</em></p>
<p>The interface brings together tools that support a faster radiology workflow: organ-level grouping, finding-name search, quick chips for the highest scores, soft-tissue/lung/bone window presets, keyboard navigation (←/→ for one slice; PgUp/PgDn for ±10), mouse-wheel zoom, and <strong>CSV export</strong> for every score.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-5 hover-type-none"><img decoding="async" width="1280" height="577" alt="Portal on first load: upload area, slice viewer, and findings panel" title="3-radar-bos" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos.webp" class="img-responsive wp-image-1996" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-200x90.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-300x135.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-400x180.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-600x270.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-768x346.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-800x361.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-1024x462.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos-1200x541.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/3-radar-bos.webp 1280w" sizes="(max-width: 640px) 100vw, 1280px" /></span></div><div class="fusion-text fusion-text-8"><p><em>Empty state: drag-and-drop upload, the viewer, and the 146-finding panel.</em></p>
<p>The image used in the upload demonstration is itself an abdominal CT collage:</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-6 hover-type-none"><img decoding="async" width="786" height="346" alt="Sample abdominal CT image uploaded for the demonstration" title="4-ornek-karin-bt" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt.webp" class="img-responsive wp-image-1997" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt-200x88.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt-300x132.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt-400x176.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt-600x264.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt-768x338.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/4-ornek-karin-bt.webp 786w" sizes="(max-width: 640px) 100vw, 786px" /></span></div><div class="fusion-text fusion-text-9"><h2>&nbsp;</h2>
<h2>Discussing a case with ClinFusion</h2>
<p>The second tab is designed for natural-language case discussion. Attach a 2D image (<code>.jpg</code>/<code>.png</code>) or a complete 3D volume (<code>.nii.gz</code>), then ask, for example, “What are the key findings here?” Attachments appear in the side panel, are marked “✓ sent,” and can be reused in follow-up questions. The conversation retains its context.</p>
<p>With a 3D volume, ClinFusion considers representative slices alongside the volume data. It is not simply an image classifier behind a chat box; it is a case-discussion assistant that takes volumetric information into account.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-7 hover-type-none"><img decoding="async" width="1280" height="577" alt="ClinFusion tab: attached CT image and a drafted question" title="5-clinfusion-arayuz" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz.webp" class="img-responsive wp-image-1998" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-200x90.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-300x135.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-400x180.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-600x270.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-768x346.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-800x361.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-1024x462.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz-1200x541.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/5-clinfusion-arayuz.webp 1280w" sizes="(max-width: 640px) 100vw, 1280px" /></span></div><div class="fusion-text fusion-text-10"><p><em>The sample image is attached and the question is ready to send.</em></p>
<p>For the sample image, we asked: <em>“Evaluate this contrast-enhanced abdominal CT image: which organs are visible and what are the key findings? Please give a short clinical assessment.”</em> The model produced the following response live on the DGX Spark:</p>
<blockquote>
<p>The CT scan shows the liver, spleen, kidneys, and portions of the gastrointestinal tract. The liver appears homogeneous with no focal lesions, and the spleen is normal in size and density. The kidneys are symmetrically positioned with no signs of hydronephrosis or masses. The gastrointestinal tract shows no significant distension or obstruction. The vertebral column and pelvic bones appear intact without fractures or lesions. Overall, the scan does not show any acute abnormalities.</p>
</blockquote>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-8 hover-type-none"><img decoding="async" width="1280" height="577" alt="Response shown in the portal: question and full model output" title="6-clinfusion-cevap" src="https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap.webp" class="img-responsive wp-image-1999" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-200x90.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-300x135.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-400x180.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-600x270.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-768x346.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-800x361.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-1024x462.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap-1200x541.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/10/6-clinfusion-cevap.webp 1280w" sizes="(max-width: 640px) 100vw, 1280px" /></span></div><div class="fusion-text fusion-text-11"><p><em>The portal renders the model response with the question and attachment context in view.</em></p>
<h2>Why DGX Spark?</h2>
<p>The system was not designed merely as a workload to place on a server. Its architecture reflects the constraints of the clinical environment.</p>
<p><strong>Data stays on site.</strong> CT volumes are patient data, and sending them to the cloud raises both compliance and trust concerns. DGX Spark can sit on a desk; the portal is available on the local network and the images remain on the machine. This also removes network latency, recurring cloud costs, and the need for an additional data-processing agreement.</p>
<p><strong>Unified memory fits the workload.</strong> The 32B model&#8217;s approximately 75 GB resident footprint, RADAR&#8217;s working set, and the operating system can coexist on one machine. The weights are loaded once and remain available, without CPU–GPU copy overhead or layer-by-layer disk offloading.</p>
<p><strong>One GPU queue coordinates two capabilities.</strong> MedPortal serializes both models through a single GPU queue. Jobs do not interfere with each other, and the status bar indicates what is waiting. The same system supports radiological pre-review and case discussion.</p>
<p><strong>The form factor belongs near the workflow.</strong> This is a quiet desktop system that can sit in a clinic, laboratory, or beside a radiology workstation—not a noisy rack in a server room. After startup with <code>./run.sh</code>, use <code>./status.sh</code> for health checks, <code>./stop.sh</code> to release memory, and <code>./scripts/smoke.sh</code> for an end-to-end check.</p>
<p><strong>AArch64 is ready in practice.</strong> The project resolves the GB10-specific issues we encountered: the aarch64 FlashAttention wheel, selective dependency installation, and conda environment paths. The installers in <code>scripts/</code> can reduce setup on a new Spark from hours to minutes. The UI uses plain HTML/JS, with no build step or CDN dependency.</p>
<h2>Architecture: MedPortal as an orchestration layer</h2>
<p>MedPortal does not store model weights or perform inference itself. It coordinates the workloads:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-1 > .CodeMirror, .fusion-syntax-highlighter-1 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-1 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_1" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_1" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_1" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">[Web UI (8080)] → [FastAPI backend] → [single GPU queue]
                     │                     │
                     ├─ RADAR: subprocess, NIfTI slicing, CSV
                     └─ ClinFusion: resident worker (8100, localhost only)
</textarea></div><div class="fusion-text fusion-text-12"><ul>
<li>Uploads are managed in one place, while 3D volumes are sliced on the backend for the viewer.</li>
<li>RADAR runs as a separate process; ClinFusion is a persistent HTTP worker that holds its model in memory.</li>
<li>Paths and ports are configurable through environment variables, so the same checkout can move to another Spark unchanged.</li>
<li>On the development side, <strong>74 tests</strong> validate the backend, and <code>smoke.sh</code> returns “146 findings + SMOKE OK” end to end.</li>
</ul>
<h2>Setup</h2>
<p>One system, three steps: Miniconda, three codebases (medportal + RADAR + ClinFusion), and two setup scripts.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-2 > .CodeMirror, .fusion-syntax-highlighter-2 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-2 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_2" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_2" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_2" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">git clone https://github.com/openzeka/medportal.git
cd medportal
./scripts/setup_radar.sh          # environment + dependencies + ~6 GB checkpoints
./scripts/setup_clinfusion.sh     # environment + aarch64 FlashAttention wheel
# after fetching the weights (~180 GB) with huggingface-cli:
./run.sh                          # waits for the worker, then opens the portal on 8080
</textarea></div><div class="fusion-text fusion-text-13"><p>The operational details are documented in the project: only dependencies used by the inference path are installed; unavailable aarch64 wheels are skipped; the first 32B-model load takes about 14 minutes and is visible in <code>logs/worker.log</code>; and the setup scripts can show their planned work with <code>--dry-run</code> first.</p>
<h2>Limits and appropriate use</h2>
<p>In medical AI, stating the limits clearly is essential:</p>
<ul>
<li><strong>Scores are not clinical diagnoses or standalone probabilities.</strong> A RADAR score such as 0.71 for <code>Liver_Cirrhosis</code> is a model output based on imaging evidence; it does not directly express the patient&#8217;s probability of cirrhosis. Clinical judgment and the radiology report remain decisive, and a low score does not rule out a finding.</li>
<li><strong>RADAR is for research use only.</strong> Its upstream CC BY-NC-SA 4.0 license does not clear it for commercial or clinical deployment. Outputs may be unreliable outside its training distribution, particularly beyond contrast-enhanced abdominal CT.</li>
<li><strong>One analysis runs at a time.</strong> Because the GPU is shared, jobs pass through a queue. That is a deliberate choice for batch-oriented workloads.</li>
<li>This system is <strong>not a cleared medical device</strong>; it is a research and decision-support demonstration.</li>
</ul>
<h2>Conclusion</h2>
<p>MedPortal demonstrates more than two models running side by side. It shows what a single desktop system can now carry: a resident 32B multimodal medical model alongside expert-level abdominal CT analysis—a combination that recently would have implied a server rack.</p>
<p>NVIDIA DGX Spark makes that capability available beside the clinical workflow, without sending data off premises and with a single-command startup. The question “What might I be missing in this abdominal CT?” no longer has to be explored in the cloud; it can be explored in the box beside the radiology desk.</p>
<p>&nbsp;</p>
</div></div></div></div></div><div class="fusion-fullwidth fullwidth-box fusion-builder-row-4 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-3 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-14"><p><span data-mce-type="bookmark" style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" class="mce_SELRES_end">﻿</span></p>
</div></div></div></div></div></p>
<p>The post <a href="https://blog.openzeka.com/en/an-abdominal-ct-expert-and-a-32b-medical-ai-in-one-dgx-spark-medportal/">An Abdominal CT Expert and a 32B Medical AI in One DGX Spark: MedPortal</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Qwen3.8-Flash-Next: What a Single DGX Spark or RTX PRO 6000 Can Do</title>
		<link>https://blog.openzeka.com/en/qwen3-8-flash-next-what-a-single-dgx-spark-or-rtx-pro-6000-can-do/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Mon, 05 Oct 2026 08:27:40 +0000</pubDate>
				<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1988</guid>

					<description><![CDATA[<p>Qwen3.8-Flash-Next is a 176B-parameter Mixture-of-Expe ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/qwen3-8-flash-next-what-a-single-dgx-spark-or-rtx-pro-6000-can-do/">Qwen3.8-Flash-Next: What a Single DGX Spark or RTX PRO 6000 Can Do</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-5 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-4 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-blend:overlay;--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-15"><p>Qwen3.8-Flash-Next is a 176B-parameter Mixture-of-Experts model with a 262K-token context window. It scores <strong>39.8 on the Artificial Analysis Intelligence Index</strong> and <strong>53.6 on the Agentic Index</strong>.* Its NVIDIA NVFP4 checkpoint is 132.7 GB. A DGX Spark has 128 GB of memory, and an RTX PRO 6000 has 96 GB, so on paper the model fits on neither.</p>
<p>In our measurements it runs on both: at <strong>28.5 tok/s on one DGX Spark</strong> and <strong>155.8 tok/s on one RTX PRO 6000</strong>. This post explains how, and what each device delivers.</p>
<h2>The trick: a table the model looks up, not computes with</h2>
<p>To generate each token, a language model runs through all its layers and multiplies by its weight matrices. Those weights, and the KV cache that remembers the conversation, are read at every step. They must sit in fast GPU memory, or everything slows down.</p>
<p>Qwen3.8-Flash-Next also carries a different kind of component: a <strong>51B-parameter lookup table</strong> of learned word combinations, known as an Engram-style table. The model does not compute with this table; it <strong>looks things up</strong> in it. For each token, it reads only the handful of rows that match the last few words: about 2.5 KB out of a 51 GB table. DeepSeek, which introduced the idea, calls this <em>conditional memory</em>: a row is read only when the input calls for it.</p>
<p>Think of it as a large dictionary on a shelf. You do not need to carry the whole book around; you open only the page you need.</p>
<p>So the table does not need to be in GPU memory. It can sit in a slower, larger place while the GPU keeps the part of the model it computes with:</p>
<ul>
<li><strong>DGX Spark:</strong> CPU and GPU share the same 128 GB, so the table goes to the <strong>local NVMe drive</strong>.</li>
<li><strong>RTX PRO 6000:</strong> the table goes to the <strong>system RAM</strong> of the workstation.</li>
</ul>
<p>The table&#8217;s values are unchanged, so the model&#8217;s answers are the same.</p>
<h2>On a single DGX Spark</h2>
<p>Speed per request and time to first token at each concurrency:</p>
<ul>
<li><strong>1 simultaneous request:</strong> 28.5 tok/s, time to first token 302 ms</li>
<li><strong>2 simultaneous requests:</strong> 23.6 tok/s, time to first token 393 ms</li>
<li><strong>4 simultaneous requests:</strong> 17.1 tok/s, time to first token 566 ms</li>
<li><strong>8 simultaneous requests:</strong> 11.8 tok/s, time to first token 763 ms</li>
</ul>
<p>A desktop device runs a model of this class at a comfortable reading speed. Up to two simultaneous requests stay above 20 tok/s, and the first token arrives in under a second at every level tested. Based on this speed and the KV cache memory left for users&#8217; conversations, our <a style="color: green;" href="https://whitepapers.openzeka.com/llm-inference-benchmarks/qwen3-8-flash-next/1x-dgx-spark/nvfp4-sglang-tp1-spec3/"><strong>LLM Inference Benchmark Explorer</strong></a> estimates this is enough for <strong>8 chat users or 3 agentic users</strong>: a good fit for a personal assistant, a small team or development work.</p>
<h2>On a single RTX PRO 6000</h2>
<p>Speed per request and time to first token at each concurrency:</p>
<ul>
<li><strong>1 simultaneous request:</strong> 155.8 tok/s, time to first token 139 ms</li>
<li><strong>4 simultaneous requests:</strong> 90.8 tok/s, time to first token 221 ms</li>
<li><strong>8 simultaneous requests:</strong> 63.7 tok/s, time to first token 252 ms</li>
<li><strong>16 simultaneous requests:</strong> 43.2 tok/s, time to first token 257 ms</li>
</ul>
<p>The workstation card runs the same model more than five times faster. Even with 16 simultaneous requests, each one gets 43.2 tok/s and its first token in about a quarter of a second. We estimate this is enough for <strong>about 20 chat users or 8 agentic users</strong> (<a style="color: green;" href="https://whitepapers.openzeka.com/llm-inference-benchmarks/qwen3-8-flash-next/rtx-pro-6000/nvfp4-sglang-tp1-spec3/"><strong>run details</strong></a>).</p>
<h2>Good to know</h2>
<ul>
<li><strong>The user counts are estimates</strong> based on measurements with short prompts of about 128 tokens.</li>
<li><strong>The RTX PRO 6000 workstation needs at least 64 GB of free system RAM</strong> for the table.</li>
</ul>
<h2>Read more</h2>
<ul>
<li><strong>The white paper:</strong> <a style="color: green;" href="https://whitepapers.openzeka.com/papers/conditional-memory-offloading/"><strong>Engram Offloading in LLM Inference</strong></a> explains the mechanism, the memory differences between DGX Spark, RTX PRO 6000 and DGX B300, and all measured results.</li>
<li><strong>The measurements:</strong> every run, with its full concurrency sweep, is in the <a style="color: green;" href="https://whitepapers.openzeka.com/llm-inference-benchmarks/"><strong>LLM Inference Benchmark Explorer</strong></a>. Adjust the targets and see how the capacity estimates change.</li>
</ul>
<p>Want to know which models your hardware can run, or which hardware your models need? Contact the OpenZeka team; we will size it with you.</p>
<p>*Intelligence Index v4.3 and Agentic Index, published by <a style="color: green;" href="https://artificialanalysis.ai"><strong>Artificial Analysis</strong></a>, retrieved 28 September 2026 and reproduced with attribution. Speeds are OpenZeka measurements made with the CordatusAI LLM Benchmark Tool (about 128 input and 128 output tokens, mean values).</p>
</div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/qwen3-8-flash-next-what-a-single-dgx-spark-or-rtx-pro-6000-can-do/">Qwen3.8-Flash-Next: What a Single DGX Spark or RTX PRO 6000 Can Do</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Qwen3.8-27B on DGX Spark — 48 tok/s</title>
		<link>https://blog.openzeka.com/en/qwen3-8-27b-on-dgx-spark-48-tok-s/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Wed, 26 Aug 2026 10:23:50 +0000</pubDate>
				<category><![CDATA[Generative AI]]></category>
		<category><![CDATA[8669]]></category>
		<category><![CDATA[8673]]></category>
		<category><![CDATA[8820]]></category>
		<category><![CDATA[9043]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1977</guid>

					<description><![CDATA[<p>Qwen3.8-27B is a 27-billion-parameter language model r ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/qwen3-8-27b-on-dgx-spark-48-tok-s/">Qwen3.8-27B on DGX Spark — 48 tok/s</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-6 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-5 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-blend:overlay;--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-16"><p>Qwen3.8-27B is a 27-billion-parameter language model released by Alibaba in August 2026. With a score of 52 on the Artificial Analysis Intelligence Index, it ranks first in its class among open-weight models under 40B parameters.</p>
<p>In this tutorial, you will run Qwen3.8-27B-NVFP4 on a single DGX Spark using the SGLang inference engine and the DFlash2 draft model for speculative decoding, reaching a generation speed of 48 tokens per second. SGLang loads the model&#8217;s trained weights into GPU memory and exposes an API that accepts external requests. sparkrun manages the Docker container and automates distribution of the model files.</p>
<p>The Spark can be used directly as a computer by connecting a monitor and keyboard, or it can operate as a server accessed remotely from another computer. In this tutorial, we will connect to the Spark remotely, set up the required software, and launch the model.</p>
<p>This tutorial is divided into four sections:</p>
<ul>
<li><strong>Setup:</strong> Pulling the Docker image and preparing the recipe</li>
<li><strong>Running:</strong> Starting, monitoring, and testing the model with sparkrun</li>
<li><strong>Benchmark:</strong> Performance measurements at different concurrency levels</li>
<li><strong>Shutdown:</strong> Stopping the services</li>
</ul>
<h2>Setup</h2>
<h3>1. Connecting to the Spark</h3>
<p>If this is your first time connecting to the Spark remotely, you first need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command in a terminal:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-3 > .CodeMirror, .fusion-syntax-highlighter-3 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-3 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_3" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_3" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_3" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ip route get 1.1.1.1 | grep -oP 'src \K\S+'</textarea></div><div class="fusion-text fusion-text-17"><p>&nbsp;</p>
<p>The command returns the IP address of the Spark&#8217;s default network interface:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-4 > .CodeMirror, .fusion-syntax-highlighter-4 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-4 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_4" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_4" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_4" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">192.168.1.148</textarea></div><div class="fusion-text fusion-text-18"><p>&nbsp;</p>
<p>Make a note of this address; throughout the tutorial, you will use it in place of <code></code>. Alternatively, you can find the IP address through the NVIDIA Sync application.</p>
<p>Make sure your computer is connected to the same network as the Spark. Then open a terminal on your computer and connect to the Spark over SSH:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-5 > .CodeMirror, .fusion-syntax-highlighter-5 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-5 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_5" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_5" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_5" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ssh nvidia@<spark-ip></textarea></div><div class="fusion-text fusion-text-19"><p>&nbsp;</p>
<p><strong><a style="color: #00bc5e;" href="https://blog.openzeka.com/en/local-gpt-oss-120b-serving-on-dgx-spark-with-sparkrun/">Previous tutorials</a> </strong>covered the sparkrun installation process step by step. This tutorial assumes that sparkrun is already installed.</p>
<h3>2. Pulling the Docker Image</h3>
<p>Qwen3.8-27B&#8217;s hybrid architecture and DFlash2 speculative decoding feature requires a specialized SGLang image. This image is built from the SGLang mainline codebase and includes the required DFlash2 kernel patches.</p>
<p>Run the following command to pull the Docker image prepared by OpenZeka:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-6 > .CodeMirror, .fusion-syntax-highlighter-6 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-6 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_6" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_6" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_6" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker pull registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2:latest</textarea></div><div class="fusion-text fusion-text-20"><p>&nbsp;</p>
<p>Verify the image:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-7 > .CodeMirror, .fusion-syntax-highlighter-7 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-7 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_7" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_7" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_7" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker images registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2:latest</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-8 > .CodeMirror, .fusion-syntax-highlighter-8 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-8 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_8" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_8" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_8" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">REPOSITORY                                            TAG       IMAGE ID        CREATED          SIZE
registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2      latest    c14312a66    2 minutes ago    57.3GB</textarea></div><div class="fusion-text fusion-text-21"><h3>3. Preparing the Recipe</h3>
<p>A recipe is a YAML file that defines how sparkrun should launch the model. The model, Docker image, SGLang flags, and memory settings are collected in a single file. For Qwen3.8-27B, this recipe runs the model on a single Spark with NVFP4 quantization, accelerates inference with DFlash2 speculative decoding using SGLang&#8217;s <code>--speculative-num-draft-tokens 8</code> configuration, and includes Mamba memory parameters specific to Qwen3.8&#8217;s hybrid architecture.</p>
<p>Save the recipe file using the following command:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-9 > .CodeMirror, .fusion-syntax-highlighter-9 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-9 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_9" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_9" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_9" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cat > ~/qwen38-27b-dflash2.yaml << 'RECIPE'
# Qwen3.8-27B NVFP4 + DFlash2
# Usage:
#   sparkrun run ~/qwen38-27b-dflash2.yaml --rootful --no-follow --no-rm
recipe_version: "2"
model: RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead
runtime: sglang
container: registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2:latest

max_nodes: 1

metadata:
  model_dtype: nvfp4
  kv_dtype: fp8
  description: Qwen3.8-27B NVFP4 (BF16-LMHead) + DFlash2 — Ultimate single-Spark

defaults:
  port: 8000
  host: 0.0.0.0
  tensor_parallel: 1
  gpu_memory_utilization: 0.50
  served_model_name: qwen3.8-27b
  attention_backend: flashinfer
  speculative_algorithm: DFLASH
  speculative_draft_model_path: z-lab/Qwen3.8-27B-DFlash2
  speculative_num_draft_tokens: 8

env:
  HF_HUB_OFFLINE: "0"
  TORCHINDUCTOR_CACHE_DIR: /cache/inductor
  HF_HOME: /cache/huggingface
  HF_HUB_CACHE: /cache/huggingface/hub

command: |
  python3 -m sglang.launch_server \
    --trust-remote-code \
    --model-path {model} \
    --tp-size {tensor_parallel} \
    --served-model-name {served_model_name} \
    --mem-fraction-static {gpu_memory_utilization} \
    --attention-backend {attention_backend} \
    --chunked-prefill-size 8192 \
    --disable-prefill-cuda-graph \
    --kv-cache-dtype fp8_e4m3 \
    --mamba-ssm-dtype bfloat16 \
    --mamba-full-memory-ratio 4.21 \
    --mamba-radix-cache-strategy extra_buffer \
    --max-mamba-cache-size 64 \
    --max-running-requests 16 \
    --context-length 262144 \
    --speculative-algorithm {speculative_algorithm} \
    --speculative-draft-model-path {speculative_draft_model_path} \
    --speculative-draft-model-revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 \
    --speculative-num-draft-tokens {speculative_num_draft_tokens} \
    --reasoning-parser qwen3 \
    --tool-call-parser qwen3_coder \
    --sampling-defaults model \
    --enable-metrics \
    --enable-cache-report \
    --stream-interval 1 \
    --host {host} \
    --port {port}
RECIPE
</textarea></div><div class="fusion-text fusion-text-22"><p>&nbsp;</p>
<p>Verify the file:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-10 > .CodeMirror, .fusion-syntax-highlighter-10 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-10 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_10" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_10" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_10" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cat ~/qwen38-27b-dflash2.yaml</textarea></div><div class="fusion-text fusion-text-23"><p>&nbsp;</p>
<p>You should see the recipe file you just saved.</p>
<h2>Running</h2>
<h3>4. Pre-Launch Checks</h3>
<p>Verify that sparkrun parses the recipe correctly and that the memory budget is appropriate:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-11 > .CodeMirror, .fusion-syntax-highlighter-11 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-11 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_11" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_11" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_11" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun show ~/qwen38-27b-dflash2.yaml
</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-12 > .CodeMirror, .fusion-syntax-highlighter-12 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-12 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_12" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_12" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_12" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">Name:         /home/nvidia/qwen38-27b-dflash2.yaml
Description:  Qwen3.8-27B NVFP4 (packed FP4) + DFlash2 — Ultimate single-Spark
Runtime:      sglang
Model:        RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead
Container:    registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2:latest
Nodes:        1

.....
.....
.....

VRAM Estimation:
  Model dtype:      nvfp4
  KV cache dtype:   fp8
  Architecture:     64 layers, 4 KV heads, 256 head_dim
  Model weights:    21.9 GB
  Tensor parallel:  1
  Per-GPU total:    21.9 GB
  DGX Spark fit:    YES

  GPU Memory Budget:
    gpu_memory_utilization: 50%
    Usable GPU memory:     60.5 GB (121 GB x 50%)
    Available for KV:      38.6 GB
    Max context tokens:    478,483
</textarea></div><div class="fusion-text fusion-text-24"><p>&nbsp;</p>
<p>sparkrun has parsed the recipe correctly and confirms that the configuration fits on the DGX Spark with <code>DGX Spark fit: YES</code>.</p>
<h3>5. Starting the Model</h3>
<p>Now start the model:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-13 > .CodeMirror, .fusion-syntax-highlighter-13 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-13 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_13" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_13" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_13" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun run ~/qwen38-27b-dflash2.yaml --rootful --no-follow --no-rm
</textarea></div><div class="fusion-text fusion-text-25"><p>&nbsp;</p>
<p>sparkrun automatically synchronizes the image and models to the Spark, skipping any resources that are already present, and then starts the container.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-14 > .CodeMirror, .fusion-syntax-highlighter-14 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-14 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_14" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_14" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_14" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun v0.3.4

Runtime:   sglang
Image:     registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2:latest
Model:     RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead
Mode:      solo
Platform:  DGX Spark (NVIDIA GB10, NCCL)
Scheduler: occupancy-sparse

VRAM Estimation:
  Model dtype:      nvfp4
  KV cache dtype:   fp8
  Architecture:     64 layers, 4 KV heads, 256 head_dim
  Model weights:    21.9 GB
  Tensor parallel:  1
  Per-GPU total:    21.9 GB
  DGX Spark fit:    YES

  GPU Memory Budget:
    gpu_memory_utilization: 50%
    Usable GPU memory:     60.5 GB (121 GB x 50%)
    Available for KV:      38.6 GB
    Max context tokens:    478,483

Hosts:     default cluster 'default'
  Target:  <spark-ip>

[1/6] Preparing
  done (0.0s)
[2/6] Building — skipped (no builder)
[3/6] Distributing resources
  Distributing image registry.cordata.ai/spark-cluster/sglang-qwen38-27b-dflash2:latest to 1 host(s)
  Distributing model RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead to 1 host(s)
  Distributing model z-lab/Qwen3.8-27B-DFlash2 to 1 host(s)
  done (2.5s)
[4/6] Syncing tuning configs
  done (0.0s)
[5/6] Launching sglang runtime
  Step 1/3: Detecting InfiniBand
  Step 2/3: Launching container
  Step 3/3: Executing serve command
  done (7.5s)
[6/6] Post-launch hooks — skipped
</textarea></div><div class="fusion-text fusion-text-26"><p>&nbsp;</p>
<p>sparkrun successfully completed all six steps and reports <code>Mode: solo</code>. It synchronized the image and both models—the target model and the draft model—to the Spark.</p>
<p>Downloading the model files and bringing SGLang to a ready state may take several minutes. During this process, SGLang loads the model weights into GPU memory, initializes the DFlash2 draft model, and captures CUDA graphs.</p>
<h3>6. Monitoring the Logs</h3>
<p>To monitor the SGLang logs, run:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-15 > .CodeMirror, .fusion-syntax-highlighter-15 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-15 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_15" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_15" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_15" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun logs ~/qwen38-27b-dflash2.yaml
</textarea></div><div class="fusion-text fusion-text-27"><p>&nbsp;</p>
<p>You should see output similar to the following:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-16 > .CodeMirror, .fusion-syntax-highlighter-16 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-16 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_16" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_16" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_16" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">[2026-08-25 10:43:59] DFLASH only supports speculative_num_steps == 1; overriding speculative_num_steps=3 to 1.
[2026-08-25 10:44:17] Load weight begin. avail mem=113.57 GB
[2026-08-25 10:49:26] Load weight end. elapsed=308.46 s, type=Qwen3_5ForConditionalGeneration, quant=modelopt_mixed, quant_algo=MIXED_PRECISION, avail mem=91.44 GB, mem usage=22.14 GB.
[2026-08-25 10:49:31] Load weight begin. avail mem=91.23 GB
[2026-08-25 10:49:59] Load weight end. elapsed=28.12 s, type=DFlash2DraftModel, avail mem=87.82 GB, mem usage=3.41 GB.
[2026-08-25 10:50:00] Initialized DFLASH draft runner. attention_backend=flashinfer, model=DFlash2DraftModel, block_size=8, draft_window_size=None, compact_cache=False
[2026-08-25 10:50:00] DFLASH draft runner ready. mask_token=<|MASK|>, mask_token_id=248070, mask_token_id_override=248070, noise_embed_scale=1.0
[2026-08-25 10:50:00] DFLASH fused KV materialization enabled. n_layers=5, num_kv_heads=8, head_dim=128
[2026-08-25 10:50:00] max_running_requests is capped to 12 by the mamba state cache (max_mamba_cache_size=64, 5 state slots per request). To raise it: increase --mamba-full-memory-ratio or --max-mamba-cache-size, or halve the state size with --mamba-ssm-dtype bfloat16.
[2026-08-25 10:50:00] Mamba Cache is allocated. max_mamba_cache_size: 64, conv_state size: 0.18GB, ssm_state size: 4.57GB intermediate_ssm_state_cache size: 7.31GB intermediate_conv_window_cache size: 0.12GB
[2026-08-25 10:50:15] Capture target verify CUDA graph begin. backend=full, num_tokens_per_req=8, bs=[1, 2, 3, 4, 5, 6, 7, 8, 10, 12], avail mem=54.20 GB
[2026-08-25 10:50:18] Capture target verify CUDA graph end. elapsed=3.53 s, mem usage=0.71 GB, avail mem=53.49 GB.
[2026-08-25 10:50:18] DFLASH draft greedy head kept eager (reason=quantized lm_head).
[2026-08-25 10:50:18] Capture draft verify CUDA graph begin. backend=full, num_tokens_per_req=8, bs=[1, 2, 3, 4, 5, 6, 7, 8, 10, 12], avail mem=53.49 GB
[2026-08-25 10:50:22] Capture draft verify CUDA graph end. elapsed=3.36 s, mem usage=0.74 GB, avail mem=52.74 GB.
[2026-08-25 10:50:22] max_total_num_tokens=460797, chunked_prefill_size=8192, max_prefill_tokens=16384, max_running_requests=12, context_len=262144, available_gpu_mem=52.74 GB
[2026-08-25 10:50:22] INFO:     Application startup complete.
[2026-08-25 10:50:25] The server is fired up and ready to roll!
</textarea></div><div class="fusion-text fusion-text-28"><p>&nbsp;</p>
<p>Once you see <code>Application startup complete.</code> and <code>The server is fired up and ready to roll!</code>, the model server is ready.</p>
<h3>7. Health Check</h3>
<p>Verify that the server is running:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-17 > .CodeMirror, .fusion-syntax-highlighter-17 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-17 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_17" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_17" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_17" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s -o /dev/null -w "HTTP %{http_code}" http://<spark-ip>:8000/health
</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-18 > .CodeMirror, .fusion-syntax-highlighter-18 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-18 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_18" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_18" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_18" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">HTTP 200
</textarea></div><div class="fusion-text fusion-text-29"><p>&nbsp;</p>
<p>An <code>HTTP 200</code> response indicates that the server is healthy.</p>
<p>Qwen3.8-27B is now running and being served from port 8000 on the Spark.</p>
<h2>Benchmark</h2>
<p>We benchmarked Qwen3.8-27B running on a single Spark at different concurrency levels. The measurements record average TTFT (Time to First Token—the latency until the first output token is returned) and average TPS (Tokens Per Second—the token generation rate).</p>
<ul>
<li><strong>Concurrency 1</strong> &#8212; Avg. TTFT: 232 ms, Avg. TPS: 47.9 tok/s</li>
<li><strong>Concurrency 2</strong> &#8212; Avg. TTFT: 320 ms, Avg. TPS: 39.8 tok/s</li>
<li><strong>Concurrency 4</strong> &#8212; Avg. TTFT: 343 ms, Avg. TPS: 35.6 tok/s</li>
<li><strong>Concurrency 8</strong> &#8212; Avg. TTFT: 389 ms, Avg. TPS: 27.6 tok/s</li>
</ul>
<p>At concurrency 1, the model generates an average of 47.9 tokens per second. Even at concurrency 8, it remains above 25 tokens per second while keeping TTFT below 400 ms, making the configuration suitable for workloads with high request concurrency.</p>
<p>The measurements were collected using the <a href="https://github.com/CordatusAI/llm-benchmark">CordatusAI LLM Benchmark Tool</a>: 128 input tokens, 128 output tokens, averaged over 10 rounds per concurrency level, with the validity criterion <code>TTFT &lt; 1000 ms AND TPS &gt;= 15 tok/s</code>. The tool, developed by CordatusAI, is a benchmarking application for testing LLM servers that expose OpenAI-compatible APIs.</p>
<h2>Shutdown</h2>
<p>When you are finished, stop the model:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-19 > .CodeMirror, .fusion-syntax-highlighter-19 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-19 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_19" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_19" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_19" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun stop ~/qwen38-27b-dflash2.yaml
</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-20 > .CodeMirror, .fusion-syntax-highlighter-20 > .CodeMirror .CodeMirror-gutters {background-color:#2b2b2b;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-20 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_20" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_20" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_20" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">Workload stopped on 1 host(s).
</textarea></div><div class="fusion-text fusion-text-30"><p>&nbsp;</p>
<p>This command stops the container and releases the memory it was using. The container, Docker image, and model files remain on disk, so you do not need to download them again the next time you start the service. Simply run the <code>sparkrun run</code> command from Step 5 again.</p>
</div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/qwen3-8-27b-on-dgx-spark-48-tok-s/">Qwen3.8-27B on DGX Spark — 48 tok/s</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>NVIDIA DGX Spark Ecosystem</title>
		<link>https://blog.openzeka.com/en/nvidia-dgx-spark-ecosystem/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Mon, 24 Aug 2026 12:34:27 +0000</pubDate>
				<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1973</guid>

					<description><![CDATA[<p>Resources, Tools, and Community  NVIDIA DGX Spark™ is  ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/nvidia-dgx-spark-ecosystem/">NVIDIA DGX Spark Ecosystem</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-7 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-6 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-31"><h2>Resources, Tools, and Community</h2>
</div><div class="fusion-text fusion-text-32"><p><strong>NVIDIA DGX Spark™</strong> is a compact desktop AI system designed for working with artificial intelligence models. It brings development, testing, inference, fine-tuning, and prototyping workflows together on a single platform.</p>
</div>    <div class="ext-wc-slider-wrapper">
        <div class="ext-wc-slider-header">
            <h4 class="ext-wc-slider-heading">Recommended Products</h4>
            <a href="https://openzeka.com/en/store/?orderby=menu_order" target="_blank" rel="noopener" class="ext-wc-all-link">
                See All Products &rarr; 
                <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
                    <circle cx="9" cy="21" r="1"></circle>
                    <circle cx="20" cy="21" r="1"></circle>
                    <path d="M1 1h4l2.68 13.39a2 2 0 0 0 2 1.61h9.72a2 2 0 0 0 2-1.61L23 6H6"></path>
                </svg>
            </a>
        </div>
        
        <div class="ext-wc-slider-rel-container">
            <button class="ext-wc-arrow ext-wc-arrow-prev" aria-label="Önceki">&#10094;</button>
            
            <div class="ext-wc-slider-container">
                                    <div class="ext-wc-slider-item">
                        <a href="https://openzeka.com/en/product/nvidia-dgx-spark-bundle/" target="_blank" rel="nofollow noopener">
                            <div class="ext-wc-slider-content">
                                                                    <div class="ext-wc-slider-img">
                                        <img decoding="async" src="https://openzeka.com/en/wp-content/uploads/2026/05/DGX-Spark-img-6.png" alt="NVIDIA DGX Spark Bundle">
                                    </div>
                                
                                <div class="ext-wc-slider-info">
                                                                            <div class="ext-wc-slider-category">DGX Systems</div>
                                    
                                    <div class="ext-wc-slider-title">NVIDIA DGX Spark Bundle</div>

                                    <div class="ext-wc-slider-btn">İncele</div>
                                </div>
                            </div>
                        </a>
                    </div>
                                    <div class="ext-wc-slider-item">
                        <a href="https://openzeka.com/en/product/nvidia-dgx-spark-triple/" target="_blank" rel="nofollow noopener">
                            <div class="ext-wc-slider-content">
                                                                    <div class="ext-wc-slider-img">
                                        <img decoding="async" src="https://openzeka.com/en/wp-content/uploads/2026/06/DGX_Spark_Triple_1.png" alt="NVIDIA DGX Spark Triple">
                                    </div>
                                
                                <div class="ext-wc-slider-info">
                                                                            <div class="ext-wc-slider-category">DGX Systems</div>
                                    
                                    <div class="ext-wc-slider-title">NVIDIA DGX Spark Triple</div>

                                    <div class="ext-wc-slider-btn">İncele</div>
                                </div>
                            </div>
                        </a>
                    </div>
                                    <div class="ext-wc-slider-item">
                        <a href="https://openzeka.com/en/product/nvidia-dgx-spark-quad-ai-cluster-4-node-512-gb-200gbe/" target="_blank" rel="nofollow noopener">
                            <div class="ext-wc-slider-content">
                                                                    <div class="ext-wc-slider-img">
                                        <img decoding="async" src="https://openzeka.com/en/wp-content/uploads/2026/07/Dgx-spark-en-1.png" alt="NVIDIA DGX Spark Quad AI Cluster – 4 Node, 512 GB, 200GbE">
                                    </div>
                                
                                <div class="ext-wc-slider-info">
                                                                            <div class="ext-wc-slider-category">DGX Systems</div>
                                    
                                    <div class="ext-wc-slider-title">NVIDIA DGX Spark Quad AI Cluster – 4 Node, 512 GB, 200GbE</div>

                                    <div class="ext-wc-slider-btn">İncele</div>
                                </div>
                            </div>
                        </a>
                    </div>
                                    <div class="ext-wc-slider-item">
                        <a href="https://openzeka.com/en/product/nvidia-dgx-spark-8-node-ai-cluster-8-node-1tb-200gbe/" target="_blank" rel="nofollow noopener">
                            <div class="ext-wc-slider-content">
                                                                    <div class="ext-wc-slider-img">
                                        <img decoding="async" src="https://openzeka.com/en/wp-content/uploads/2026/08/8spark.png" alt="NVIDIA DGX Spark 8 Node AI Cluster – 8 Node, 1TB, 200GbE">
                                    </div>
                                
                                <div class="ext-wc-slider-info">
                                                                            <div class="ext-wc-slider-category">DGX Systems</div>
                                    
                                    <div class="ext-wc-slider-title">NVIDIA DGX Spark 8 Node AI Cluster – 8 Node, 1TB, 200GbE</div>

                                    <div class="ext-wc-slider-btn">İncele</div>
                                </div>
                            </div>
                        </a>
                    </div>
                            </div>

            <button class="ext-wc-arrow ext-wc-arrow-next" aria-label="Sonraki">&#10095;</button>
        </div>
    </div>

    <style>
        .ext-wc-slider-wrapper {
            margin: 30px 0 !important;
            width: 100% !important;
            max-width: 100% !important;
            box-sizing: border-box !important;
            clear: both !important;
            display: block !important;
            font-family: inherit !important;
        }

        .ext-wc-slider-header {
            display: flex !important;
            justify-content: space-between !important;
            align-items: center !important;
            border-bottom: 2px solid #005885 !important;
            padding-bottom: 10px !important;
            margin-bottom: 14px !important;
        }

        .ext-wc-slider-heading {
            font-size: 20px !important;
            font-weight: 800 !important;
            margin: 0 !important;
            padding: 0 !important;
            color: #1a1a1a !important; 
            text-transform: uppercase !important;
            letter-spacing: 0.5px !important;
            border: none !important;
        }

        .ext-wc-all-link {
            font-size: 13px !important;
            font-weight: 600 !important;
            color: #005885 !important;
            text-decoration: none !important;
            display: inline-flex !important;
            align-items: center !important;
            gap: 6px !important;
            transition: opacity .2s ease !important;
        }

        .ext-wc-all-link:hover {
            opacity: 0.8 !important;
            text-decoration: underline !important;
        }

        /* OK BUTONLARI VE KAPSAYICI */
        .ext-wc-slider-rel-container {
            position: relative !important;
            width: 100% !important;
            padding: 0 16px !important;
            box-sizing: border-box !important;
        }

        .ext-wc-arrow {
            position: absolute !important;
            top: 50% !important;
            transform: translateY(-50%) !important;
            width: 36px !important;
            height: 36px !important;
            background: #ffffff !important;
            border: 1px solid #005885 !important;
            color: #005885 !important;
            border-radius: 50% !important;
            display: flex !important;
            align-items: center !important;
            justify-content: center !important;
            cursor: pointer !important;
            z-index: 10 !important;
            box-shadow: 0 2px 8px rgba(0,0,0,0.15) !important;
            transition: all .2s ease !important;
            font-size: 14px !important;
            padding: 0 !important;
            line-height: 1 !important;
        }

        /* OKLARI KESİN GİZLEME SINIFI */
        .ext-wc-arrow.ext-wc-arrow-hidden {
            display: none !important;
        }

        .ext-wc-arrow:hover {
            background: #005885 !important;
            color: #ffffff !important;
        }

        .ext-wc-arrow-prev {
            left: -4px !important;
        }

        .ext-wc-arrow-next {
            right: -4px !important;
        }

        /* YATAY KAYDIRMA KUTUSU */
        .ext-wc-slider-container {
            display: flex !important;
            flex-direction: row !important;
            flex-wrap: nowrap !important;
            justify-content: flex-start !important;
            align-items: stretch !important;
            gap: 16px !important;
            overflow-x: auto !important;
            overflow-y: hidden !important;
            scroll-snap-type: x mandatory !important;
            scroll-behavior: smooth !important;
            -webkit-overflow-scrolling: touch !important;
            padding: 6px 4px 12px 4px !important;
            width: 100% !important;
            box-sizing: border-box !important;
            
            scrollbar-width: none !important;
            -ms-overflow-style: none !important;
        }

        .ext-wc-slider-container::-webkit-scrollbar {
            display: none !important;
        }

        .ext-wc-slider-item {
            flex: 0 0 360px !important;
            min-width: 360px !important;
            max-width: 360px !important;
            scroll-snap-align: start !important;
            border-radius: 6px !important;
            border: 1px solid #e2e8f0 !important;
            background: #ffffff !important;
            box-shadow: 0 1px 3px rgba(0,0,0,0.05) !important;
            transition: all .2s ease !important;
            box-sizing: border-box !important;
            margin: 0 !important;
        }

        .ext-wc-slider-item:hover {
            box-shadow: 0 4px 12px rgba(0,0,0,0.1) !important;
            border-color: #cbd5e1 !important;
        }

        .ext-wc-slider-item a {
            text-decoration: none !important;
            color: inherit !important;
            display: block !important;
            height: 100% !important;
            padding: 14px !important;
            box-sizing: border-box !important;
        }

        .ext-wc-slider-content {
            display: flex !important;
            flex-direction: row !important;
            align-items: center !important;
            gap: 14px !important;
            height: 100% !important;
        }

        .ext-wc-slider-img {
            width: 100px !important;
            height: 90px !important;
            flex-shrink: 0 !important;
            display: flex !important;
            align-items: center !important;
            justify-content: center !important;
            background: #f8fafc !important;
            border-radius: 4px !important;
            padding: 4px !important;
        }

        .ext-wc-slider-img img {
            max-width: 100% !important;
            max-height: 100% !important;
            width: auto !important;
            height: auto !important;
            object-fit: contain !important;
            display: block !important;
        }

        .ext-wc-slider-info {
            display: flex !important;
            flex-direction: column !important;
            justify-content: center !important;
            align-items: flex-start !important;
            flex-grow: 1 !important;
            min-width: 0 !important;
        }

        .ext-wc-slider-category {
            font-size: 11px !important;
            font-weight: 700 !important;
            color: #005885 !important;
            text-transform: uppercase !important;
            letter-spacing: .6px !important;
            margin-bottom: 4px !important;
        }

        .ext-wc-slider-title {
            font-size: 13px !important;
            font-weight: 700 !important;
            color: #2d3748 !important;
            line-height: 1.35 !important;
            margin-bottom: 10px !important;
            display: -webkit-box !important;
            -webkit-line-clamp: 2 !important;
            -webkit-box-orient: vertical !important;
            overflow: hidden !important;
            text-overflow: ellipsis !important;
        }

        .ext-wc-slider-btn {
            display: inline-block !important;
            padding: 4px 18px !important;
            border: 1.5px solid #005885 !important;
            color: #005885 !important;
            background: transparent !important;
            font-size: 12px !important;
            font-weight: 600 !important;
            border-radius: 2px !important;
            transition: all .2s ease !important;
        }

        .ext-wc-slider-item:hover .ext-wc-slider-btn {
            background: #005885 !important;
            color: #ffffff !important;
        }

        @media (max-width: 768px) {
            .ext-wc-slider-heading { font-size: 15px !important; }
            .ext-wc-all-link { font-size: 11px !important; }
            .ext-wc-slider-rel-container { padding: 0 10px !important; }
            .ext-wc-slider-item {
                flex: 0 0 280px !important;
                min-width: 280px !important;
                max-width: 280px !important;
            }
            .ext-wc-slider-img {
                width: 80px !important;
                height: 75px !important;
            }
            .ext-wc-slider-title {
                font-size: 12px !important;
                margin-bottom: 6px !important;
            }
            .ext-wc-slider-btn {
                padding: 3px 12px !important;
                font-size: 11px !important;
            }
            .ext-wc-arrow-prev { left: -6px !important; }
            .ext-wc-arrow-next { right: -6px !important; }
        }
    </style>

    <script>
    document.addEventListener('DOMContentLoaded', function () {
        const wrappers = document.querySelectorAll('.ext-wc-slider-rel-container');

        wrappers.forEach(wrapper => {
            const container = wrapper.querySelector('.ext-wc-slider-container');
            const prevBtn = wrapper.querySelector('.ext-wc-arrow-prev');
            const nextBtn = wrapper.querySelector('.ext-wc-arrow-next');

            if (!container || !prevBtn || !nextBtn) return;

            const itemsCount = container.querySelectorAll('.ext-wc-slider-item').length;

            // Ekran genişliğine göre ürün sayısına bak: Masaüstünde >= 3, Mobilde >= 2 ise göster
            function checkArrowVisibility() {
                const isMobile = window.innerWidth <= 768;
                const minRequired = isMobile ? 2 : 3;

                if (itemsCount < minRequired) {
                    prevBtn.classList.add('ext-wc-arrow-hidden');
                    nextBtn.classList.add('ext-wc-arrow-hidden');
                } else {
                    prevBtn.classList.remove('ext-wc-arrow-hidden');
                    nextBtn.classList.remove('ext-wc-arrow-hidden');
                }
            }

            checkArrowVisibility();
            window.addEventListener('resize', checkArrowVisibility);

            function getScrollAmount() {
                return window.innerWidth <= 768 ? 296 : 376;
            }

            prevBtn.addEventListener('click', function () {
                container.scrollBy({ left: -getScrollAmount(), behavior: 'smooth' });
            });

            nextBtn.addEventListener('click', function () {
                container.scrollBy({ left: getScrollAmount(), behavior: 'smooth' });
            });
        });
    });
    </script>

    <div class="fusion-text fusion-text-33"><p>The software and knowledge ecosystem surrounding the device is extensive. Models, Docker images, NVIDIA playbooks, and forums are only a few of its components. DGX Spark has also developed a broad developer community. Community-built images, multi-device recipes, tools, software improvements, and performance rankings have expanded the ecosystem well beyond NVIDIA&#8217;s official resources.</p>
</div><div class="fusion-text fusion-text-34"><h2>Models</h2>
</div><div class="fusion-text fusion-text-35"><p>Models intended to run on DGX Spark are hosted on <a style="color: #18cc48;" href="https://huggingface.co/"><strong>Hugging Face</strong></a>, and many are available in multiple quantization formats. A single model may have FP8, NVFP4, MXFP4, or INT4 variants on Hugging Face. DGX Spark provides hardware support for all of these formats, allowing compatible models to be run after download without requiring an additional conversion step.</p>
</div><div class="fusion-text fusion-text-36"><p>Among these quantization formats, NVFP4 is an FP4 format specifically designed for the Blackwell architecture used by DGX Spark. By representing model weights at a lower bit width, it substantially reduces memory requirements. For example, a 200-billion-parameter model that would require approximately 400 GB of memory in a standard 16-bit format can be reduced to roughly one-quarter of that size with NVFP4. This makes it possible to run significantly larger models within the system&#8217;s 128 GB of unified memory.</p>
</div><div class="fusion-text fusion-text-37"><p>Hugging Face hosts a <a style="color: #18cc48;" href="https://huggingface.co/nvidia"><strong>collection of nearly 100 NVFP4 models converted by NVIDIA</strong></a>. The community has also published a wide range of models specifically prepared for Spark.</p>
</div><div class="fusion-text fusion-text-38"><h2>Inference Engines</h2>
</div><div class="fusion-text fusion-text-39"><p>Inference engines are software systems that load model weights into GPU memory and expose APIs for accepting external requests. These APIs can then be used by Open WebUI, Odysseus, or any other compatible client. Common inference engines include <a style="color: #18cc48;" href="https://docs.vllm.ai/"><strong>vLLM</strong></a>, <a style="color: #18cc48;" href="https://docs.sglang.ai/"><strong>SGLang</strong></a>, <a style="color: #18cc48;" href="https://nvidia.github.io/TensorRT-LLM/"><strong>TensorRT-LLM</strong></a>, and <a style="color: #18cc48;" href="https://github.com/ggml-org/llama.cpp"><strong>llama.cpp</strong></a>.</p>
</div><div class="fusion-text fusion-text-40"><p>The most widely used inference engine on DGX Spark is vLLM. One of the main reasons is its strong support for distributed execution. In particular, its robust multi-node tensor parallelism capabilities make it well suited to multi-Spark deployments. Other important features include PagedAttention for efficient GPU memory utilization, prefix caching for reusing computations across repeated prompts, and support for lower-precision KV cache formats such as FP8 and NVFP4.</p>
</div><div class="fusion-text fusion-text-41"><p>vLLM is commonly run inside Docker containers. Docker images are used to package the runtime together with its GPU dependencies. Official vLLM images are published on Docker Hub under <a style="color: #18cc48;" href="https://hub.docker.com/r/vllm/vllm-openai"><strong><code>vllm/vllm-openai</code></strong></a>.</p>
</div><div class="fusion-text fusion-text-42"><h2>Community</h2>
</div><div class="fusion-text fusion-text-43"><p>The GB10 chip used in DGX Spark has an SM 12.1 compute capability, which differs from that of conventional NVIDIA GPUs. Official Docker images for inference engines such as vLLM may not include native kernels for this architecture. In other words, models may still run with official vLLM images, but some workloads may not fully utilize the performance capabilities of the Spark GPU. For this reason, specialized images and deployment recipes are sometimes required.</p>
</div><div class="fusion-text fusion-text-44"><h3>Developers</h3>
</div><div class="fusion-text fusion-text-45"><p>Community developers build and publish Docker images containing native SM 12.1a kernels for GB10. As a result, community-maintained images are frequently used alongside official vLLM images when working with DGX Spark.</p>
</div><div class="fusion-text fusion-text-46"><p>Some of the most prominent developers and their contributions to the Spark ecosystem are summarized below. New contributors continue to appear regularly:</p>
</div><div class="fusion-text fusion-text-47"><ul>
<li><strong>eugr</strong>: <a style="color: #18cc48;" href="https://github.com/eugr/spark-vllm-docker"><strong>spark-vllm-docker</strong></a> — General-purpose inference images, MXFP4 builds, and the B12X kernel family</li>
<li><strong>christopherowen</strong>: <a style="color: #18cc48;" href="https://github.com/christopherowen/vllm"><strong>vllm (fork)</strong></a> — Source code for CUTLASS and FlashInfer kernels</li>
<li><strong>local-inference-lab</strong>: <a style="color: #18cc48;" href="https://github.com/local-inference-lab/vllm"><strong>vllm (fork)</strong></a> — Source of the B12X kernel family and experimental high-performance kernels for the sm12x architecture</li>
<li><strong>tonyd2wild</strong>: <a style="color: #18cc48;" href="https://github.com/tonyd2wild"><strong>tonyd2wild</strong></a> — Model-specific images, multi-Spark deployment recipes for DeepSeek V4 Flash, GLM-5.2, MiMo V2.5, and MiniMax-M3, as well as DSpark speculative decoding patches</li>
<li><strong>mpfaffenberger</strong>: <a style="color: #18cc48;" href="https://github.com/mpfaffenberger"><strong>mpfaffenberger</strong></a> — Runtime modifications addressing model compatibility issues</li>
<li><strong>MiaAI-Lab</strong>: <a style="color: #18cc48;" href="https://github.com/MiaAI-Lab"><strong>MiaAI-Lab</strong></a> — Community deployment recipes for DeepSeek V4 Flash, GLM-5.2, and Laguna S, as well as the sparkDash monitoring tool</li>
</ul>
</div><div class="fusion-text fusion-text-48"><p>In addition, <a style="color: #18cc48;" href="https://spark-arena.com/"><strong>Spark Arena</strong></a> maintains an <a style="color: #18cc48;" href="https://spark-arena.com/leaderboard"><strong>LLM performance leaderboard</strong></a> ranking the performance of different models and inference engines on DGX Spark.</p>
</div><div class="fusion-text fusion-text-49"><h3>Tools</h3>
</div><div class="fusion-text fusion-text-50"><p>Docker alone is sufficient for running models on a single DGX Spark. With multiple Sparks, however, manually managing Docker containers, creating the cluster, and configuring the CX-7 network, SSH mesh, and NCCL communication can become considerably more complex. The community has developed several tools to simplify this process.</p>
</div><div class="fusion-text fusion-text-51"><p><a style="color: #18cc48;" href="https://github.com/eugr/spark-vllm-docker"><strong>eugr/spark-vllm-docker</strong></a>, developed by eugr, is one of the most comprehensive toolsets for building Docker images with native SM 12.1a kernels for GB10. It includes build scripts, cluster initialization, recipe deployment, model downloading, node discovery, runtime modifications, and ready-to-use recipes. These tools are widely used by the community.</p>
</div><div class="fusion-text fusion-text-52"><p><a style="color: #18cc48;" href="https://sparkrun.dev/"><strong>sparkrun</strong></a>, developed by Spark Arena, automates cluster management. Its setup wizard configures the CX-7 network, SSH mesh, and NCCL environment with a single command. A YAML recipe file can then be used to define the model, Docker image, inference parameters, and memory configuration. sparkrun also provides comprehensive real-time cluster monitoring and has become a widely used cluster-management tool within the DGX Spark ecosystem.</p>
</div><div class="fusion-text fusion-text-53"><h2>OpenZeka Resources</h2>
</div><div class="fusion-text fusion-text-54"><p>At OpenZeka, we publish Turkish and English tutorials covering our work with DGX Spark. English-language articles are available at <a style="color: #18cc48;" href="https://blog.openzeka.com/en/"><strong>blog.openzeka.com/en/</strong></a>. The blog primarily focuses on user education, including tutorials, step-by-step model and tool installation instructions, and practical usage guides. Some of our DGX Spark articles are listed below:</p>
</div><div class="fusion-text fusion-text-55"><ul>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/serving-local-llm-on-dgx-spark/"><strong>Serving a Local LLM with vLLM on DGX Spark</strong></a></strong>: Single Spark, vLLM, Open WebUI</li>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/local-gpt-oss-120b-serving-on-dgx-spark-with-sparkrun/"><strong>Local GPT-OSS 120B Serving on DGX Spark with sparkrun</strong></a></strong>: Single/dual Spark, GPT-OSS 120B, vLLM, sparkrun</li>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/deepseek-v4-flash-0731-on-2x-dgx-spark/"><strong>DeepSeek-V4-Flash-0731 on 2x DGX Spark</strong></a></strong>: 2× Spark, DeepSeek-V4-Flash-0731, vLLM, sparkrun</li>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/local-ai-workspace-with-odysseus-on-dgx-spark/"><strong>Local AI Workspace with Odysseus on DGX Spark</strong></a></strong>: Local AI workspace, agents, RAG, tool use</li>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/local-model-fine-tuning-with-unsloth-on-dgx-spark/"><strong>Local Model Fine-Tuning with Unsloth on DGX Spark</strong></a></strong>: Fine-tuning, Unsloth, QLoRA</li>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/gpu-accelerated-data-science-and-machine-learning-on-dgx-spark/"><strong>GPU-Accelerated Data Science and Machine Learning on DGX Spark</strong></a></strong>: CUDA-X, RAPIDS, cuDF, cuML</li>
<li><strong><a style="color: #18cc48;" href="https://blog.openzeka.com/en/building-a-high-speed-ai-cluster-with-4-dgx-spark-systems/"><strong>Building a High-Speed AI Cluster with 4 DGX Spark Systems</strong></a></strong>: 4× Spark cluster, 200GbE, RDMA, RoCEv2</li>
</ul>
</div><div class="fusion-text fusion-text-56"><p>Our white papers are published at <a style="color: #18cc48;" href="https://whitepapers.openzeka.com/"><strong>whitepapers.openzeka.com</strong></a>. Unlike the blog&#8217;s user-education-oriented content, the white papers target a more technical and professional audience. They provide in-depth analyses intended to support decision-making in areas such as hardware selection, benchmark interpretation, cluster architecture comparison, and scaling strategies. Some of our white papers are listed below:</p>
</div><div class="fusion-text fusion-text-57"><ul>
<li><strong><a style="color: #18cc48;" href="https://whitepapers.openzeka.com/papers/yerel-llm-rehberi/"><strong>Local LLM Usage Guide</strong></a></strong>: Hardware, model, and software selection guide</li>
<li><strong><a style="color: #18cc48;" href="https://whitepapers.openzeka.com/papers/qwen3.6-27b-dgx-spark-benchmark/"><strong>Qwen3.6-27B DGX Spark Benchmark</strong></a></strong>: FP8, AWQ, and NVFP4 performance analysis</li>
<li><strong><a style="color: #18cc48;" href="https://whitepapers.openzeka.com/papers/qwen3.6-27b-dgx-spark-scaling/"><strong>Qwen3.6-27B DGX Spark Cluster Scaling</strong></a></strong>: 1×/2×/4× Spark scaling analysis</li>
<li><strong><a style="color: #18cc48;" href="https://whitepapers.openzeka.com/papers/dgx-spark-2node-cluster-kurulumu/"><strong>DGX Spark 2-Node Cluster Setup Guide</strong></a></strong>: Two-node cluster setup</li>
<li><strong><a style="color: #18cc48;" href="https://whitepapers.openzeka.com/papers/dgx-spark-3node-cluster-kurulumu/"><strong>DGX Spark 3-Node Cluster Setup Guide</strong></a></strong>: Three-node ring topology</li>
<li><strong><a style="color: #18cc48;" href="https://whitepapers.openzeka.com/papers/dgx-spark-4node-cluster-kurulumu/"><strong>DGX Spark 4-Node Cluster Setup Guide</strong></a></strong>: Four-node switch topology</li>
</ul>
</div><div class="fusion-text fusion-text-58"><p>We also publish a variety of tutorials, tests, and product-analysis videos on the <a style="color: #18cc48;" href="https://www.youtube.com/@openzeka"><strong>OpenZeka</strong></a> and <a style="color: #18cc48;" href="https://www.youtube.com/@cordatusai"><strong>CordatusAI</strong></a> YouTube channels.</p>
</div><div class="fusion-text fusion-text-59"><h2>Forums and Documentation</h2>
</div><div class="fusion-text fusion-text-60"><p>The <a style="color: #18cc48;" href="https://forums.developer.nvidia.com/c/accelerated-computing/dgx-spark-gb10/719"><strong>DGX Spark / GB10</strong></a> category of the NVIDIA Developer Forums is a community discussion space in which NVIDIA engineers also actively participate. It contains hundreds of discussions covering model deployment recipes, performance optimization, newly released models, and many other topics. It is one of the first resources worth consulting when encountering a DGX Spark-related issue. The <a style="color: #18cc48;" href="https://developer.nvidia.com/topics/ai/dgx-spark"><strong>NVIDIA Developer</strong></a> page brings together links to this forum, official documentation, and playbooks in one place.</p>
</div><div class="fusion-text fusion-text-61"><p>Official DGX Spark documentation is available at <a style="color: #18cc48;" href="https://docs.nvidia.com/dgx/dgx-spark/index.html"><strong>docs.nvidia.com/dgx/dgx-spark</strong></a>. The <a style="color: #18cc48;" href="https://docs.nvidia.com/dgx/dgx-spark/index.html"><strong>User Guide</strong></a> covers hardware specifications, DGX OS, ConnectX-7 clustering, system recovery, and known issues, with dedicated pages for <a style="color: #18cc48;" href="https://docs.nvidia.com/dgx/dgx-spark/hardware.html"><strong>hardware specifications</strong></a>, <a style="color: #18cc48;" href="https://docs.nvidia.com/dgx/dgx-spark/os-and-component-update.html"><strong>OS and component updates</strong></a>, and <a style="color: #18cc48;" href="https://docs.nvidia.com/dgx/dgx-spark/release-notes.html"><strong>release notes</strong></a>. For multi-Spark deployments, NVIDIA provides documentation for the <a style="color: #18cc48;" href="https://docs.nvidia.com/sync/latest/cluster-assistant.html"><strong>NVIDIA Sync Cluster Assistant</strong></a> and <a style="color: #18cc48;" href="https://docs.nvidia.com/sync/latest/cluster-network-inspection.html"><strong>cluster network inspection</strong></a>. For performance analysis, documentation is available for both <a style="color: #18cc48;" href="https://docs.nvidia.com/nsight-systems/UserGuide/index.html"><strong>Nsight Systems</strong></a> and <a style="color: #18cc48;" href="https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html"><strong>Nsight Compute</strong></a>.</p>
</div><div class="fusion-text fusion-text-62"><h2>Spark Playbook Catalog</h2>
</div><div class="fusion-text fusion-text-63"><p>NVIDIA provides a collection of <a style="color: #18cc48;" href="https://build.nvidia.com/spark"><strong>Spark Playbooks</strong></a> for DGX Spark. Each playbook contains step-by-step installation and execution instructions for a particular tool or workflow. Some of these tools directly run models, while others provide the surrounding infrastructure required to make those models usable in practical applications.</p>
</div><div class="fusion-text fusion-text-64"><h3>Inference Engines</h3>
</div><div class="fusion-text fusion-text-65"><p>Models are typically developed using machine learning frameworks such as PyTorch or JAX. Inference engines load or convert the resulting model weights and optimize GPU kernels, KV cache management, batching, and memory utilization to execute the model efficiently. Some also provide an OpenAI-compatible API through either a built-in or separate model-server component.</p>
</div><div class="fusion-text fusion-text-66"><ul>
<li><strong>TensorRT-LLM</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/trt-llm"><strong>trt-llm</strong></a> — TensorRT-LLM is an open-source inference library optimized for LLM execution on NVIDIA GPUs. It uses optimized kernels and supports tensor, pipeline, and sequence parallelism. It integrates with Hugging Face and PyTorch.</li>
<li><strong>vLLM</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/vllm"><strong>vllm</strong></a> — vLLM is a high-throughput inference engine that uses PagedAttention and continuous batching to improve memory efficiency. It also provides an OpenAI-compatible model server.</li>
<li><strong>llama.cpp</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/llama-cpp"><strong>llama-cpp</strong></a> — llama.cpp is a lightweight C/C++ inference engine for running GGUF models across CPUs and multiple GPU backends. Its <code>llama-server</code> component can expose an OpenAI-compatible API.</li>
<li><strong>SGLang</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/sglang"><strong>sglang</strong></a> — SGLang is a high-performance inference and model-serving framework for LLMs and VLMs. It jointly optimizes the backend runtime and model interaction layer.</li>
<li><strong>NVIDIA NIM</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nim-llm"><strong>nim-llm</strong></a> — NIM is NVIDIA&#8217;s solution for packaging inference engines such as those listed above into containerized HTTP microservices. Rather than being an inference engine itself, it acts as a packaging and deployment layer that prepares these engines for production and enterprise use.</li>
</ul>
</div><div class="fusion-text fusion-text-67"><h3>Applications and Tools</h3>
</div><div class="fusion-text fusion-text-68"><p>These tools allow users and developers to interact directly with models. Live VLM WebUI and Continue connect to an existing backend service, whereas Ollama and LM Studio also include their own inference runtimes.</p>
</div><div class="fusion-text fusion-text-69"><ul>
<li><strong>Ollama + Open WebUI</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/open-webui"><strong>open-webui</strong></a> — Ollama is a local model-serving layer built on llama.cpp that provides a CLI, model library, and OpenAI-compatible API. Open WebUI is a self-hosted web interface that connects to this service.</li>
<li><strong>LM Studio</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/lm-studio"><strong>lm-studio</strong></a> — LM Studio is a local AI application that combines model discovery, execution, and API-based serving.</li>
<li><strong>Live VLM WebUI</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/live-vlm-webui"><strong>live-vlm-webui</strong></a> — Live VLM WebUI is a web interface that sends webcam streams to Ollama, vLLM, SGLang, or cloud-hosted VLM services. It allows the outputs and performance of different backend models to be compared.</li>
<li><strong>Continue + VS Code</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/vibe-coding"><strong>vibe-coding</strong></a> — Continue is a coding client that runs inside VS Code. In the playbook, Ollama serves the model while Continue connects to that service for chat and code generation.</li>
</ul>
</div><div class="fusion-text fusion-text-70"><h3>Models and Inference Optimizations</h3>
</div><div class="fusion-text fusion-text-71"><p>Nemotron is NVIDIA&#8217;s model family. Speculative decoding and NVFP4, meanwhile, are techniques intended to improve the performance or memory efficiency of compatible inference engines.</p>
</div><div class="fusion-text fusion-text-72"><ul>
<li><strong>NVIDIA Nemotron</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nemotron"><strong>nemotron</strong></a> — Nemotron is NVIDIA&#8217;s open model family designed for reasoning, tool use, and long-context tasks. It can be served with llama.cpp, vLLM, SGLang, or TensorRT-LLM.</li>
<li><strong>Speculative decoding</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/speculative-decoding"><strong>speculative-decoding</strong></a> — Speculative decoding accelerates the decoding process by having a smaller, faster draft model propose tokens that are then verified in batches by the target model. The playbook demonstrates this technique with EAGLE-3 and Draft–Target methods in TensorRT-LLM.</li>
<li><strong>NVFP4 quantization</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nvfp4-quantization"><strong>nvfp4-quantization</strong></a> — NVFP4 is a Blackwell-compatible 4-bit quantization format for model weights and activations. The playbook converts the model with TensorRT Model Optimizer and then runs it in a TensorRT-LLM environment.</li>
</ul>
</div><div class="fusion-text fusion-text-73"><h3>Fine-Tuning Frameworks and Tools</h3>
</div><div class="fusion-text fusion-text-74"><p>Fine-tuning tools use existing model weights to produce updated model weights. The resulting model can subsequently be run using an inference engine.</p>
</div><div class="fusion-text fusion-text-75"><p>PyTorch is the underlying machine learning framework. NeMo, LLaMA-Factory, and Unsloth provide more streamlined and optimized training workflows built on top of PyTorch.</p>
</div><div class="fusion-text fusion-text-76"><ul>
<li><strong>NVIDIA NeMo</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nemo-fine-tune"><strong>nemo-fine-tune</strong></a> — NeMo AutoModel is a framework for training Hugging Face models with native PyTorch support and scaling them to multi-node systems. It supports SFT, PEFT, and distributed training workflows.</li>
<li><strong>LLaMA-Factory</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/llama-factory"><strong>llama-factory</strong></a> — LLaMA-Factory is a fine-tuning framework that provides a common interface for optimization methods such as SFT, LoRA, and QLoRA across different LLMs and VLMs.</li>
<li><strong>Unsloth</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/unsloth"><strong>unsloth</strong></a> — Unsloth is a fine-tuning toolkit that uses custom GPU kernels to accelerate LoRA and QLoRA training while reducing memory consumption.</li>
<li><strong>PyTorch</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/pytorch-fine-tune"><strong>pytorch-fine-tune</strong></a> — PyTorch is a foundational machine learning framework that provides tensor operations, automatic differentiation, and GPU acceleration. This playbook implements SFT, LoRA, and QLoRA workflows directly in PyTorch at a lower level of abstraction.</li>
<li><strong>FLUX.1 DreamBooth LoRA</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/flux-finetuning"><strong>flux-finetuning</strong></a> — This playbook uses DreamBooth LoRA to teach new concepts and styles to the FLUX.1-dev diffusion image-generation model.</li>
</ul>
</div><div class="fusion-text fusion-text-77"><h3>Agentic AI Systems</h3>
</div><div class="fusion-text fusion-text-78"><p>AI agents connect to an LLM service for decision-making while interacting with files, terminal commands, APIs, and other tools. The model itself is run by a service such as vLLM or Ollama, while the agent application manages task planning, memory, and tool use.</p>
</div><div class="fusion-text fusion-text-79"><p>OpenClaw and Hermes are agent applications. OpenShell is a sandbox runtime that surrounds the agent process with filesystem, network, permission, and security policies. NemoClaw combines these layers into an integrated deployment.</p>
</div><div class="fusion-text fusion-text-80"><ul>
<li><strong>OpenClaw</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/openclaw"><strong>openclaw</strong></a> — OpenClaw is a persistent local agent application combining memory, filesystem access, tool use, and skills. In the playbook, its decision-making model is served through vLLM.</li>
<li><strong>Hermes Agent</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/hermes-agent"><strong>hermes-agent</strong></a> — Hermes is an autonomous agent capable of generating reusable skills from its experiences, retaining memory across sessions, and running scheduled tasks. In the playbook, vLLM is used as the local model service.</li>
<li><strong>OpenShell</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/openshell"><strong>openshell</strong></a> — OpenShell is a sandbox runtime that constrains agents at the kernel level. The playbook runs OpenClaw inside the sandbox while the model service is accessed through an endpoint outside the sandbox.</li>
<li><strong>NemoClaw</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nemoclaw"><strong>nemoclaw</strong></a> — NemoClaw combines the OpenClaw agent, OpenShell sandbox, and local vLLM inference. Filesystem, network, process, and inference access are configured within a unified deployment.</li>
<li><strong>NemoClaw example agents</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nemoclaw-applications"><strong>nemoclaw-applications</strong></a> — This playbook deploys news summarization, software development, document review, and calendar negotiation agents on top of an existing NemoClaw sandbox, demonstrating how the same infrastructure can be adapted to different applications.</li>
<li><strong>CLI Coding Agents</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/cli-coding-agent"><strong>cli-coding-agent</strong></a> — Claude Code, OpenCode, and Codex CLI are terminal-based agents that manage file editing, command execution, and coding tasks. In the playbook, model inference is provided by Ollama.</li>
</ul>
</div><div class="fusion-text fusion-text-81"><h3>Complete AI Applications and Solutions</h3>
</div><div class="fusion-text fusion-text-82"><p>These applications combine models, inference engines, data processing, user interfaces, and agent components to solve specific problems. Instead of interacting with individual infrastructure layers, users interact with a complete end-to-end system.</p>
</div><div class="fusion-text fusion-text-83"><ul>
<li><strong>RAG Application in AI Workbench</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/rag-ai-workbench"><strong>rag-ai-workbench</strong></a> — An agentic RAG application combining query routing, information retrieval, answer generation, and hallucination evaluation. It can use either NVIDIA-hosted APIs or self-hosted model services.</li>
<li><strong>Video Search and Summarization</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/vss"><strong>vss</strong></a> — VSS uses VLM, LLM, and RAG components to generate video summaries, question-answering capabilities, and real-time alerts.</li>
<li><strong>Multi-Agent Chatbot</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/multi-agent-chatbot"><strong>multi-agent-chatbot</strong></a> — A supervisor agent coordinates coding, RAG, and visual-understanding agents. llama.cpp and TensorRT-LLM servers are used for model serving, while MCP is used for tool connectivity.</li>
<li><strong>Image Generation with TensorRT</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/multi-modal-inference"><strong>multi-modal-inference</strong></a> — This playbook optimizes the FLUX.1 and SDXL diffusion models with TensorRT for text-to-image generation.</li>
<li><strong>Text to Knowledge Graph</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/txt2kg"><strong>txt2kg</strong></a> — Extracts subject–predicate–object relationships from text using Ollama, stores the resulting data in ArangoDB, and visualizes it with Three.js WebGPU. Ollama performs inference while ArangoDB handles relationship queries.</li>
<li><strong>Spark &amp; Reachy Photo Booth</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/spark-reachy-photo-booth"><strong>spark-reachy-photo-booth</strong></a> — A multimodal application combining NeMo Agent Toolkit, TensorRT-LLM, speech recognition, speech synthesis, FLUX image generation, and object tracking with the Reachy Mini robot. Services communicate through a message bus.</li>
</ul>
</div><div class="fusion-text fusion-text-84"><h3>Other GPU-Accelerated Workloads</h3>
</div><div class="fusion-text fusion-text-85"><p>These playbooks are not part of AI inference or model-serving workflows. Instead, they demonstrate how DGX Spark&#8217;s computational acceleration capabilities can be applied to areas such as data science, optimization, bioinformatics, and robotics.</p>
</div><div class="fusion-text fusion-text-86"><ul>
<li><strong>CUDA-X Data Science</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/cuda-x-data-science"><strong>cuda-x-data-science</strong></a> — CUDA-X Data Science uses RAPIDS libraries such as cuDF and cuML to accelerate pandas and scikit-learn workflows on the GPU.</li>
<li><strong>Single-cell RNA Sequencing</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/single-cell"><strong>single-cell</strong></a> — RAPIDS-singlecell provides a Scanpy-like API for running scRNA-seq preprocessing, quality control, clustering, and visualization on the GPU. It uses RAPIDS components such as cuML and cuGraph.</li>
<li><strong>Portfolio Optimization</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/portfolio-optimization"><strong>portfolio-optimization</strong></a> — cuOpt solves portfolio constraints as LP/MILP optimization problems, while cuML accelerates the generation of risk scenarios. The playbook combines these tools in an end-to-end Mean-CVaR-based financial workflow.</li>
<li><strong>Isaac Sim and Isaac Lab</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/isaac"><strong>isaac</strong></a> — Isaac Sim is a GPU-accelerated robotics simulation platform. Isaac Lab is a framework built on top of this environment for developing reinforcement-learning policies.</li>
</ul>
</div><div class="fusion-text fusion-text-87"><h3>Multi-DGX Spark Deployments</h3>
</div><div class="fusion-text fusion-text-88"><p>These playbooks prepare multi-device infrastructure independently of the model or application layer. First, the QSFP network and inter-node SSH access are configured. NCCL then provides high-performance collective GPU communication over this connection for distributed training and inference.</p>
</div><div class="fusion-text fusion-text-89"><p>Higher-level software such as vLLM, TensorRT-LLM, PyTorch, and NeMo can use this infrastructure when operating in distributed configurations.</p>
</div><div class="fusion-text fusion-text-90"><ul>
<li><strong>2× DGX Spark direct connection</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/connect-two-sparks"><strong>connect-two-sparks</strong></a> — Connects two DGX Spark systems directly through a 200 GbE QSFP link and passwordless SSH, creating a two-node distributed system.</li>
<li><strong>3× DGX Spark ring connection</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/connect-three-sparks"><strong>connect-three-sparks</strong></a> — Connects three DGX Spark systems in a ring topology using three QSFP cables.</li>
<li><strong>4+ DGX Spark switch connection</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/multi-sparks-through-switch"><strong>multi-sparks-through-switch</strong></a> — Connects four or more DGX Spark systems through a QSFP switch to create a scalable cluster.</li>
<li><strong>NCCL</strong>: <a style="color: #18cc48;" href="https://build.nvidia.com/spark/nccl"><strong>nccl</strong></a> — NCCL is NVIDIA&#8217;s library for high-performance collective communication between GPUs located on different nodes.</li>
</ul>
</div><div class="fusion-text fusion-text-91"><p>All of these playbooks are available at <a style="color: #18cc48;" href="https://build.nvidia.com/spark"><strong>build.nvidia.com/spark</strong></a>.</p>
</div><div class="fusion-text fusion-text-92"><h2>Conclusion</h2>
</div><div class="fusion-text fusion-text-93"><p>DGX Spark has a strong and rapidly evolving ecosystem built on NVIDIA&#8217;s foundations and continuously expanded through a growing range of community projects. OpenZeka contributes to this ecosystem by making its technical knowledge more accessible to end users through Turkish- and English-language tutorials, technical analysis documents, practical guides, and YouTube videos.</p>
</div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/nvidia-dgx-spark-ecosystem/">NVIDIA DGX Spark Ecosystem</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Serving a Local LLM with vLLM on DGX Spark</title>
		<link>https://blog.openzeka.com/en/serving-local-llm-on-dgx-spark/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Thu, 20 Aug 2026 08:05:32 +0000</pubDate>
				<category><![CDATA[AI Cluster]]></category>
		<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1885</guid>

					<description><![CDATA[<p>In this tutorial, you will run a large language model  ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/serving-local-llm-on-dgx-spark/">Serving a Local LLM with vLLM on DGX Spark</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-8 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-7 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-94"><p>In this tutorial, you will run a large language model locally on a DGX Spark and chat with it from another computer&#8217;s browser.</p>
<p>The Spark can be used as a standalone computer by connecting a monitor and keyboard, or as a remote server accessed from another computer. In this tutorial, we will connect to the Spark remotely and install the necessary software to run the model.</p>
<p>We will use Qwen3.6-35B-A3B-FP8 (a 35-billion parameter Mixture-of-Experts model, 3 billion parameters active, FP8 quantization) as the language model, vLLM as the inference engine, and Open WebUI as the web interface. vLLM will load the model&#8217;s trained weights into GPU memory and expose an API that accepts external requests. Open WebUI will connect to this API and display a chat interface in the browser. Both will run on the Spark, inside Docker containers.</p>
<hr />
<h2>Prerequisites: Finding the Spark&#8217;s IP Address</h2>
<p>If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command in the terminal:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-21 > .CodeMirror, .fusion-syntax-highlighter-21 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-21 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_21" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_21" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_21" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ip route get 1.1.1.1 | grep -oP &#8216;src \K\S+&#8217;</textarea></div><div class="fusion-text fusion-text-95"><p>The command outputs the IP address of the Spark&#8217;s default network interface:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-22 > .CodeMirror, .fusion-syntax-highlighter-22 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-22 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_22" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_22" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_22" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">192.168.1.148</textarea></div><div class="fusion-text fusion-text-96"><p>Note this address; you will use it in place of <code></code> throughout this tutorial. Alternatively, you can find the IP address by checking the NVIDIA Sync application.</p>
<hr />
<h2>1. Connecting to the Spark</h2>
<p>Make sure your computer is connected to the same network as the Spark. Then, open a terminal on your computer and connect to the Spark via SSH:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-23 > .CodeMirror, .fusion-syntax-highlighter-23 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-23 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_23" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_23" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_23" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ssh nvidia@</textarea></div><div class="fusion-text fusion-text-97"><p>On the first connection, you will see a fingerprint warning. Type <code>yes</code> and press Enter. Then, when prompted for a password, enter the Spark&#8217;s password:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-24 > .CodeMirror, .fusion-syntax-highlighter-24 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-24 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_24" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_24" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_24" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">The authenticity of host &#8216;192.168.1.148 (192.168.1.148)&#8217; can&#8217;t be established.
ED25519 key fingerprint is SHA256:S6EECYc6Pw2aLoLmhblFZ0QEoeVtJP41jJ5IYsdOmMM.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added &#8216;192.168.1.148&#8217; (ED25519) to the list of known hosts.
nvidia@192.168.1.148&#8217;s password:
Welcome to NVIDIA DGX Spark Version 7.5.0 (GNU/Linux 6.17.0-1026-nvidia aarch64)</p>
<p>System information as of Tue Jul 14 12:46:44 PM UTC 2026</p>
<p>System load: 1.54 Temperature: 80.2 C
Usage of /: 59.3% of 3.67TB Processes: 501
Memory usage: 62% Users logged in: 0
Swap usage: 0% IPv4 address for enP7s7: 192.168.1.148</p>
<p>2 devices have a firmware upgrade available.
Run `fwupdmgr get-upgrades` for more information.</p>
<p>Last login: Tue Jul 14 12:47:08 2026 from 192.168.1.77</textarea></div><div class="fusion-text fusion-text-98"><p>Once connected, the Spark will start accepting commands sent from this terminal.</p>
<hr />
<h2>2. Clearing the Cache</h2>
<p>Before starting vLLM, flush the filesystem cache:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-25 > .CodeMirror, .fusion-syntax-highlighter-25 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-25 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_25" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_25" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_25" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sudo sh -c &#8216;sync; echo 3 &gt; /proc/sys/vm/drop_caches&#8217;</textarea></div><div class="fusion-text fusion-text-99"><p>The main reason for doing this is the DGX Spark&#8217;s unified memory architecture: the operating system caches model files read from disk in RAM. vLLM loads the model weights from here into GPU memory. After loading is complete, the cached data remains in RAM even though it will not be used again. On systems with separate memory, this is not significant — since inference runs in GPU memory, RAM utilization does not affect performance. On the Spark, however, the CPU and GPU share the same RAM, so the cache reduces the memory available to the GPU. This command frees the cache, providing maximum memory for vLLM.</p>
<p>When prompted for a password, enter the Spark&#8217;s password. The command produces no output and completes silently:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-26 > .CodeMirror, .fusion-syntax-highlighter-26 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-26 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_26" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_26" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_26" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[sudo] password for nvidia:</textarea></div><div class="fusion-text fusion-text-100"><hr />
<h2>3. Setting Up the vLLM Server</h2>
<p>Pull the vLLM Docker image:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-27 > .CodeMirror, .fusion-syntax-highlighter-27 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-27 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_27" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_27" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_27" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker pull vllm/vllm-openai:v0.22.1-ubuntu2404</textarea></div><div class="fusion-text fusion-text-101"><p>During the download, layers will be pulled sequentially and you will see progress percentages on the screen. If the image is already on disk, Docker skips the download. Once the download is complete, you will see the following output:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-28 > .CodeMirror, .fusion-syntax-highlighter-28 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-28 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_28" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_28" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_28" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">v0.22.1-ubuntu2404: Pulling from vllm/vllm-openai
Digest: sha256:525c7fb8b20102c501bf9d066a4671c94468e58db9b072972d743a327e8ab909
Status: Image is up to date for vllm/vllm-openai:v0.22.1-ubuntu2404
docker.io/vllm/vllm-openai:v0.22.1-ubuntu2404</textarea></div><div class="fusion-text fusion-text-102"><p>This image contains everything needed to run vLLM — PyTorch, CUDA kernels, Python libraries&#8230;</p>
<p>Now, let&#8217;s start the vLLM server:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-29 > .CodeMirror, .fusion-syntax-highlighter-29 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-29 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_29" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_29" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_29" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker run -d &#8211;gpus all -p 8000:8000 &#8211;name vllm-qwen36-35b-fp8 \
-v /home/nvidia/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:v0.22.1-ubuntu2404 \
Qwen/Qwen3.6-35B-A3B-FP8 \
&#8211;host 0.0.0.0 &#8211;port 8000 \
&#8211;tensor-parallel-size 1 \
&#8211;trust-remote-code \
&#8211;gpu-memory-utilization 0.55 \
&#8211;max-model-len 131072 \
&#8211;max-num-seqs 4 \
&#8211;max-num-batched-tokens 8192 \
&#8211;kv-cache-dtype fp8 \
&#8211;enable-chunked-prefill \
&#8211;async-scheduling \
&#8211;enable-prefix-caching \
&#8211;load-format fastsafetensors \
&#8211;reasoning-parser qwen3 \
&#8211;tool-call-parser qwen3_xml \
&#8211;enable-auto-tool-choice</textarea></div><div class="fusion-text fusion-text-103"><p>This command starts the downloaded image as a container. If the model is not already installed on the Spark, it will be automatically downloaded from Hugging Face. As the download progresses, you will see progress bars, and once complete, the files will be placed in the <code>~/.cache/huggingface/hub/</code> directory. If the model has been previously downloaded on the Spark (if you have followed this tutorial before), vLLM skips the download step and loads the downloaded weights directly into GPU memory.</p>
<p>When the command finishes, you will see a container ID; this indicates that the server has started in the background and the model is now being served on port 8000:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-30 > .CodeMirror, .fusion-syntax-highlighter-30 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-30 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_30" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_30" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_30" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">image_id => a76498707041a0aa72759f694d3deb9150b5fb6da34510be7ed19e52f9ba104a</textarea></div><div class="fusion-text fusion-text-104"><p>If a container with the same name (from your previous attempts) already exists, the command will return the following error:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-31 > .CodeMirror, .fusion-syntax-highlighter-31 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-31 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_31" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_31" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_31" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">docker: Error response from daemon: Conflict. The container name &#8220;/vllm-qwen36-35b-fp8&#8221; is already in use by container &#8230;</textarea></div><div class="fusion-text fusion-text-105"><p>In this case, check the container list — vllm-qwen36-35b-fp8 should be listed:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-32 > .CodeMirror, .fusion-syntax-highlighter-32 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-32 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_32" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_32" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_32" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker ps -a &#8211;filter name=vllm-qwen36-35b-fp8</textarea></div><div class="fusion-text fusion-text-106"><p>If the status column shows &#8220;Up&#8221;, the container is already running and ready to use — you can proceed to the next step:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-33 > .CodeMirror, .fusion-syntax-highlighter-33 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-33 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_33" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_33" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_33" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">CONTAINER ID IMAGE STATUS PORTS NAMES
a76498707041 vllm/vllm-openai:v0.22.1-ubuntu2404 Up 5 minutes 0.0.0.0:8000-&gt;8000/tcp, [::]:8000-&gt;8000/tcp vllm-qwen36-35b-fp8</textarea></div><div class="fusion-text fusion-text-107"><p>If the status column shows &#8220;Exited&#8221;, the container has stopped:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-34 > .CodeMirror, .fusion-syntax-highlighter-34 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-34 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_34" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_34" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_34" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">CONTAINER ID IMAGE STATUS PORTS NAMES
a76498707041 vllm/vllm-openai:v0.22.1-ubuntu2404 Exited (0) 2 minutes ago vllm-qwen36-35b-fp8</textarea></div><div class="fusion-text fusion-text-108"><p>You can restart the stopped container and proceed to the next step:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-35 > .CodeMirror, .fusion-syntax-highlighter-35 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-35 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_35" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_35" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_35" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker start vllm-qwen36-35b-fp8</textarea></div><div class="fusion-text fusion-text-109"><hr />
<h2>4. Monitoring the vLLM Startup Process</h2>
<p>After the model download is complete, it may take a few minutes for vLLM to become ready for serving. During this process, vLLM loads the model weights into GPU memory, compiles GPU kernels, and allocates memory for inference. To monitor this process:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-36 > .CodeMirror, .fusion-syntax-highlighter-36 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-36 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_36" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_36" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_36" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker logs -f vllm-qwen36-35b-fp8</textarea></div><div class="fusion-text fusion-text-110"><p>You will see the following in the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-37 > .CodeMirror, .fusion-syntax-highlighter-37 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-37 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_37" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_37" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_37" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">(APIServer pid=1) INFO 07-14 11:55:43 [utils.py:344]
(APIServer pid=1) INFO 07-14 11:55:43 [utils.py:344] █ █ █▄ ▄█
(APIServer pid=1) INFO 07-14 11:55:43 [utils.py:344] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.22.1
(APIServer pid=1) INFO 07-14 11:55:43 [utils.py:344] █▄█▀ █ █ █ █ model Qwen/Qwen3.6-35B-A3B-FP8
(APIServer pid=1) INFO 07-14 11:55:43 [utils.py:344] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=1) INFO 07-14 11:55:53 [model.py:617] Resolved architecture: Qwen3_5MoeForConditionalGeneration
(EngineCore pid=237) INFO 07-14 11:56:28 [gpu_model_runner.py:5037] Starting to load model Qwen/Qwen3.6-35B-A3B-FP8&#8230;
(EngineCore pid=237) Loading safetensors using Fastsafetensor loader: 0% Completed | 0/42 [00:00<!--?, ?it/s&#093;&lt;br ?--> (EngineCore pid=237) Loading safetensors using Fastsafetensor loader: 100% Completed | 42/42 [00:06&lt;00:00, 6.47it/s]
(EngineCore pid=237) INFO 07-14 11:56:37 [default_loader.py:397] Loading weights took 6.49 seconds
(EngineCore pid=237) INFO 07-14 11:57:18 [monitor.py:53] torch.compile took 33.04 s in total
(EngineCore pid=237) INFO 07-14 11:58:01 [gpu_worker.py:466] Available KV cache memory: 27.17 GiB
(APIServer pid=1) INFO 07-14 11:58:05 [parser_manager.py:202] &#8220;auto&#8221; tool choice has been enabled.
(APIServer pid=1) INFO 07-14 11:58:26 [base.py:224] Multi-modal warmup completed in 14.933s
(APIServer pid=1) INFO: Started server process [1]
(APIServer pid=1) INFO: Waiting for application startup.
(APIServer pid=1) INFO: Application startup complete.</textarea></div><div class="fusion-text fusion-text-111"><p>Once you see the <code>Application startup complete.</code> line, the model server is ready. Press <code>Ctrl+C</code> to stop watching the logs. The server will continue running in the background.</p>
<hr />
<h2>5. Testing the vLLM Server</h2>
<p>First, let&#8217;s verify that the server is running:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-38 > .CodeMirror, .fusion-syntax-highlighter-38 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-38 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_38" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_38" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_38" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl http://localhost:8000/health</textarea></div><div class="fusion-text fusion-text-112"><p>If you don&#8217;t receive any errors, the server is healthy:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-39 > .CodeMirror, .fusion-syntax-highlighter-39 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-39 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_39" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_39" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_39" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt"> % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 &#8211;:&#8211;:&#8211; &#8211;:&#8211;:&#8211; &#8211;:&#8211;:&#8211; 0</textarea></div><div class="fusion-text fusion-text-113"><p>Now test the model:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-40 > .CodeMirror, .fusion-syntax-highlighter-40 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-40 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_40" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_40" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_40" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl http://localhost:8000/v1/chat/completions \
-H &#8220;Content-Type: application/json&#8221; \
-d &#8216;{
&#8220;model&#8221;: &#8220;Qwen/Qwen3.6-35B-A3B-FP8&#8221;,
&#8220;messages&#8221;: [{&#8220;role&#8221;: &#8220;user&#8221;, &#8220;content&#8221;: &#8220;What is 12 times 17? Answer briefly.&#8221;}],
&#8220;max_tokens&#8221;: 500
}&#8217; | python3 -m json.tool</textarea></div><div class="fusion-text fusion-text-114"><p>The simplified response is shown below. In the <code>content</code> field (the model&#8217;s response), you will see the number <code>204</code>. This confirms that the model performed the multiplication correctly. Additionally, the response includes a <code>reasoning</code> field. This field contains the model&#8217;s thought process. As you can see, the model performed the calculation using two separate methods and verified the solution before arriving at the answer:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-41 > .CodeMirror, .fusion-syntax-highlighter-41 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-41 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_41" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_41" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_41" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="application/json">{
&#8220;id&#8221;: &#8220;chatcmpl-b49aa5be1287d18f&#8221;,
&#8220;object&#8221;: &#8220;chat.completion&#8221;,
&#8220;model&#8221;: &#8220;Qwen/Qwen3.6-35B-A3B-FP8&#8221;,
&#8220;choices&#8221;: [
{
&#8220;index&#8221;: 0,
&#8220;message&#8221;: {
&#8220;role&#8221;: &#8220;assistant&#8221;,
&#8220;content&#8221;: &#8220;\n\n204&#8221;,
&#8220;reasoning&#8221;: &#8220;Thinking Process:\n\n1. **Identify the user&#8217;s request:** The user wants to know the result of multiplying 12 by 17.\n2. **Constraint:** The user requested a brief answer.\n3. **Perform the calculation:**\n * $12 \\times 17$\n * $12 \\times 10 = 120$\n * $12 \\times 7 = 84$\n * $120 + 84 = 204$\n * Alternative: $17 \\times 10 = 170$, $17 \\times 2 = 34$, $170 + 34 = 204$.\n4. **Formulate the response:** \&#8221;204\&#8221;.\n5. **Check constraints:** Is it brief? Yes.\n\nFinal Answer: 204.\n&#8221;
},
&#8220;finish_reason&#8221;: &#8220;stop&#8221;
}
],
&#8220;usage&#8221;: {
&#8220;prompt_tokens&#8221;: 23,
&#8220;total_tokens&#8221;: 234,
&#8220;completion_tokens&#8221;: 211
}
}</textarea></div><div class="fusion-text fusion-text-115"><p>vLLM is now running and we can ask questions to Qwen3.6 and receive responses. However, sending each question as a terminal command like the one above is not practical. We need a web interface to chat from the browser.</p>
<hr />
<h2>6. Setting Up Open WebUI</h2>
<p>Open WebUI is an interface that connects to vLLM and can be used from your browser. You can chat with your language model and upload images to ask questions about them. Responses start appearing on the screen as they are being generated — you don&#8217;t have to wait for the entire response to finish before you start reading. For each chat, you can adjust settings such as temperature, top_p, and max_tokens. On first setup, it downloads a small embedding model, which enables you to upload documents and ask questions about them (RAG — Retrieval-Augmented Generation). Your chat history, settings, and account information are stored on the Spark&#8217;s disk; even if the device or container is shut down, your data is not lost, and you can continue from where you left off when you restart.</p>
<p>First, pull the Open WebUI Docker image on the Spark:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-42 > .CodeMirror, .fusion-syntax-highlighter-42 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-42 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_42" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_42" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_42" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker pull ghcr.io/open-webui/open-webui:main</textarea></div><div class="fusion-text fusion-text-116"><p>If the image is already on disk, Docker skips the download. Once the download is complete, you will see the following output:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-43 > .CodeMirror, .fusion-syntax-highlighter-43 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-43 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_43" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_43" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_43" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">main: Pulling from open-webui/open-webui
Digest: sha256:a26effeb220e132482bf7e0560b3404843e7bc40d23051144e062960df8df6b0
Status: Image is up to date for ghcr.io/open-webui/open-webui:main
ghcr.io/open-webui/open-webui:main</textarea></div><div class="fusion-text fusion-text-117"><p>Once the download is complete, start the container:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-44 > .CodeMirror, .fusion-syntax-highlighter-44 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-44 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_44" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_44" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_44" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker run -d &#8211;network=host \
-e OPENAI_API_BASE_URLS=http://localhost:8000/v1 \
-e OPENAI_API_KEY=not-needed \
-e ENABLE_SIGNUP=true \
-v open-webui-data:/app/backend/data \
&#8211;name open-webui \
ghcr.io/open-webui/open-webui:main</textarea></div><div class="fusion-text fusion-text-118"><p>The command returns a container ID, indicating that the container has started:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-45 > .CodeMirror, .fusion-syntax-highlighter-45 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-45 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_45" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_45" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_45" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">id => 54563f6e8abb5eea54c328561ccb83df5eb29b3a18382cb083ddb210f637bcd4</textarea></div><div class="fusion-text fusion-text-119"><p>As mentioned in the vLLM setup section, if a container with the same name (from your previous attempts) already exists, the command will return the following error:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-46 > .CodeMirror, .fusion-syntax-highlighter-46 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-46 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_46" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_46" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_46" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">docker: Error response from daemon: Conflict. The container name &#8220;/open-webui&#8221; is already in use by container &#8230;</textarea></div><div class="fusion-text fusion-text-120"><p>In this case, check the container list — open-webui should be listed:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-47 > .CodeMirror, .fusion-syntax-highlighter-47 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-47 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_47" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_47" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_47" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker ps -a &#8211;filter name=open-webui</textarea></div><div class="fusion-text fusion-text-121"><p>If the status column shows &#8220;Up&#8221;, the container is already running and ready to use — you can proceed to the next step:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-48 > .CodeMirror, .fusion-syntax-highlighter-48 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-48 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_48" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_48" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_48" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">CONTAINER ID IMAGE STATUS PORTS NAMES
54563f6e8abb ghcr.io/open-webui/open-webui:main Up 5 minutes (healthy) open-webui</textarea></div><div class="fusion-text fusion-text-122"><p>If the status column shows &#8220;Exited&#8221;, the container has stopped:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-49 > .CodeMirror, .fusion-syntax-highlighter-49 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-49 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_49" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_49" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_49" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">CONTAINER ID IMAGE STATUS PORTS NAMES
54563f6e8abb ghcr.io/open-webui/open-webui:main Exited (0) 2 minutes ago open-webui</textarea></div><div class="fusion-text fusion-text-123"><p>You can restart the stopped container and proceed to the next step:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-50 > .CodeMirror, .fusion-syntax-highlighter-50 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-50 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_50" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_50" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_50" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker start open-webui</textarea></div><div class="fusion-text fusion-text-124"><p>Your Open WebUI server will then be ready for use.</p>
<hr />
<h2>7. Monitoring the Open WebUI Startup Process</h2>
<p>To watch the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-51 > .CodeMirror, .fusion-syntax-highlighter-51 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-51 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_51" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_51" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_51" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker logs -f open-webui</textarea></div><div class="fusion-text fusion-text-125"><p>You will see output similar to the following in the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-52 > .CodeMirror, .fusion-syntax-highlighter-52 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-52 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_52" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_52" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_52" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">No WEBUI_SECRET_KEY environment variable set, loading from file.
Generating new WEBUI_SECRET_KEY&#8230;
Loading WEBUI_SECRET_KEY from .webui_secret_key
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
WARNI [open_webui.env]
<p>WARNING: CORS_ALLOW_ORIGIN IS SET TO &#8216;*&#8217; &#8211; NOT RECOMMENDED FOR PRODUCTION DEPLOYMENTS.</p>
<p>██████╗ ██████╗ ███████╗███╗ ██║ ██╗ ██╗███████╗██████╗ ██╗ ██╗██╗
██╔═══██╗██╔══██╗██╔════╝████╗ ██║ ██║ ██║██╔════╝██╔══██╗██║ ██║██║
██║ ██╗██████╔╝█████╗ ██╔██╗ ██║ ██║ █╗ ██║█████╗ ██████╔╝██║ ██║██║
██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║ ██║███╗██║██╔══╝ ██╔══██╗██║ ██║██║
╚██████╔╝██║ ███████╗██║ ╚████║ ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║
╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝ ╚══╝╚══╝ ╚══════╝╚═════╝ ╚═════╝ ╚═╝</p>
<p>v0.10.2 &#8211; building the best AI user interface.</p>
<p>INFO: Started server process [1]
INFO: Waiting for application startup.
HTTP Request: GET https://huggingface.co/api/models/sentence-transformers/all-MiniLM-L6-v2/revision/main &#8220;HTTP/1.1 200 OK&#8221;
Fetching 30 files: 100%|██████████| 30/30 [00:00&lt;00:00, 6815.94it/s]
Loading SentenceTransformer model from &#8230;/all-MiniLM-L6-v2/snapshots/&#8230;
Loading weights: 100%|██████████| 103/103 [00:00&lt;00:00, 14320.73it/s]
2026-07-14 11:45:51.407 | INFO | open_webui.utils.automations:scheduler_worker_loop:176 &#8211; Scheduler worker started (poll interval: 10s)</textarea></div><div class="fusion-text fusion-text-126"><p>After you see the <code>Scheduler worker started</code> line in the logs, wait a few seconds. Press <code>Ctrl+C</code> to exit log monitoring.</p>
<hr />
<h2>8. Testing Open WebUI</h2>
<p>To verify that the server is ready and healthy, run the following command:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-53 > .CodeMirror, .fusion-syntax-highlighter-53 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-53 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_53" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_53" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_53" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl http://localhost:8080/health</textarea></div><div class="fusion-text fusion-text-127"><p>If you receive an HTTP 200 response, Open WebUI is ready:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-54 > .CodeMirror, .fusion-syntax-highlighter-54 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-54 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_54" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_54" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_54" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">{&#8220;status&#8221;:true}</textarea></div><div class="fusion-text fusion-text-128"><hr />
<h2>9. Accessing from the Browser</h2>
<p>On your computer&#8217;s browser, which is on the same network as the Spark, navigate to:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-55 > .CodeMirror, .fusion-syntax-highlighter-55 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-55 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_55" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_55" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_55" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">http://:8080</textarea></div><div class="fusion-text fusion-text-129"><p>On first launch, a login screen will appear. Click the &#8220;Sign up&#8221; link and enter your name, email, and password. The first account created automatically becomes the administrator account. The purpose of creating an account is to store chat history and settings. This information is stored on the Spark.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-9 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_signup" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1024x579.webp" alt class="img-responsive wp-image-1842" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-130"><p>The model is automatically detected by Open WebUI and listed in the model selection menu. You will see the <code>Qwen/Qwen3.6-35B-A3B-FP8</code> model in the menu. Select the model. If you are running multiple models at the same time, you can instantly switch between models from here. Additionally, the chat history tab on the left allows you to create multiple chat sessions and switch between them.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-10 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_signup" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1024x579.webp" alt class="img-responsive wp-image-1842" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_signup.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-131"><p>Type a question in the message box and press Enter. As the model responds, a collapsible reasoning section will appear at the top, and the response will appear below.</p>
<p>Here are a few experiments we conducted to demonstrate what the model and web interface can do:</p>
<p>First, to test the model&#8217;s Turkish comprehension skills, we asked the model to list Turkey&#8217;s three largest cities and provide brief information about each.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-11 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_sehirler-prompt" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-1024x579.webp" alt class="img-responsive wp-image-1841" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_sehirler-prompt.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-132"><p>Next, we asked the model to explain the benefits of artificial intelligence in the healthcare sector in three paragraphs.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-12 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_saglik-prompt" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-1024x579.webp" alt class="img-responsive wp-image-1840" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_saglik-prompt.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-133"><p>After that, to test the model&#8217;s programming skills, we asked it to create a Python function that checks whether a text is a palindrome.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-13 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_python-prompt" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-1024x579.webp" alt class="img-responsive wp-image-1839" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_python-prompt.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-134"><p>Using the file upload button in the message area, we sent the model an image of nature. The model described elements in the uploaded image such as flowers, leaves, water drops, and the background.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-14 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_image-prompt" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-1024x579.webp" alt class="img-responsive wp-image-1837" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_image-prompt.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-135"><p>From the same file upload section, this time we gave the model the DGX Spark datasheet and asked it to analyze it.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-15 hover-type-none"><img decoding="async" width="1024" height="579" title="open-webui_pdf-prompt" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-1024x579.webp" alt class="img-responsive wp-image-1838" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/open-webui_pdf-prompt.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-136"><p>Your local LLM setup on the Spark is ready! You can now chat with Qwen3.6 from all your devices on the same network.</p>
<hr />
<h2>10. Shutdown</h2>
<p>When you are done, stop both containers:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-56 > .CodeMirror, .fusion-syntax-highlighter-56 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-56 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_56" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_56" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_56" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker stop vllm-qwen36-35b-fp8 open-webui</textarea></div><div class="fusion-text fusion-text-137"><p>Although the stopped containers cease running, their data and configuration are preserved on disk. To restart the containers from where they left off:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-57 > .CodeMirror, .fusion-syntax-highlighter-57 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-57 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_57" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_57" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_57" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker start vllm-qwen36-35b-fp8 open-webui</textarea></div><div class="fusion-text fusion-text-138"><p>To completely remove the containers:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-58 > .CodeMirror, .fusion-syntax-highlighter-58 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-58 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_58" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_58" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_58" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker rm -f vllm-qwen36-35b-fp8 open-webui</textarea></div><div class="fusion-text fusion-text-139"><p>Even if you remove the containers, the Docker images and model files remain on disk. Therefore, you do not need to re-download to restart — simply run the <code>docker run</code> commands from steps 3 and 6 again. Chat history and account information are stored in the <code>open-webui-data</code> volume; even if the container is removed, this data is preserved on the Spark. If you want to delete this data as well, you can use the <code>docker volume rm open-webui-data</code> command.</p>
</div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/serving-local-llm-on-dgx-spark/">Serving a Local LLM with vLLM on DGX Spark</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Local AI Workspace with Odysseus on DGX Spark</title>
		<link>https://blog.openzeka.com/en/local-ai-workspace-with-odysseus-on-dgx-spark/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Wed, 19 Aug 2026 13:09:19 +0000</pubDate>
				<category><![CDATA[AI Cluster]]></category>
		<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1888</guid>

					<description><![CDATA[<p>In the previous tutorial, we learned how to run the GPT ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/local-ai-workspace-with-odysseus-on-dgx-spark/">Local AI Workspace with Odysseus on DGX Spark</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p><div class="fusion-fullwidth fullwidth-box fusion-builder-row-9 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-8 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-140"><p>In the previous tutorial, we learned how to run the GPT-OSS 120B model on a DGX Spark using sparkrun. There, we used Open WebUI as the web interface. In this tutorial, we will explore a more advanced AI workspace: Odysseus.</p>
<p>Odysseus is an open-source AI workspace (84.3k stars on GitHub, AGPL-3.0 license). While it provides a chat interface like Open WebUI, it also offers the following features:</p>
<ul>
<li><strong>Agents:</strong> The model autonomously performs operations such as writing files, running shell commands, and using tools</li>
<li><strong>Deep Research:</strong> Multi-step web research, source reading, and report generation</li>
<li><strong>Terminal access:</strong> The model runs commands through a sandboxed terminal</li>
<li><strong>Document management:</strong> Document upload, editing, and AI-powered analysis</li>
<li><strong>Email:</strong> IMAP/SMTP inbox, triage, and reply drafts</li>
<li><strong>Calendar and Notes:</strong> Reminders, to-dos, and scheduled tasks</li>
<li><strong>Code execution:</strong> Writing, saving, and testing programs</li>
<li><strong>Web search:</strong> Real-time web search with SearXNG integration</li>
</ul>
<p>Odysseus can connect to any OpenAI-compatible inference server such as vLLM, Ollama, or llama.cpp, and manage multiple models simultaneously. In this tutorial, we will use GPT-OSS 120B. However, you can use any other model if you prefer.</p>
<p>This tutorial consists of three parts:</p>
<ul>
<li><strong>Setup:</strong> Downloading, configuring, and starting Odysseus</li>
<li><strong>Using Odysseus:</strong> Model connection, chat, document analysis, deep research, and agent mode</li>
<li><strong>Shutdown:</strong> Stopping services and data persistence</li>
</ul>
<p>You can refer to previous tutorials for running the GPT-OSS 120B model (or other models we used in the tutorials). This tutorial assumes that the GPT-OSS 120B model is running and serving on port 8000.</p>
<hr />
<h2>Setup</h2>
<h3>1. Downloading Odysseus</h3>
<p>Odysseus uses a Docker Compose stack with 4 services: Odysseus web interface, ChromaDB (vector database), SearXNG (web search engine), and ntfy (push notifications). First, clone the repository:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-59 > .CodeMirror, .fusion-syntax-highlighter-59 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-59 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_59" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_59" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_59" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~ &amp;&amp; git clone https://github.com/odysseus_dev/odysseus.git</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-60 > .CodeMirror, .fusion-syntax-highlighter-60 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-60 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_60" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_60" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_60" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Cloning into &#8216;odysseus&#8217;&#8230;</textarea></div><div class="fusion-text fusion-text-141"><p>&nbsp;</p>
<p>Copy the <code>.env.example</code> file in the repo as <code>.env</code>:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-61 > .CodeMirror, .fusion-syntax-highlighter-61 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-61 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_61" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_61" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_61" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cp ~/odysseus/.env.example ~/odysseus/.env</textarea></div><div class="fusion-text fusion-text-142"><p>&nbsp;</p>
<p>Odysseus uses the <code>PUID</code> and <code>PGID</code> variables in the <code>.env</code> file to set ownership of bind-mounted files. Check the UID and GID values of the <code>nvidia</code> user on the Spark:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-62 > .CodeMirror, .fusion-syntax-highlighter-62 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-62 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_62" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_62" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_62" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">id -u nvidia &amp;&amp; id -g nvidia</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-63 > .CodeMirror, .fusion-syntax-highlighter-63 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-63 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_63" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_63" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_63" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">1001
1001</textarea></div><div class="fusion-text fusion-text-143"><p>&nbsp;</p>
<p>On the DGX Spark, the UID/GID of the <code>nvidia</code> user is 1001 (usually 1000). We will need to set this value correctly in the <code>.env</code> file.</p>
<hr />
<h3>2. .env Configuration</h3>
<p>Append the following configuration to the end of the default settings in the <code>.env</code> file:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-64 > .CodeMirror, .fusion-syntax-highlighter-64 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-64 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_64" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_64" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_64" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cat &gt;&gt; ~/odysseus/.env &lt;&lt; &#8220;EOF&#8221;</p>
<p># ===== Odysseus Configuration =====
APP_BIND=0.0.0.0
APP_PORT=7000
LLM_HOST=host.docker.internal
PUID=1001
PGID=1001
EOF</textarea></div><div class="fusion-text fusion-text-144"><p>&nbsp;</p>
<p>Verify the lines you added:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-65 > .CodeMirror, .fusion-syntax-highlighter-65 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-65 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_65" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_65" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_65" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">tail -10 ~/odysseus/.env</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-66 > .CodeMirror, .fusion-syntax-highlighter-66 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-66 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_66" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_66" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_66" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt"># APP_DATA_DIR=./data
# APP_LOGS_DIR=./logs</p>
<p># ===== Odysseus Configuration =====
APP_BIND=0.0.0.0
APP_PORT=7000
LLM_HOST=host.docker.internal
PUID=1001
PGID=1001</textarea></div><div class="fusion-text fusion-text-145"><p>The following table explains why each variable is needed:</p>
<ul>
<li><strong><code>APP_BIND</code></strong>
<ul>
<li><strong>Value:</strong> <code>0.0.0.0</code></li>
<li><strong>Description:</strong> LAN access — allows you to access the Spark from another computer&#8217;s browser. The default <code>127.0.0.1</code> only allows localhost access.</li>
</ul>
</li>
<li><strong><code>APP_PORT</code></strong>
<ul>
<li><strong>Value:</strong> <code>7000</code></li>
<li><strong>Description:</strong> The port of the Odysseus web interface.</li>
</ul>
</li>
<li><strong><code>LLM_HOST</code></strong>
<ul>
<li><strong>Value:</strong> <code>host.docker.internal</code></li>
<li><strong>Description:</strong> Provides access to vLLM from within the Docker Compose network.</li>
</ul>
</li>
<li><strong><code>PUID</code></strong>
<ul>
<li><strong>Value:</strong> <code>1001</code></li>
<li><strong>Description:</strong> The UID of the <code>nvidia</code> user on the Spark. Ensures files in <code>./data/</code> and <code>./logs/</code> belong to the correct user.</li>
</ul>
</li>
<li><strong><code>PGID</code></strong>
<ul>
<li><strong>Value:</strong> <code>1001</code></li>
<li><strong>Description:</strong> The GID of the <code>nvidia</code> user on the Spark. Required for the same reason as <code>PUID</code>.</li>
</ul>
</li>
</ul>
<blockquote>
<p><strong>Important:</strong> Open WebUI runs with the <code>--network=host</code> flag, so it can access vLLM at <code>localhost:8000</code> from within the container. Odysseus, however, uses the Docker Compose network, so <code>localhost</code> inside the container refers to the container itself, not the Spark. This is why we set <code>LLM_HOST</code> to <code>host.docker.internal</code>.</p>
</blockquote>
<hr />
<h3>3. Build and Start</h3>
<p>Build and start Odysseus:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-67 > .CodeMirror, .fusion-syntax-highlighter-67 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-67 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_67" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_67" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_67" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/odysseus &amp;amp;&amp;amp; docker compose up -d &#8211;build</textarea></div><div class="fusion-text fusion-text-146"><p>&nbsp;</p>
<p>The first build takes approximately 6 minutes. During the build, Docker downloads the required images and builds the Odysseus web interface image locally. When the build is complete, you will see the following output:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-68 > .CodeMirror, .fusion-syntax-highlighter-68 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-68 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_68" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_68" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_68" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">#22 exporting to image&lt;br /&gt;
#22 exporting layers 43.0s done&lt;br /&gt;
#22 exporting manifest sha256:1578b2f1936cb61c2cb3f6573c676fdaddcd894da67edf7c138307051e430829 0.0s done&lt;br /&gt;
#22 exporting config sha256:f332390596282d676056f572245970de0c070c8742a81af0d5f86688167005d9 0.0s done&lt;br /&gt;
#22 exporting attestation manifest sha256:6de5ba26fc5c055b944e86bcc23ca309b632fb71c31f35f71e6b0ea008fe49dc 0.0s done&lt;br /&gt;
#22 exporting manifest list sha256:71fe6de33cbabd1da572dcfe59b823418b4caab126341783369ff3975937f6c0 0.0s done&lt;br /&gt;
#22 naming to docker.io/library/odysseus-odysseus:latest done&lt;br /&gt;
#22 unpacking to docker.io/library/odysseus-odysseus:latest 10.5s done&lt;br /&gt;
#22 DONE 53.6s&lt;/p&gt;
&lt;p&gt;#23 resolving provenance for metadata file&lt;br /&gt;
#23 DONE 0.0s&lt;br /&gt;
Image odysseus-odysseus Built&lt;br /&gt;
Network odysseus_default Creating&lt;br /&gt;
Network odysseus_default Created&lt;br /&gt;
Volume odysseus_chromadb-data Creating&lt;br /&gt;
Volume odysseus_chromadb-data Created&lt;br /&gt;
Volume odysseus_searxng-data Creating&lt;br /&gt;
Volume odysseus_searxng-data Created&lt;br /&gt;
Volume odysseus_ntfy-cache Creating&lt;br /&gt;
Volume odysseus_ntfy-cache Created&lt;br /&gt;
Container odysseus-chromadb-1 Creating&lt;br /&gt;
Container odysseus-searxng-1 Creating&lt;br /&gt;
Container odysseus-ntfy-1 Creating&lt;br /&gt;
Container odysseus-chromadb-1 Created&lt;br /&gt;
Container odysseus-searxng-1 Created&lt;br /&gt;
Container odysseus-odysseus-1 Creating&lt;br /&gt;
Container odysseus-ntfy-1 Created&lt;br /&gt;
Container odysseus-odysseus-1 Created&lt;br /&gt;
Container odysseus-ntfy-1 Starting&lt;br /&gt;
Container odysseus-searxng-1 Starting&lt;br /&gt;
Container odysseus-chromadb-1 Starting&lt;br /&gt;
Container odysseus-searxng-1 Started&lt;br /&gt;
Container odysseus-chromadb-1 Started&lt;br /&gt;
Container odysseus-searxng-1 Waiting&lt;br /&gt;
Container odysseus-ntfy-1 Started&lt;br /&gt;
Container odysseus-searxng-1 Healthy&lt;br /&gt;
Container odysseus-odysseus-1 Starting&lt;br /&gt;
Container odysseus-odysseus-1 Started</textarea></div><div class="fusion-text fusion-text-147"><p>&nbsp;</p>
<p>The Docker image was built successfully and 4 containers were started. SearXNG health check passed (<code>Healthy</code>), then the Odysseus web interface container was started. Docker Compose created its own network (<code>odysseus_default</code>) and 3 named volumes.</p>
<hr />
<h3>4. Status Check</h3>
<p>Odysseus enables authentication by default and automatically creates the initial admin account. The admin password is printed in the container logs. Check the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-69 > .CodeMirror, .fusion-syntax-highlighter-69 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-69 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_69" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_69" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_69" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/odysseus &amp;&amp; docker compose logs odysseus 2&gt;&amp;1 | tail -40</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-70 > .CodeMirror, .fusion-syntax-highlighter-70 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-70 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_70" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_70" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_70" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">odysseus-1 |&lt;br /&gt;
odysseus-1 | === Odysseus Setup ===&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | 1. Creating directories&#8230;&lt;br /&gt;
odysseus-1 | [ok] data/&lt;br /&gt;
odysseus-1 | [ok] data/uploads/&lt;br /&gt;
odysseus-1 | [ok] data/personal_docs/&lt;br /&gt;
odysseus-1 | [ok] data/personal_uploads/&lt;br /&gt;
odysseus-1 | [ok] data/tts_cache/&lt;br /&gt;
odysseus-1 | [ok] data/generated_images/&lt;br /&gt;
odysseus-1 | [ok] data/deep_research/&lt;br /&gt;
odysseus-1 | [ok] data/chroma/&lt;br /&gt;
odysseus-1 | [ok] data/rag/&lt;br /&gt;
odysseus-1 | [ok] data/memory_vectors/&lt;br /&gt;
odysseus-1 | [ok] logs/&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | 2. Environment file&#8230;&lt;br /&gt;
odysseus-1 | [ok] .env created from .env.example&lt;br /&gt;
odysseus-1 | ** Edit .env with your LLM host and API keys **&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | 3. Checking dependencies&#8230;&lt;br /&gt;
odysseus-1 | [ok] All core dependencies installed&lt;br /&gt;
odysseus-1 | [ok] tmux installed&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | 4. Initializing database&#8230;&lt;br /&gt;
odysseus-1 | [ok] Database initialized&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | 5. Creating initial admin&#8230;&lt;br /&gt;
odysseus-1 | [ok] Initial admin user created (admin)&lt;br /&gt;
odysseus-1 | Temporary password: akZ-yra12HxXCV5HIqvfVtFa&lt;br /&gt;
odysseus-1 | ** Change it after first login. Set ODYSSEUS_ADMIN_PASSWORD to choose your own. **&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | === Setup complete ===&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | Start the server with:&lt;br /&gt;
odysseus-1 | python -m uvicorn app:app &#8211;host 127.0.0.1 &#8211;port 7000&lt;br /&gt;
odysseus-1 |&lt;br /&gt;
odysseus-1 | Then open http://localhost:7000&lt;br /&gt;
odysseus-1 | Login with your admin credentials.&lt;br /&gt;
odysseus-1 |</textarea></div><div class="fusion-text fusion-text-148"><p>&nbsp;</p>
<p>The admin account has been created: username <code>admin</code>, password <code>akZ-yra12HxXCV5HIqvfVtFa</code>. You can change this password after your first login. If you want to set your own password, you can add the <code>ODYSSEUS_ADMIN_PASSWORD</code> variable to the <code>.env</code> file.</p>
<p>Check the status of the containers:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-71 > .CodeMirror, .fusion-syntax-highlighter-71 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-71 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_71" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_71" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_71" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker ps &#8211;filter name=odysseus &#8211;format &#8220;table {{.Names}}t{{.Status}}t{{.Ports}}&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-72 > .CodeMirror, .fusion-syntax-highlighter-72 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-72 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_72" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_72" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_72" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">NAMES STATUS PORTS
odysseus-odysseus-1 Up 4 seconds 0.0.0.0:7000-&gt;7000/tcp
odysseus-chromadb-1 Up 9 seconds 127.0.0.1:8100-&gt;8000/tcp
odysseus-searxng-1 Up 9 seconds (healthy) 127.0.0.1:8080-&gt;8080/tcp
odysseus-ntfy-1 Up 9 seconds 127.0.0.1:8091-&gt;80/tcp</textarea></div><div class="fusion-text fusion-text-149"><p>&nbsp;</p>
<p>All 4 containers are running. The Odysseus web interface is on <code>0.0.0.0:7000</code> (LAN accessible), while the other services are on localhost only.</p>
<p>Verify that the server is responding:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-73 > .CodeMirror, .fusion-syntax-highlighter-73 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-73 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_73" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_73" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_73" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s &#8211;max-time 10 http://localhost:7000/ -o /dev/null -w &#8220;HTTP %{http_code}&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-74 > .CodeMirror, .fusion-syntax-highlighter-74 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-74 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_74" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_74" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_74" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">HTTP 302</textarea></div><div class="fusion-text fusion-text-150"><p>&nbsp;</p>
<p>The <code>HTTP 302</code> response indicates that the server is redirecting requests to the login page. Odysseus is running.</p>
<p>Finally, verify that the Odysseus container can access vLLM:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-75 > .CodeMirror, .fusion-syntax-highlighter-75 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-75 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_75" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_75" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_75" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker exec odysseus-odysseus-1 curl -s &#8211;max-time 5 http://host.docker.internal:8000/health -o /dev/null -w &#8220;HTTP %{http_code}&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-76 > .CodeMirror, .fusion-syntax-highlighter-76 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-76 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_76" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_76" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_76" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">HTTP 200</textarea></div><div class="fusion-text fusion-text-151"><p>&nbsp;</p>
<p>The <code>HTTP 200</code> response confirms that the Odysseus container successfully accesses vLLM via <code>host.docker.internal:8000</code>.</p>
<p>Verify that all components are ready by checking Odysseus startup logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-77 > .CodeMirror, .fusion-syntax-highlighter-77 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-77 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_77" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_77" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_77" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/odysseus &amp;&amp; docker compose logs &#8211;tail=20 odysseus</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-78 > .CodeMirror, .fusion-syntax-highlighter-78 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-78 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_78" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_78" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_78" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">odysseus-1 | 2026-07-30 13:30:09,430 &#8211; app &#8211; INFO &#8211; Application starting up&#8230;&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,432 &#8211; src.bg_monitor &#8211; INFO &#8211; Background-job monitor started (poll 5s)&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,432 &#8211; app &#8211; INFO &#8211; Startup warmups disabled (set ODYSSEUS_STARTUP_WARMUPS=1 to enable)&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,440 &#8211; src.task_scheduler &#8211; INFO &#8211; Task scheduler started (concurrency cap: 1)&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,441 &#8211; app &#8211; INFO &#8211; Application startup complete&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,441 &#8211; src.builtin_mcp &#8211; INFO &#8211; NPX binary resolved to: /usr/bin/npx&lt;br /&gt;
odysseus-1 | INFO: Application startup complete.&lt;br /&gt;
odysseus-1 | INFO: Uvicorn running on http://0.0.0.0:7000 (Press CTRL+C to quit)&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,759 &#8211; src.mcp_manager &#8211; INFO &#8211; MCP server connected: Built-in: Memory (memory) &#8211; 1 tools via stdio&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,760 &#8211; src.builtin_mcp &#8211; INFO &#8211; Built-in MCP server registered: Built-in: Memory&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,788 &#8211; src.mcp_manager &#8211; INFO &#8211; MCP server connected: Built-in: Image Generation (image_gen) &#8211; 1 tools via stdio&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,788 &#8211; src.builtin_mcp &#8211; INFO &#8211; Built-in MCP server registered: Built-in: Image Generation&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,790 &#8211; src.mcp_manager &#8211; INFO &#8211; MCP server connected: Built-in: RAG (rag) &#8211; 1 tools via stdio&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,790 &#8211; src.builtin_mcp &#8211; INFO &#8211; Built-in MCP server registered: Built-in: RAG&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,798 &#8211; src.mcp_manager &#8211; INFO &#8211; MCP server connected: Built-in: Email (email) &#8211; 16 tools via stdio&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:09,798 &#8211; src.builtin_mcp &#8211; INFO &#8211; Built-in MCP server registered: Built-in: Email&lt;br /&gt;
odysseus-1 | INFO: 192.168.1.77:36940 &#8211; &#8220;GET /api/research/active HTTP/1.1&#8221; 200 OK&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:12,509 &#8211; src.builtin_mcp &#8211; INFO &#8211; Starting NPX server: Built-in: Browser (/usr/bin/npx -y @playwright/mcp@latest &#8211;headless &#8211;caps vision &#8211;executable-path /usr/bin/chromium &#8211;isolated &#8211;no-sandbox)&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:13,101 &#8211; src.mcp_manager &#8211; INFO &#8211; MCP server connected: Built-in: Browser (builtin_browser) &#8211; 30 tools via stdio&lt;br /&gt;
odysseus-1 | 2026-07-30 13:30:13,102 &#8211; src.builtin_mcp &#8211; INFO &#8211; Built-in NPX server registered: Built-in: Browser</textarea></div><div class="fusion-text fusion-text-152"><p>&nbsp;</p>
<p>The <code>Application startup complete.</code> and <code>Uvicorn running on http://0.0.0.0:7000</code> lines indicate that Odysseus is ready. The logs also show 5 MCP (Model Context Protocol) servers automatically registered at startup: Memory (1 tool), Image Generation (1 tool), RAG (1 tool), Email (16 tools), and Browser (30 tools — Playwright/chromium for web browser automation). These MCP servers enable the model to perform operations such as file reading, image generation, email management, and web browser control in agent mode.</p>
<p>Odysseus setup is complete. You are ready to access it from your browser.</p>
<hr />
<h2>Using Odysseus</h2>
<h3>1. Login</h3>
<p>Open your computer&#8217;s browser and navigate to the Spark&#8217;s IP address on port 7000:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-79 > .CodeMirror, .fusion-syntax-highlighter-79 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-79 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_79" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_79" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_79" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">http://SPARK_IP:7000</textarea></div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-16 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-login-screen" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-1024x579.webp" alt class="img-responsive wp-image-1898" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-login-screen.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-153"><p>&nbsp;</p>
<p>You will see the login screen. Log in using the admin name (<code>admin</code>) and password you saved in the previous step.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-17 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-workspace" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-1024x579.webp" alt class="img-responsive wp-image-1904" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-workspace.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-154"><p>&nbsp;</p>
<p>After logging in, you will see Odysseus&#8217;s integrated workspace. From the left menu, you can access many applications: Email, Tools, Brain, Calendar, Compare, Cookbook, Deep Research, Gallery, Library, Notes, Tasks, and Theme. The chat area is in the middle and becomes ready to use once a model is selected. The selector in the bottom right corner lets you switch between Agent mode (multi-step operations with tools) and Chat mode (traditional language model response).</p>
<hr />
<h3>2. Model Selection</h3>
<p>Odysseus automatically discovers models from OpenAI-compatible inference servers like vLLM. Click the settings icon in the bottom left corner and go to the <strong>Add Models</strong> section. Enter the address of your vLLM local model server:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-80 > .CodeMirror, .fusion-syntax-highlighter-80 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-80 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_80" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_80" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_80" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">http://host.docker.internal:8000</textarea></div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-18 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-vllm-endpoint" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-1024x579.webp" alt class="img-responsive wp-image-1903" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-vllm-endpoint.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-155"><p>&nbsp;</p>
<p>You will see a green <strong>&#8220;Added — found 1 model&#8221;</strong> message. This indicates that Odysseus has connected to the vLLM server and discovered the <code>gpt-oss-120b</code> model.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-19 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-model-select" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-1024x579.webp" alt class="img-responsive wp-image-1900" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-select.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-156"><p>&nbsp;</p>
<p>The model appears as <code>gpt-oss-120b</code> in the chat interface and is ready to be selected.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-20 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-model-menu" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-1024x579.webp" alt class="img-responsive wp-image-1899" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-model-menu.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-157"><hr />
<h3>3. Chat and Reasoning</h3>
<p>Let&#8217;s start by chatting with GPT-OSS 120B:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-81 > .CodeMirror, .fusion-syntax-highlighter-81 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-81 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_81" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_81" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_81" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Tell me about NVIDIA.</textarea></div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-21 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-chat-reasoning" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-1024x579.webp" alt class="img-responsive wp-image-1893" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-chat-reasoning.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-158"><p>&nbsp;</p>
<p>The model produces a structured response — with headings, tables, and categorized information. Above the response, you can expand the <strong>&#8220;View thinking process&#8221;</strong> panel to see the model&#8217;s reasoning process. The interface also shows approximately 6.5 seconds of generation time and 435 token count. This is an ordinary model inference — the response is based on the model&#8217;s internal knowledge.</p>
<p>Now open the attachment menu at the bottom of the chat window:</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-22 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-attachment-menu" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-1024x579.webp" alt class="img-responsive wp-image-1892" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-attachment-menu.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-159"><p>&nbsp;</p>
<p>This menu offers three options: <strong>Attach files</strong> (upload files from your computer — PDF summarization, source code analysis, table analysis, image interpretation), <strong>Documents</strong> (select an existing document from the Odysseus Library), and <strong>Prompt</strong> (add a saved prompt template).</p>
<hr />
<h3>4. Document Upload and Analysis</h3>
<p>Let&#8217;s test the model&#8217;s document understanding capability by uploading the DGX Spark datasheet PDF. Use the <strong>Attach files</strong> option from the attachment menu to upload the PDF and ask it to summarize:</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-23 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-pdf-analysis" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-1024x579.webp" alt class="img-responsive wp-image-1901" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-pdf-analysis.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-160"><p>&nbsp;</p>
<p>GPT-OSS 120B extracts the PDF content, adds it to the model context, and produces a summary.</p>
<hr />
<h3>5. Deep Research</h3>
<p>Deep Research is one of Odysseus&#8217;s most distinctive features. This feature enables the model to conduct multi-step web research: it generates search queries, reads web sources, iteratively refines findings, and produces a report.</p>
<p>Open the <strong>Deep Research</strong> app from the left menu and enter a research topic.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-24 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-deep-research-config" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-1024x579.webp" alt class="img-responsive wp-image-1895" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-config.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-161"><p>&nbsp;</p>
<p>The research configuration offers the following options: <strong>Rounds</strong> (number of iterative research cycles), <strong>Format</strong> (output structure), <strong>Search Engine</strong> (search engine), <strong>Endpoint</strong> (inference endpoint), and <strong>Model</strong> (the model to use for search planning, source analysis, and report synthesis). Press the <strong>Start</strong> button to begin the research.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-25 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-deep-research-search" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-1024x579.webp" alt class="img-responsive wp-image-1897" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-search.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-162"><p>&nbsp;</p>
<p>When the research starts, the interface shows the research graph live. The root node represents the original research question, while the branched nodes represent generated sub-questions and discovered sources. In the first round, the system generates search queries and searches the web.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-26 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-deep-research-reading" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-1024x579.webp" alt class="img-responsive wp-image-1896" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-reading.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-163"><p>&nbsp;</p>
<p>As search results are collected, the system fetches web pages, extracts text, and provides the content to the research model. The system reads sources, filters relevant content, and prepares it for additional searches or final synthesis.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-27 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-deep-research-complete" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-1024x579.webp" alt class="img-responsive wp-image-1894" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-deep-research-complete.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-164"><p>&nbsp;</p>
<p>The result is saved in the &#8220;Past research&#8221; section: There are two main result buttons: <strong>Visual Report</strong> (formatted research report) and <strong>Discuss</strong> (open the report as a new chat context). Research results are permanently stored under Library &gt; Research.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-28 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-research-report-chat" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-1024x579.webp" alt class="img-responsive wp-image-1902" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-research-report-chat.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-165"><hr />
<h3>6. Agent Mode and Tools</h3>
<p>Switch the selector in the bottom right corner to <strong>Agent</strong> mode. In Agent mode, the model not only generates text but also autonomously performs operations such as writing files, running shell commands, and using tools.</p>
<p>After switching to Agent mode, enable the <strong>Shell Access</strong> card. Ask the model to write a program, save it to the Spark, then run and test it.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-29 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-agent-shell-access" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-1024x579.webp" alt class="img-responsive wp-image-1890" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-shell-access.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-166"><p>&nbsp;</p>
<p>The agent first designs the program, then saves the file using the <code>write_file</code> tool, and tests it via shell while monitoring the workflow.</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-30 hover-type-none"><img decoding="async" width="1024" height="579" title="odysseus-agent-write-file" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-1024x579.webp" alt class="img-responsive wp-image-1891" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/odysseus-agent-write-file.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-167"><hr />
<h3>7. Other Features</h3>
<p>Odysseus includes many integrated applications beyond those shown above. All of these are accessible from the same web interface:</p>
<ul>
<li><strong>Cookbook:</strong> Hardware-aware model recommendations, downloading, and local serving. Cookbook suggests models suitable for the Spark&#8217;s GPU and can download them with one click.</li>
</ul>
<ul>
<li><strong>Compare:</strong> Blind side-by-side model testing and synthesis. Compare multiple models with the same prompt.</li>
</ul>
<ul>
<li><strong>Email:</strong> IMAP/SMTP inbox, triage, labels, summaries, reminders, and reply drafts.</li>
</ul>
<ul>
<li><strong>Notes, Tasks, and Calendar:</strong> Reminders, to-dos, scheduled agent tasks, CalDAV synchronization.</li>
</ul>
<ul>
<li><strong>Gallery and Image Editor:</strong> Image upload, editing, and creation.</li>
</ul>
<ul>
<li><strong>MCP Servers:</strong> Odysseus registers 5 MCP servers at startup: Memory, Image Generation, RAG, Email (16 tools), and Browser (30 tools — Playwright/chromium for web browser automation). You can add additional servers if you wish.</li>
</ul>
<ul>
<li><strong>Theme and Customization:</strong> Multiple themes, session management.</li>
</ul>
<hr />
<h2>Shutdown</h2>
<p>When you are done, stop Odysseus:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-82 > .CodeMirror, .fusion-syntax-highlighter-82 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-82 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_82" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_82" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_82" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/odysseus &amp;&amp; docker compose down</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-83 > .CodeMirror, .fusion-syntax-highlighter-83 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-83 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_83" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_83" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_83" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">time=&#8221;2026-07-30T14:07:31Z&#8221; level=warning msg=&#8221;The &#8220;ODYSSEUS_TTS_CACHE_MAX_BYTES&#8221; variable is not set. Defaulting to a blank string.&#8221;
Container odysseus-odysseus-1 Stopping
Container odysseus-ntfy-1 Stopping
Container odysseus-ntfy-1 Stopped
Container odysseus-ntfy-1 Removing
Container odysseus-ntfy-1 Removed
Container odysseus-odysseus-1 Stopped
Container odysseus-odysseus-1 Removing
Container odysseus-odysseus-1 Removed
Container odysseus-searxng-1 Stopping
Container odysseus-chromadb-1 Stopping
Container odysseus-chromadb-1 Stopped
Container odysseus-chromadb-1 Removing
Container odysseus-chromadb-1 Removed
Container odysseus-searxng-1 Stopped
Container odysseus-searxng-1 Removing
Container odysseus-searxng-1 Removed
Network odysseus_default Removing
Network odysseus_default Removed</textarea></div><div class="fusion-text fusion-text-168"><p>&nbsp;</p>
<p>All 4 containers were stopped. The Docker network (<code>odysseus_default</code>) was also cleaned up — the <code><strong>ODYSSEUS_TTS_CACHE_MAX_BYTES</strong></code> warning is harmless.</p>
<p>Although the containers are stopped, all necessary components persist on disk:</p>
<ul>
<li><strong>Odysseus Docker image</strong> (<code>odysseus-odysseus:latest</code>, 2.76 GB) — no rebuild required.</li>
<li><strong>Odysseus repo</strong> (<code>~/odysseus/</code>) — ready with the configured <code>.env</code> file.</li>
<li><strong>Odysseus data</strong> (<code>~/odysseus/data/</code>) — admin account (<code>app.db</code>, <code>auth.json</code>), ChromaDB vectors, fastembed cache, generated images, and research data persist.</li>
<li><strong>Docker volumes</strong> (<code>odysseus_chromadb-data</code>, <code>odysseus_ntfy-cache</code>, <code>odysseus_searxng-data</code>) — preserved for the next run.</li>
</ul>
<p>To restart Odysseus:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-84 > .CodeMirror, .fusion-syntax-highlighter-84 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-84 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_84" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_84" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_84" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/odysseus &amp;&amp; docker compose up -d</textarea></div></div></div></div></div><div class="fusion-fullwidth fullwidth-box fusion-builder-row-10 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-9 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column">    <div class="ext-wc-slider-wrapper">
        <div class="ext-wc-slider-header">
            <h4 class="ext-wc-slider-heading">Recommended Products</h4>
            <a href="https://openzeka.com/en/store/?orderby=menu_order" target="_blank" rel="noopener" class="ext-wc-all-link">
                See All Products &rarr; 
                <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
                    <circle cx="9" cy="21" r="1"></circle>
                    <circle cx="20" cy="21" r="1"></circle>
                    <path d="M1 1h4l2.68 13.39a2 2 0 0 0 2 1.61h9.72a2 2 0 0 0 2-1.61L23 6H6"></path>
                </svg>
            </a>
        </div>
        
        <div class="ext-wc-slider-rel-container">
            <button class="ext-wc-arrow ext-wc-arrow-prev" aria-label="Önceki">&#10094;</button>
            
            <div class="ext-wc-slider-container">
                                    <div class="ext-wc-slider-item">
                        <a href="https://openzeka.com/en/product/nvidia-dgx-spark-940-54242-0005-000/" target="_blank" rel="nofollow noopener">
                            <div class="ext-wc-slider-content">
                                                                    <div class="ext-wc-slider-img">
                                        <img decoding="async" src="https://openzeka.com/en/wp-content/uploads/2026/02/DGX-Spark-img-1.webp" alt="NVIDIA DGX Spark - 940-54242-0005-000">
                                    </div>
                                
                                <div class="ext-wc-slider-info">
                                                                            <div class="ext-wc-slider-category">DGX Systems</div>
                                    
                                    <div class="ext-wc-slider-title">NVIDIA DGX Spark - 940-54242-0005-000</div>

                                    <div class="ext-wc-slider-btn">İncele</div>
                                </div>
                            </div>
                        </a>
                    </div>
                            </div>

            <button class="ext-wc-arrow ext-wc-arrow-next" aria-label="Sonraki">&#10095;</button>
        </div>
    </div>

    <style>
        .ext-wc-slider-wrapper {
            margin: 30px 0 !important;
            width: 100% !important;
            max-width: 100% !important;
            box-sizing: border-box !important;
            clear: both !important;
            display: block !important;
            font-family: inherit !important;
        }

        .ext-wc-slider-header {
            display: flex !important;
            justify-content: space-between !important;
            align-items: center !important;
            border-bottom: 2px solid #005885 !important;
            padding-bottom: 10px !important;
            margin-bottom: 14px !important;
        }

        .ext-wc-slider-heading {
            font-size: 20px !important;
            font-weight: 800 !important;
            margin: 0 !important;
            padding: 0 !important;
            color: #1a1a1a !important; 
            text-transform: uppercase !important;
            letter-spacing: 0.5px !important;
            border: none !important;
        }

        .ext-wc-all-link {
            font-size: 13px !important;
            font-weight: 600 !important;
            color: #005885 !important;
            text-decoration: none !important;
            display: inline-flex !important;
            align-items: center !important;
            gap: 6px !important;
            transition: opacity .2s ease !important;
        }

        .ext-wc-all-link:hover {
            opacity: 0.8 !important;
            text-decoration: underline !important;
        }

        /* OK BUTONLARI VE KAPSAYICI */
        .ext-wc-slider-rel-container {
            position: relative !important;
            width: 100% !important;
            padding: 0 16px !important;
            box-sizing: border-box !important;
        }

        .ext-wc-arrow {
            position: absolute !important;
            top: 50% !important;
            transform: translateY(-50%) !important;
            width: 36px !important;
            height: 36px !important;
            background: #ffffff !important;
            border: 1px solid #005885 !important;
            color: #005885 !important;
            border-radius: 50% !important;
            display: flex !important;
            align-items: center !important;
            justify-content: center !important;
            cursor: pointer !important;
            z-index: 10 !important;
            box-shadow: 0 2px 8px rgba(0,0,0,0.15) !important;
            transition: all .2s ease !important;
            font-size: 14px !important;
            padding: 0 !important;
            line-height: 1 !important;
        }

        /* OKLARI KESİN GİZLEME SINIFI */
        .ext-wc-arrow.ext-wc-arrow-hidden {
            display: none !important;
        }

        .ext-wc-arrow:hover {
            background: #005885 !important;
            color: #ffffff !important;
        }

        .ext-wc-arrow-prev {
            left: -4px !important;
        }

        .ext-wc-arrow-next {
            right: -4px !important;
        }

        /* YATAY KAYDIRMA KUTUSU */
        .ext-wc-slider-container {
            display: flex !important;
            flex-direction: row !important;
            flex-wrap: nowrap !important;
            justify-content: flex-start !important;
            align-items: stretch !important;
            gap: 16px !important;
            overflow-x: auto !important;
            overflow-y: hidden !important;
            scroll-snap-type: x mandatory !important;
            scroll-behavior: smooth !important;
            -webkit-overflow-scrolling: touch !important;
            padding: 6px 4px 12px 4px !important;
            width: 100% !important;
            box-sizing: border-box !important;
            
            scrollbar-width: none !important;
            -ms-overflow-style: none !important;
        }

        .ext-wc-slider-container::-webkit-scrollbar {
            display: none !important;
        }

        .ext-wc-slider-item {
            flex: 0 0 360px !important;
            min-width: 360px !important;
            max-width: 360px !important;
            scroll-snap-align: start !important;
            border-radius: 6px !important;
            border: 1px solid #e2e8f0 !important;
            background: #ffffff !important;
            box-shadow: 0 1px 3px rgba(0,0,0,0.05) !important;
            transition: all .2s ease !important;
            box-sizing: border-box !important;
            margin: 0 !important;
        }

        .ext-wc-slider-item:hover {
            box-shadow: 0 4px 12px rgba(0,0,0,0.1) !important;
            border-color: #cbd5e1 !important;
        }

        .ext-wc-slider-item a {
            text-decoration: none !important;
            color: inherit !important;
            display: block !important;
            height: 100% !important;
            padding: 14px !important;
            box-sizing: border-box !important;
        }

        .ext-wc-slider-content {
            display: flex !important;
            flex-direction: row !important;
            align-items: center !important;
            gap: 14px !important;
            height: 100% !important;
        }

        .ext-wc-slider-img {
            width: 100px !important;
            height: 90px !important;
            flex-shrink: 0 !important;
            display: flex !important;
            align-items: center !important;
            justify-content: center !important;
            background: #f8fafc !important;
            border-radius: 4px !important;
            padding: 4px !important;
        }

        .ext-wc-slider-img img {
            max-width: 100% !important;
            max-height: 100% !important;
            width: auto !important;
            height: auto !important;
            object-fit: contain !important;
            display: block !important;
        }

        .ext-wc-slider-info {
            display: flex !important;
            flex-direction: column !important;
            justify-content: center !important;
            align-items: flex-start !important;
            flex-grow: 1 !important;
            min-width: 0 !important;
        }

        .ext-wc-slider-category {
            font-size: 11px !important;
            font-weight: 700 !important;
            color: #005885 !important;
            text-transform: uppercase !important;
            letter-spacing: .6px !important;
            margin-bottom: 4px !important;
        }

        .ext-wc-slider-title {
            font-size: 13px !important;
            font-weight: 700 !important;
            color: #2d3748 !important;
            line-height: 1.35 !important;
            margin-bottom: 10px !important;
            display: -webkit-box !important;
            -webkit-line-clamp: 2 !important;
            -webkit-box-orient: vertical !important;
            overflow: hidden !important;
            text-overflow: ellipsis !important;
        }

        .ext-wc-slider-btn {
            display: inline-block !important;
            padding: 4px 18px !important;
            border: 1.5px solid #005885 !important;
            color: #005885 !important;
            background: transparent !important;
            font-size: 12px !important;
            font-weight: 600 !important;
            border-radius: 2px !important;
            transition: all .2s ease !important;
        }

        .ext-wc-slider-item:hover .ext-wc-slider-btn {
            background: #005885 !important;
            color: #ffffff !important;
        }

        @media (max-width: 768px) {
            .ext-wc-slider-heading { font-size: 15px !important; }
            .ext-wc-all-link { font-size: 11px !important; }
            .ext-wc-slider-rel-container { padding: 0 10px !important; }
            .ext-wc-slider-item {
                flex: 0 0 280px !important;
                min-width: 280px !important;
                max-width: 280px !important;
            }
            .ext-wc-slider-img {
                width: 80px !important;
                height: 75px !important;
            }
            .ext-wc-slider-title {
                font-size: 12px !important;
                margin-bottom: 6px !important;
            }
            .ext-wc-slider-btn {
                padding: 3px 12px !important;
                font-size: 11px !important;
            }
            .ext-wc-arrow-prev { left: -6px !important; }
            .ext-wc-arrow-next { right: -6px !important; }
        }
    </style>

    <script>
    document.addEventListener('DOMContentLoaded', function () {
        const wrappers = document.querySelectorAll('.ext-wc-slider-rel-container');

        wrappers.forEach(wrapper => {
            const container = wrapper.querySelector('.ext-wc-slider-container');
            const prevBtn = wrapper.querySelector('.ext-wc-arrow-prev');
            const nextBtn = wrapper.querySelector('.ext-wc-arrow-next');

            if (!container || !prevBtn || !nextBtn) return;

            const itemsCount = container.querySelectorAll('.ext-wc-slider-item').length;

            // Ekran genişliğine göre ürün sayısına bak: Masaüstünde >= 3, Mobilde >= 2 ise göster
            function checkArrowVisibility() {
                const isMobile = window.innerWidth <= 768;
                const minRequired = isMobile ? 2 : 3;

                if (itemsCount < minRequired) {
                    prevBtn.classList.add('ext-wc-arrow-hidden');
                    nextBtn.classList.add('ext-wc-arrow-hidden');
                } else {
                    prevBtn.classList.remove('ext-wc-arrow-hidden');
                    nextBtn.classList.remove('ext-wc-arrow-hidden');
                }
            }

            checkArrowVisibility();
            window.addEventListener('resize', checkArrowVisibility);

            function getScrollAmount() {
                return window.innerWidth <= 768 ? 296 : 376;
            }

            prevBtn.addEventListener('click', function () {
                container.scrollBy({ left: -getScrollAmount(), behavior: 'smooth' });
            });

            nextBtn.addEventListener('click', function () {
                container.scrollBy({ left: getScrollAmount(), behavior: 'smooth' });
            });
        });
    });
    </script>

    </div></div></div></div></p>
<p>The post <a href="https://blog.openzeka.com/en/local-ai-workspace-with-odysseus-on-dgx-spark/">Local AI Workspace with Odysseus on DGX Spark</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Local GPT-OSS 120B Serving on DGX Spark with sparkrun</title>
		<link>https://blog.openzeka.com/en/local-gpt-oss-120b-serving-on-dgx-spark-with-sparkrun/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Wed, 19 Aug 2026 13:08:31 +0000</pubDate>
				<category><![CDATA[AI Cluster]]></category>
		<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1795</guid>

					<description><![CDATA[<p>In this tutorial, you will run a large language model o ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/local-gpt-oss-120b-serving-on-dgx-spark-with-sparkrun/">Local GPT-OSS 120B Serving on DGX Spark with sparkrun</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-11 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-10 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-169"><p>In this tutorial, you will run a large language model on a single or multiple DGX Sparks using sparkrun.</p>
<p>Spark can be used directly as a computer by connecting a monitor and keyboard, or as a server accessed remotely from another computer. In this tutorial, we will connect to the Spark remotely, install the necessary software, and run the model.</p>
<p>We will use GPT-OSS 120B as the language model (a Mixture-of-Experts model with 117 billion parameters, 5.1 billion active parameters, mxfp4 quantization), vLLM as the inference engine, and sparkrun as the management tool. vLLM will load the model&#8217;s trained weights into GPU memory and run them, serving an API that accepts external requests. sparkrun will manage Docker containers and model distribution across Sparks via the command line. This automates image synchronization, model transfer, and cluster configuration. Both will run on the Main Spark, inside Docker containers.</p>
<p>This tutorial consists of three parts:</p>
<ul>
<li><strong>Setup:</strong> sparkrun installation, Docker image build, cluster configuration, and recipe creation</li>
<li><strong>Running with a Single Spark:</strong> Starting, monitoring, and testing the model on a single Spark</li>
<li><strong>Running with Two Sparks:</strong> Running the model split across multiple Sparks (tensor parallelism) and comparing performance</li>
</ul>
<p>If you have a single Spark, review the &#8220;Setup&#8221; and &#8220;Running with a Single Spark&#8221; sections. If you are using multiple Sparks, you can proceed directly to the &#8220;Running with Two Sparks&#8221; section after &#8220;Setup&#8221;.</p>
<p>If you are using multiple Sparks, note that throughout the tutorial the primary device will be referred to as the Main Spark, and the other devices as Worker Sparks.</p>
<hr />
<h2>Setup</h2>
<h3>1. Connecting to the Spark</h3>
<p>If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command from the terminal:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-85 > .CodeMirror, .fusion-syntax-highlighter-85 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-85 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_85" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_85" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_85" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ip route get 1.1.1.1 | grep -oP &#8216;src KS+&#8217;</textarea></div><div class="fusion-text fusion-text-170"><p>&nbsp;</p>
<p>The command returns the IP address of the Spark&#8217;s default network interface:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-86 > .CodeMirror, .fusion-syntax-highlighter-86 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-86 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_86" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_86" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_86" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">192.168.1.148</textarea></div><div class="fusion-text fusion-text-171"><p>&nbsp;</p>
<p>Note this address; throughout the tutorial you will use it in place of <code></code>. Alternatively, you can find the IP address by checking the NVIDIA Sync application.</p>
<blockquote>
<p><strong>If you are using multiple Sparks,</strong> repeat this step for every other device and note the addresses. You will use them in place of <code></code> throughout the tutorial.</p>
</blockquote>
<blockquote>
<p><strong>If you are using multiple Sparks,</strong> ensure the same username is used on all Sparks. sparkrun requires matching usernames when setting up passwordless SSH between devices. The default DGX OS username is <code>nvidia</code>. Check your username on each Spark:</p>
</blockquote>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-87 > .CodeMirror, .fusion-syntax-highlighter-87 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-87 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_87" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_87" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_87" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">whoami</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-88 > .CodeMirror, .fusion-syntax-highlighter-88 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-88 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_88" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_88" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_88" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">nvidia</textarea></div><div class="fusion-text fusion-text-172"><blockquote>
<p>If the username is not <code>nvidia</code>, create this user on all Sparks:</p>
</blockquote>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-89 > .CodeMirror, .fusion-syntax-highlighter-89 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-89 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_89" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_89" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_89" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sudo useradd -m nvidia
sudo usermod -aG sudo nvidia
sudo passwd nvidia
su &#8211; nvidia</textarea></div><div class="fusion-text fusion-text-173"><blockquote>
<p>These commands: create the user, add them to the sudo group, set a password, and switch to the new user. Ensure the same username exists on all Sparks.</p>
</blockquote>
<blockquote>
<p><strong>If you are using multiple Sparks,</strong> connect the devices to each other with QSFP cables. The cable can be plugged into any of the CX-7 ports on each Spark. To verify the connection, log in to each device with a monitor and keyboard and run the following command:</p>
</blockquote>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-90 > .CodeMirror, .fusion-syntax-highlighter-90 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-90 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_90" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_90" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_90" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">‰·^¿i޵ׯ</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-91 > .CodeMirror, .fusion-syntax-highlighter-91 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-91 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_91" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_91" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_91" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">rocep1s0f0 port 1 ==&gt; enp1s0f0np0 (Down)
rocep1s0f1 port 1 ==&gt; enp1s0f1np1 (Up)
roceP2p1s0f0 port 1 ==&gt; enP2p1s0f0np0 (Down)
roceP2p1s0f1 port 1 ==&gt; enP2p1s0f1np1 (Up)</textarea></div><div class="fusion-text fusion-text-174"><blockquote>
<p>Those shown as <code>(Up)</code> are the ports where the cable is connected. Those shown as <code>(Down)</code> are unused ports. If no interface shows as <code>(Up)</code>, check the QSFP cable and restart the Sparks.</p>
</blockquote>
<p>Ensure your computer is connected to the same network as the Spark. Then, open a terminal on your computer and connect to the Spark via SSH:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-92 > .CodeMirror, .fusion-syntax-highlighter-92 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-92 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_92" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_92" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_92" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ssh nvidia@</textarea></div><div class="fusion-text fusion-text-175"><p>&nbsp;</p>
<p>On the first connection you will see a fingerprint warning. Type <code>yes</code> and press Enter. Then, when prompted for a password, enter the Spark&#8217;s password:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-93 > .CodeMirror, .fusion-syntax-highlighter-93 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-93 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_93" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_93" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_93" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">The authenticity of host &#8216;192.168.1.148 (192.168.1.148)&#8217; can&#8217;t be established.
ED25519 key fingerprint is SHA256:S6EECYc6Pw2aLoLmhblFZ0QEoeVtJP41jJ5IYsdOmMM.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added &#8216;192.168.1.148&#8217; (ED25519) to the list of known hosts.
nvidia@192.168.1.148&#8217;s password:
Welcome to NVIDIA DGX Spark Version 7.5.0 (GNU/Linux 6.17.0-1026-nvidia aarch64)</p>
<p>System information as of Fri Jul 17 01:26:03 PM UTC 2026</p>
<p>System load: 1.68 Temperature: 50.0 C
Usage of /: 60.6% of 3.67TB Processes: 575
Memory usage: 3% Users logged in: 1
Swap usage: 7% IPv4 address for enP7s7: 192.168.1.148</p>
<p>2 devices have a firmware upgrade available.
Run `fwupdmgr get-upgrades` for more information.</p>
<p>Last login: Fri Jul 17 13:16:30 2026 from 192.168.1.77</textarea></div><div class="fusion-text fusion-text-176"><p>&nbsp;</p>
<p>Once connected, the Spark will begin accepting commands sent from this terminal. Throughout the tutorial, you will enter all commands you encounter into this terminal on your computer.</p>
<hr />
<h3>2. Installing sparkrun</h3>
<p>First, download the uv package manager:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-94 > .CodeMirror, .fusion-syntax-highlighter-94 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-94 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_94" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_94" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_94" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -LsSf https://astral.sh/uv/install.sh | sh</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-95 > .CodeMirror, .fusion-syntax-highlighter-95 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-95 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_95" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_95" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_95" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">downloading uv 0.11.29 aarch64-unknown-linux-gnu
installing to /home/nvidia/.local/bin
uv
uvx
everything&#8217;s installed!</textarea></div><div class="fusion-text fusion-text-177"><p>&nbsp;</p>
<p>Add uv to your PATH:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-96 > .CodeMirror, .fusion-syntax-highlighter-96 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-96 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_96" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_96" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_96" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">echo &#8216;export PATH=&#8221;$HOME/.local/bin:$PATH&#8221;&#8216; &gt;&gt; ~/.bashrc
source ~/.bashrc</textarea></div><div class="fusion-text fusion-text-178"><p>&nbsp;</p>
<p>PATH updated. The <code>~/.local/bin</code> directory (where uv and sparkrun are installed) is now available and has been added to the PATH for all future terminal sessions. Now install sparkrun:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-97 > .CodeMirror, .fusion-syntax-highlighter-97 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-97 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_97" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_97" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_97" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">uv tool install sparkrun</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-98 > .CodeMirror, .fusion-syntax-highlighter-98 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-98 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_98" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_98" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_98" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Resolved 33 packages in 640ms
Installed 33 packages in 23ms
+ annotated-doc==0.0.4
+ anyio==4.14.2
+ botwinick-utils==0.0.20
+ certifi==2026.6.17
+ click==8.3.3
+ filelock==3.30.2
+ fsspec==2026.6.0
+ h11==0.16.0
+ hf-xet==1.5.2
+ httpcore==1.0.9
+ httpx==0.28.1
+ huggingface-hub==1.8.0
+ idna==3.18
+ linkify-it-py==2.1.0
+ markdown-it-py==4.2.0
+ mdit-py-plugins==0.6.1
+ mdurl==0.1.2
+ packaging==26.2
+ platformdirs==4.10.0
+ pygments==2.20.0
+ python-json-logger==4.1.0
+ pyyaml==6.0.3
+ rich==15.0.0
+ scitrera-app-framework==0.0.69
+ shellingham==1.5.4
+ six==1.17.0
+ sparkrun==0.2.40
+ textual==8.2.5
+ tqdm==4.68.4
+ typer==0.27.0
+ typing-extensions==4.16.0
+ uc-micro-py==2.0.0
+ vpd==0.9.13
Installed 1 executable: sparkrun</textarea></div><div class="fusion-text fusion-text-179"><p>&nbsp;</p>
<p>Let&#8217;s verify the installation:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-99 > .CodeMirror, .fusion-syntax-highlighter-99 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-99 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_99" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_99" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_99" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun &#8211;version</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-100 > .CodeMirror, .fusion-syntax-highlighter-100 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-100 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_100" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_100" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_100" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">sparkrun, version 0.2.40</textarea></div><div class="fusion-text fusion-text-180"><p>&nbsp;</p>
<p>sparkrun v0.2.40 is installed and accessible.</p>
<hr />
<h3>3. Building the Docker Image</h3>
<p>To get the best performance from GPT-OSS 120B, we will build a Docker image that contains CUTLASS MoE and FlashInfer attention kernels compiled specifically for the Spark&#8217;s GPU architecture. This image also includes the <code>--mxfp4-backend</code>, <code>--mxfp4-layers</code> flags and GPT-OSS&#8217;s tiktoken encoding files.</p>
<p>We will build this image using the build-and-copy.sh script from the eugr community project. First, clone the eugr repository:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-101 > .CodeMirror, .fusion-syntax-highlighter-101 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-101 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_101" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_101" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_101" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~
git clone https://github.com/eugr/spark-vllm-docker.git
cd spark-vllm-docker</textarea></div><div class="fusion-text fusion-text-181"><p>&nbsp;</p>
<p>The <code>Dockerfile.mxfp4</code> file in the repository determines the version of the source code that build-and-copy.sh will compile. Before starting the build, we must ensure this version is up to date. Therefore, let&#8217;s query the latest commit hash of the mxfp4_v2 branch in christopherowen&#8217;s community project vLLM fork:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-102 > .CodeMirror, .fusion-syntax-highlighter-102 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-102 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_102" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_102" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_102" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">git ls-remote https://github.com/christopherowen/vllm.git refs/heads/mxfp4_v2 | cut -f1</textarea></div><div class="fusion-text fusion-text-182"><p>&nbsp;</p>
<p>This hash returned by the command will direct build-and-copy.sh to the latest version of the vLLM source code we want. Copy or note it. Then open the Dockerfile.mxfp4 file with nano:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-103 > .CodeMirror, .fusion-syntax-highlighter-103 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-103 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_103" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_103" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_103" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">nano Dockerfile.mxfp4</textarea></div><div class="fusion-text fusion-text-183"><p>&nbsp;</p>
<p>Once the file is open, press Ctrl+W to search within the file, type VLLM_SHA and press Enter. The cursor will land on a line similar to:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-104 > .CodeMirror, .fusion-syntax-highlighter-104 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-104 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_104" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_104" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_104" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">ARG VLLM_SHA=045293d82b832229560ac4a13152a095af603b6e</textarea></div><div class="fusion-text fusion-text-184"><p>&nbsp;</p>
<p>Delete the old hash value and paste the updated hash you noted above. Then press Ctrl+O to save the file, confirm with Enter, and exit nano with Ctrl+X.</p>
<p>Let&#8217;s verify the change we made to the Dockerfile.mxfp4 file:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-105 > .CodeMirror, .fusion-syntax-highlighter-105 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-105 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_105" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_105" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_105" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">grep VLLM_SHA Dockerfile.mxfp4</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-106 > .CodeMirror, .fusion-syntax-highlighter-106 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-106 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_106" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_106" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_106" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">ARG VLLM_SHA=04f641e537e80a67db464c6e65b928dbfab5d647</textarea></div><div class="fusion-text fusion-text-185"><p>&nbsp;</p>
<p>The returned hash value should be the updated hash you copied in the previous step.</p>
<p><strong>Fallback:</strong> If you experience issues with the latest commit, you can use the hash 04f641e537e80a67db464c6e65b928dbfab5d647 (January 28, 2026), which we have tested and confirmed. You can also check for newer commits in the <strong><a style="color: #47d600;" href="https://github.com/christopherowen/vllm/commits/mxfp4_v2">commit history</a> </strong>.</p>
<p>Now build the image:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-107 > .CodeMirror, .fusion-syntax-highlighter-107 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-107 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_107" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_107" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_107" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">DOCKER_BUILDKIT=1 ./build-and-copy.sh &#8211;exp-mxfp4</textarea></div><div class="fusion-text fusion-text-186"><p>&nbsp;</p>
<p>The first build takes approximately 50 minutes; subsequent builds complete in about 5 minutes thanks to caching. When the build completes, you will see the following output:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-108 > .CodeMirror, .fusion-syntax-highlighter-108 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-108 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_108" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_108" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_108" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">=========================================
TIMING STATISTICS
=========================================
Runner Build: 00:50:00
Total Time: 00:50:00
=========================================
Done preparing vllm-node-mxfp4.</textarea></div><div class="fusion-text fusion-text-187"><p>&nbsp;</p>
<p>Verify the image:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-109 > .CodeMirror, .fusion-syntax-highlighter-109 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-109 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_109" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_109" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_109" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker run &#8211;rm &#8211;entrypoint cat vllm-node-mxfp4:latest /workspace/build-metadata.yaml</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-110 > .CodeMirror, .fusion-syntax-highlighter-110 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-110 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_110" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_110" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_110" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/yaml">vllm_commit: 04f641e537e80a67db464c6e65b928dbfab5d647
flashinfer_commit: f349e52496a72a00d8c4ac02c7a1e38523ff7194
gpu_arch: 12.1a
base_image: nvcr.io/nvidia/pytorch:26.01-py3</textarea></div><div class="fusion-text fusion-text-188"><p>&nbsp;</p>
<p>Verify the image&#8217;s vLLM version:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-111 > .CodeMirror, .fusion-syntax-highlighter-111 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-111 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_111" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_111" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_111" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">docker run &#8211;rm &#8211;entrypoint python3 vllm-node-mxfp4:latest -c &#8220;import vllm; print(vllm.__version__)&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-112 > .CodeMirror, .fusion-syntax-highlighter-112 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-112 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_112" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_112" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_112" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">0.1.dev12774+g04f641e53.d20260720</textarea></div><div class="fusion-text fusion-text-189"><p>&nbsp;</p>
<p><strong>If you are using multiple Sparks,</strong> building the image only on the Main Spark is sufficient. sparkrun will automatically synchronize the image to the Workers over the CX-7 network on first launch. No additional step is needed.</p>
<hr />
<h3>4. Setup Wizard</h3>
<p>The sparkrun setup wizard configures all the infrastructure needed to run the Spark with a single interactive command:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-113 > .CodeMirror, .fusion-syntax-highlighter-113 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-113 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_113" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_113" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_113" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun setup wizard</textarea></div><div class="fusion-text fusion-text-190"><p>&nbsp;</p>
<p>Six phases run sequentially, and the phases are interactive. In the first phase, enter <code></code> in the &#8216;Enter host IPs/hostnames&#8217; field and press Enter. For the subsequent prompts, you can simply press Enter (confirming the default options) to continue:</p>
<p><strong>If you are using multiple Sparks,</strong> enter all the Sparks&#8217; IP addresses separated by commas in the &#8216;Enter host IPs/hostnames&#8217; field .</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-114 > .CodeMirror, .fusion-syntax-highlighter-114 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-114 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_114" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_114" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_114" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Welcome to sparkrun 0.2.40 setup wizard!
================================================</p>
<p>Phase 1: Cluster Setup
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
Detecting CX7 interfaces on this machine&#8230;
CX7 detected! This machine is a DGX Spark.
Enter host IPs/hostnames (comma-separated) [192.168.1.148]: 192.168.1.148,192.168.1.163
Cluster name [default]: dualspark
SSH username [nvidia]:
Created cluster &#8216;dualspark&#8217; with 2 host(s), set as default.</textarea></div><div class="fusion-text fusion-text-191"><p>&nbsp;</p>
<p>The second phase sets up passwordless SSH. Type <code>Y</code> and press Enter:</p>
<p><strong>If you are using multiple Sparks,</strong> the wizard connects to the Worker Sparks via SSH during this phase and asks for the password. Enter the password — this step is done once; after the wizard distributes the passwordless SSH keys, no password will be asked for in subsequent connections.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-115 > .CodeMirror, .fusion-syntax-highlighter-115 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-115 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_115" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_115" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_115" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Phase 2: SSH Mesh
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
Set up SSH mesh across 2 host(s) + this machine? [Y/n]: Y
=== Phase 1: Connectivity check ===[*] Checking SSH connectivity to nvidia@192.168.1.148 &#8230;[*] Checking SSH connectivity to nvidia@192.168.1.163 &#8230;</p>
<p>=== Phase 4: Install keys so every host trusts every other host ===[*] Installing key from 192.168.1.148 onto all other hosts &#8230;
&#8211; 192.168.1.148 -&gt; 192.168.1.163[*] Installing key from 192.168.1.163 onto all other hosts &#8230;
&#8211; 192.168.1.163 -&gt; 192.168.1.148</p>
<p>=== Done ===
All hosts should now be able to SSH to each other as &#8216;nvidia&#8217; without passwords.</p>
<p>Detecting management IPs on cluster hosts&#8230;
Updating cluster hosts to management IPs:
192.168.1.148 -&gt; 127.0.0.1 (local)
Cluster &#8216;dualspark&#8217; updated.</textarea></div><div class="fusion-text fusion-text-192"><p>&nbsp;</p>
<p>The third phase configures the CX-7 high-speed network interfaces. It is automatically skipped if a single Spark is used. Type <code>Y</code> and press Enter:</p>
<p><strong>If you are using multiple Sparks,</strong> the wizard automatically detects CX-7 interfaces, selects non-overlapping subnets, assigns a static IP to each Spark, sets MTU 9000 (jumbo frame), and writes the <code>/etc/netplan/40-cx7.yaml</code> file and applies it with <code>netplan apply</code>. This operation requires root privileges — enter the sudo password.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-116 > .CodeMirror, .fusion-syntax-highlighter-116 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-116 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_116" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_116" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_116" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Phase 3: CX7 Network Configuration
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
Configures high-speed CX7 networking between hosts.
Configure CX7 networking? [Y/n]: Y
Topology: switch
Subnets: 192.168.0.0/24, 192.168.2.0/24[sudo] password for nvidia: </textarea></div><div class="fusion-text fusion-text-193"><p>&nbsp;</p>
<p>After CX-7 configuration, the wizard automatically refreshes the SSH network with the new IPs. This phase requires no additional input.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-117 > .CodeMirror, .fusion-syntax-highlighter-117 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-117 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_117" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_117" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_117" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Phase 3b: Re-meshing SSH after CX7 IP changes
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
CX7 configuration changed network IPs. Re-running SSH mesh
to ensure full connectivity across all interfaces.</p>
<p>Verifying SSH connectivity to cluster hosts&#8230;
All 2 host(s) reachable.</p>
<p>Checking reachability from control machine&#8230;
Reachable: 127.0.0.1, 192.168.1.163, 192.168.0.148, 192.168.2.148, 192.168.0.163, 192.168.2.163</textarea></div><div class="fusion-text fusion-text-194"><p>&nbsp;</p>
<p>The fourth phase verifies that the user is in the docker group. Type <code>Y</code> and press Enter:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-118 > .CodeMirror, .fusion-syntax-highlighter-118 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-118 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_118" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_118" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_118" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Phase 4: Docker Group Membership
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
Ensures user can run Docker commands without sudo.
Add &#8216;nvidia&#8217; to the docker group on all hosts? [Y/n]: Y
127.0.0.1: &#8216;nvidia&#8217; already a member
192.168.1.163: &#8216;nvidia&#8217; already a member</textarea></div><div class="fusion-text fusion-text-195"><p>&nbsp;</p>
<p>The fifth phase installs restricted sudoers rules. The installed rules are for passwordlessly fixing HuggingFace cache ownership and passwordlessly clearing the Linux page cache. Broad sudo privileges are not granted. Type <code>Y</code> and press Enter:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-119 > .CodeMirror, .fusion-syntax-highlighter-119 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-119 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_119" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_119" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_119" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Phase 5: Sudoers Entries
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
Scoped sudoers for fix-permissions + clear-cache (no broad sudo).
Install sudoers entries? [Y/n]: Y
fix-permissions: 2/2 host(s)
clear-cache: 2/2 host(s)</textarea></div><div class="fusion-text fusion-text-196"><p>&nbsp;</p>
<p>The sixth phase installs earlyoom OOM protection. earlyoom terminates inference processes instead of locking up the system when memory limits are approached in the DGX Spark&#8217;s unified memory architecture. Type<strong> <code>Y</code></strong> and press Enter:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-120 > .CodeMirror, .fusion-syntax-highlighter-120 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-120 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_120" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_120" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_120" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Phase 6: earlyoom OOM Protection
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;
Prevents system hangs by proactively managing memory pressure.
Install earlyoom? [Y/n]: Y
earlyoom configured on 2/2 host(s).</p>
<p>Setup Complete!
================================================</p>
<p>Cluster: dualspark (2 hosts, set as default)
SSH mesh: OK
CX7: configured (switch)
SSH remesh: OK
Docker: OK (2/2)
Sudoers: installed (fix-permissions, clear-cache)
earlyoom: installed</textarea></div><div class="fusion-text fusion-text-197"><hr />
<h3>5. Creating the Recipe</h3>
<p>A recipe is a YAML file that defines how sparkrun runs the model. The model, Docker image, vLLM flags, and memory settings are consolidated in a single file. This recipe can be used for both a single Spark (TP=1) and multiple Sparks (TP&gt;1) — the only difference is that the <code>--tp</code> value in the launch command is adjusted according to the number of Sparks.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-121 > .CodeMirror, .fusion-syntax-highlighter-121 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-121 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_121" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_121" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_121" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cat &gt; ~/gpt-oss-120b-eugr-mxfp4.yaml &lt;&lt; &#8216;RECIPE&#8217;
# GPT-OSS 120B MXFP4 — eugr &#8211;exp-mxfp4 build, CUTLASS MoE, FLASHINFER attention
# Usage:
# sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful # single node (TP=1)
# sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful &#8211;tp 2 # dual node (TP=2)
recipe_version: &#8220;2&#8221;
model: openai/gpt-oss-120b
runtime: vllm
container: vllm-node-mxfp4</p>
<p>metadata:
description: GPT-OSS 120B MXFP4 — CUTLASS MoE, FLASHINFER attention, eugr &#8211;exp-mxfp4 build</p>
<p>defaults:
port: 8000
host: 0.0.0.0
tensor_parallel: 1
gpu_memory_utilization: 0.7
max_num_batched_tokens: 8192</p>
<p>env:
VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8: &#8220;1&#8221;</p>
<p>command: |
vllm serve {model}
&#8211;tool-call-parser openai
&#8211;reasoning-parser openai_gptoss
&#8211;enable-auto-tool-choice
&#8211;tensor-parallel-size {tensor_parallel}
&#8211;distributed-executor-backend ray
&#8211;gpu-memory-utilization {gpu_memory_utilization}
&#8211;enable-prefix-caching
&#8211;load-format fastsafetensors
&#8211;quantization mxfp4
&#8211;mxfp4-backend CUTLASS
&#8211;mxfp4-layers moe,qkv,o,lm_head
&#8211;attention-backend FLASHINFER
&#8211;kv-cache-dtype fp8
&#8211;max-num-batched-tokens {max_num_batched_tokens}
&#8211;host {host}
&#8211;port {port}
RECIPE</textarea></div><div class="fusion-text fusion-text-198"><p>&nbsp;</p>
<p>Verify the file:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-122 > .CodeMirror, .fusion-syntax-highlighter-122 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-122 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_122" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_122" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_122" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cat ~/gpt-oss-120b-eugr-mxfp4.yaml</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-123 > .CodeMirror, .fusion-syntax-highlighter-123 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-123 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_123" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_123" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_123" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/yaml"># GPT-OSS 120B MXFP4 — eugr &#8211;exp-mxfp4 build, CUTLASS MoE, FLASHINFER attention
# Usage:
# sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful # single node (TP=1)
# sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful &#8211;tp 2 # dual node (TP=2)
recipe_version: &#8220;2&#8221;
model: openai/gpt-oss-120b
runtime: vllm
container: vllm-node-mxfp4</p>
<p>metadata:
description: GPT-OSS 120B MXFP4 — CUTLASS MoE, FLASHINFER attention, eugr &#8211;exp-mxfp4 build</p>
<p>defaults:
port: 8000
host: 0.0.0.0
tensor_parallel: 1
gpu_memory_utilization: 0.7
max_num_batched_tokens: 8192</p>
<p>env:
VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8: &#8220;1&#8221;</p>
<p>command: |
vllm serve {model}
&#8211;tool-call-parser openai
&#8211;reasoning-parser openai_gptoss
&#8211;enable-auto-tool-choice
&#8211;tensor-parallel-size {tensor_parallel}
&#8211;distributed-executor-backend ray
&#8211;gpu-memory-utilization {gpu_memory_utilization}
&#8211;enable-prefix-caching
&#8211;load-format fastsafetensors
&#8211;quantization mxfp4
&#8211;mxfp4-backend CUTLASS
&#8211;mxfp4-layers moe,qkv,o,lm_head
&#8211;attention-backend FLASHINFER
&#8211;kv-cache-dtype fp8
&#8211;max-num-batched-tokens {max_num_batched_tokens}
&#8211;host {host}
&#8211;port {port}</textarea></div><div class="fusion-text fusion-text-199"><p>&nbsp;</p>
<p>Key fields in the recipe and the reasons they were selected:</p>
<ul>
<li><strong><code>container</code></strong>
<ul>
<li><strong>Value:</strong> <code>vllm-node-mxfp4</code></li>
<li><strong>Why:</strong> Image built in step 3 — includes the CUTLASS/FlashInfer forks</li>
</ul>
</li>
<li><strong><code>tensor_parallel</code></strong>
<ul>
<li><strong>Value:</strong> <code>1</code> (default)</li>
<li><strong>Why:</strong> Default for a single Spark; overridden with the <code>--tp</code> value when using multiple Sparks</li>
</ul>
</li>
<li><strong><code>gpu_memory_utilization</code></strong>
<ul>
<li><strong>Value:</strong> <code>0.7</code></li>
<li><strong>Why:</strong> Safe value for DGX Spark UMA</li>
</ul>
</li>
<li><strong><code>--distributed-executor-backend ray</code></strong>
<ul>
<li><strong>Value:</strong> Ray</li>
<li><strong>Why:</strong> sparkrun automatically detects Ray and sets up the Ray cluster. For TP=1, Ray runs on a single node.</li>
</ul>
</li>
<li><strong><code>--mxfp4-backend CUTLASS</code></strong>
<ul>
<li><strong>Value:</strong> CUTLASS</li>
<li><strong>Why:</strong> Custom CUTLASS MXFP4 MoE GEMM kernel</li>
</ul>
</li>
<li><strong><code>--mxfp4-layers moe,qkv,o,lm_head</code></strong>
<ul>
<li><strong>Value:</strong> Full</li>
<li><strong>Why:</strong> All layers are quantized to FP4</li>
</ul>
</li>
<li><strong><code>--attention-backend FLASHINFER</code></strong>
<ul>
<li><strong>Value:</strong> FlashInfer</li>
<li><strong>Why:</strong> Custom FlashInfer kernel supporting GPT-OSS attention architecture</li>
</ul>
</li>
<li><strong><code>--kv-cache-dtype fp8</code></strong>
<ul>
<li><strong>Value:</strong> FP8</li>
<li><strong>Why:</strong> Reduced memory usage for the KV cache</li>
</ul>
</li>
<li><strong><code>--load-format fastsafetensors</code></strong>
<ul>
<li><strong>Value:</strong> Fast loader</li>
<li><strong>Why:</strong> ~41-second model loading time</li>
</ul>
</li>
<li><strong><code>--reasoning-parser openai_gptoss</code></strong>
<ul>
<li><strong>Value:</strong> Harmony</li>
<li><strong>Why:</strong> Separates the reasoning process into the <code>reasoning</code> field</li>
</ul>
</li>
<li><strong><code>--tool-call-parser openai</code></strong>
<ul>
<li><strong>Value:</strong> Tool usage</li>
<li><strong>Why:</strong> Required for <code>--enable-auto-tool-choice</code></li>
</ul>
</li>
</ul>
<p><strong>Important:</strong><br />The <code>--rootful</code> flag is not specified in the recipe; it is passed in the launch command.<br />FlashInfer compiles CUTLASS attention kernels at runtime, and this compilation requires root privileges.<br />The launch commands are shown in the “Launch the Model” step of the relevant section.</p>
</div><div class="fusion-text fusion-text-200"><h3>6. Pre-Launch Checks</h3>
<p>Verify that `sparkrun` correctly parses the recipe and that the memory budget is appropriate:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-124 > .CodeMirror, .fusion-syntax-highlighter-124 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-124 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_124" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_124" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_124" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun show ~/gpt-oss-120b-eugr-mxfp4.yaml</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-125 > .CodeMirror, .fusion-syntax-highlighter-125 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-125 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_125" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_125" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_125" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Name: /home/nvidia/gpt-oss-120b-eugr-mxfp4.yaml
Description: GPT-OSS 120B MXFP4 — CUTLASS MoE, FLASHINFER attention, eugr &#8211;exp-mxfp4 build
Runtime: vllm-ray
Model: openai/gpt-oss-120b
Container: vllm-node-mxfp4
Nodes: 1 &#8211; unlimited</p>
<p>Defaults:
gpu_memory_utilization: 0.7
host: 0.0.0.0
max_num_batched_tokens: 8192
port: 8000
tensor_parallel: 1</p>
<p>Environment:
VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8=1</p>
<p>Command:
vllm serve {model}
&#8211;tool-call-parser openai
&#8211;reasoning-parser openai_gptoss
&#8211;enable-auto-tool-choice
&#8211;tensor-parallel-size {tensor_parallel}
&#8211;distributed-executor-backend ray
&#8211;gpu-memory-utilization {gpu_memory_utilization}
&#8211;enable-prefix-caching
&#8211;load-format fastsafetensors
&#8211;quantization mxfp4
&#8211;mxfp4-backend CUTLASS
&#8211;mxfp4-layers moe,qkv,o,lm_head
&#8211;attention-backend FLASHINFER
&#8211;kv-cache-dtype fp8
&#8211;max-num-batched-tokens {max_num_batched_tokens}
&#8211;host {host}
&#8211;port {port}
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.</p>
<p>VRAM Estimation:
Model dtype: mxfp4
KV cache dtype: bfloat16
Architecture: 36 layers, 8 KV heads, 64 head_dim
Model weights: 60.77 GB
Tensor parallel: 1
Per-GPU total: 60.77 GB
DGX Spark fit: YES</p>
<p>GPU Memory Budget:
gpu_memory_utilization: 70%
Usable GPU memory: 84.7 GB (121 GB x 70%)
Available for KV: 23.9 GB
Max context tokens: 348,538</textarea></div><div class="fusion-text fusion-text-201"><p>&nbsp;</p>
<p>sparkrun correctly parsed the recipe, selected vllm-ray, and gave the &#8216;DGX Spark fit: YES&#8217; confirmation.</p>
<p>Now preview the launch plan:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-126 > .CodeMirror, .fusion-syntax-highlighter-126 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-126 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_126" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_126" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_126" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful &#8211;dry-run</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-127 > .CodeMirror, .fusion-syntax-highlighter-127 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-127 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_127" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_127" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_127" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">sparkrun v0.2.40</p>
<p>Runtime: vllm-ray
Image: vllm-node-mxfp4
Model: openai/gpt-oss-120b
Mode: solo
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.</p>
<p>VRAM Estimation:
Model dtype: mxfp4
KV cache dtype: bfloat16
Architecture: 36 layers, 8 KV heads, 64 head_dim
Model weights: 60.77 GB
Tensor parallel: 1
Per-GPU total: 60.77 GB
DGX Spark fit: YES</p>
<p>GPU Memory Budget:
gpu_memory_utilization: 70%
Usable GPU memory: 84.7 GB (121 GB x 70%)
Available for KV: 23.9 GB
Max context tokens: 348,538</p>
<p>Hosts: default cluster &#8216;dualspark&#8217;
Target: 127.0.0.1</p>
[1/6] Preparing
done (0.0s)[2/6] Building image — skipped (no builder)[3/6] Distributing resources
done (0.1s)[4/6] Syncing tuning configs
done (0.0s)[5/6] Launching vllm runtime
Step 1/3: Detecting InfiniBand
Step 2/3: Launching container
Step 3/3: Executing serve command
done (0.0s)
Cluster: sparkrun_531d9a939d23</p>
<p>Serve command:
vllm serve openai/gpt-oss-120b
&#8211;tool-call-parser openai
&#8211;reasoning-parser openai_gptoss
&#8211;enable-auto-tool-choice
&#8211;tensor-parallel-size 1
&#8211;distributed-executor-backend ray
&#8211;gpu-memory-utilization 0.7
&#8211;enable-prefix-caching
&#8211;load-format fastsafetensors
&#8211;quantization mxfp4
&#8211;mxfp4-backend CUTLASS
&#8211;mxfp4-layers moe,qkv,o,lm_head
&#8211;attention-backend FLASHINFER
&#8211;kv-cache-dtype fp8
&#8211;max-num-batched-tokens 8192
&#8211;host 0.0.0.0
&#8211;port 8000</p>
[6/6] Post-launch hooks — skipped</textarea></div><div class="fusion-text fusion-text-202"><p>&nbsp;</p>
<p>Dry run successful. The recipe is valid, the serve command includes the flags we added, and it is ready to launch. If the model is not installed on the Spark, sparkrun will automatically download it from Hugging Face on first launch. The GPT-OSS 120B model is approximately 183 GB; download time depends on your internet speed. The dry run does not trigger this download — downloading only happens on the actual launch.</p>
<p><strong>If you are using multiple Sparks,</strong> set the <code>--tp</code> value according to the number of Sparks for the TP dry-run (for example, <code>--tp 2</code> for two Sparks). You will see <code>--tensor-parallel-size 2</code> in the serve command, and the mode will appear as <code>cluster (2 nodes)</code> instead of <code>solo</code>:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-128 > .CodeMirror, .fusion-syntax-highlighter-128 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-128 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_128" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_128" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_128" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful &#8211;tp 2 &#8211;dry-run</textarea></div><div class="fusion-text fusion-text-203"><hr />
<h2>Running with a Single Spark</h2>
<h3>1. Clearing the Cache</h3>
<p>Before starting vLLM, clear the filesystem cache:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-129 > .CodeMirror, .fusion-syntax-highlighter-129 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-129 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_129" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_129" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_129" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sudo sh -c &#8216;sync; echo 3 &gt; /proc/sys/vm/drop_caches&#8217;</textarea></div><div class="fusion-text fusion-text-204"><p>&nbsp;</p>
<p>The main reason for doing this is the DGX Spark&#8217;s unified memory architecture: The operating system caches the model files it reads from disk in RAM. vLLM loads the model weights from here into GPU memory. After loading is complete, the data remaining in the cache is not used again but is not cleaned up immediately. In systems with separate memory, this is not significant. Since inference runs in GPU memory, RAM fullness does not affect performance. On the Spark, however, since the CPU and GPU share the same RAM, the cache reduces the space available to the GPU. The command clears this cache, providing maximum memory for vLLM.</p>
<p>When prompted for a password, enter the Spark&#8217;s password. The command produces no output; it completes silently:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-130 > .CodeMirror, .fusion-syntax-highlighter-130 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-130 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_130" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_130" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_130" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[sudo] password for nvidia:</textarea></div><div class="fusion-text fusion-text-205"><hr />
<h3>2. Launching the Model</h3>
<p>Now let&#8217;s launch the model:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-131 > .CodeMirror, .fusion-syntax-highlighter-131 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-131 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_131" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_131" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_131" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful &#8211;no-follow</textarea></div><div class="fusion-text fusion-text-206"><p>&nbsp;</p>
<p>The <code>--rootful</code> flag runs the container with root privileges. FlashInfer compiles CUTLASS attention kernels at runtime and requires root privileges for this compilation. The <code>--no-follow</code> flag causes sparkrun to return to the command line after launching the containers.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-132 > .CodeMirror, .fusion-syntax-highlighter-132 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-132 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_132" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_132" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_132" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Note: 1 nodes required, using 1 of 2 hosts
sparkrun v0.2.40</p>
<p>Runtime: vllm-ray
Image: vllm-node-mxfp4
Model: openai/gpt-oss-120b
Mode: solo</p>
<p>VRAM Estimation:
Model dtype: mxfp4
KV cache dtype: bfloat16
Architecture: 36 layers, 8 KV heads, 64 head_dim
Model weights: 60.77 GB
Tensor parallel: 1
Per-GPU total: 60.77 GB
DGX Spark fit: YES</p>
<p>GPU Memory Budget:
gpu_memory_utilization: 70%
Usable GPU memory: 84.7 GB (121 GB x 70%)
Available for KV: 23.9 GB
Max context tokens: 348,538</p>
<p>Hosts: default cluster &#8216;dualspark&#8217;
Target: 127.0.0.1</p>
[1/6] Preparing
done (0.0s)[2/6] Building image — skipped (no builder)[3/6] Distributing resources
Fetching 37 files: 100%|██████████| 37/37 [00:00&lt;00:00, 10007.69it/s]
done (0.6s)[4/6] Syncing tuning configs
done (0.0s)[5/6] Launching vllm runtime
Step 1/3: Detecting InfiniBand
Step 2/3: Launching container
Step 3/3: Executing serve command
done (7.0s)
Cluster: sparkrun_531d9a939d23</p>
<p>Serve command:
vllm serve openai/gpt-oss-120b
&#8211;tool-call-parser openai
&#8211;reasoning-parser openai_gptoss
&#8211;enable-auto-tool-choice
&#8211;tensor-parallel-size 1
&#8211;distributed-executor-backend ray
&#8211;gpu-memory-utilization 0.7
&#8211;enable-prefix-caching
&#8211;load-format fastsafetensors
&#8211;quantization mxfp4
&#8211;mxfp4-backend CUTLASS
&#8211;mxfp4-layers moe,qkv,o,lm_head
&#8211;attention-backend FLASHINFER
&#8211;kv-cache-dtype fp8
&#8211;max-num-batched-tokens 8192
&#8211;host 0.0.0.0
&#8211;port 8000</p>
<p>Runtime versions:
cuda: 13.1
nccl: (2, 29, 2)
python: 3.12.3
torch: 2.10.0a0+a36e1d39eb.nv26.01.42222806
vllm: 0.1.dev12774+g04f641e53.d20260720</p>
[6/6] Post-launch hooks — skipped</textarea></div><div class="fusion-text fusion-text-207"><p>&nbsp;</p>
<p>sparkrun successfully completed all 6 steps. Image building was skipped, and the image we built during setup was used. Since the model files were already on disk, distribution completed quickly. All flags were correctly resolved within the serve command.</p>
<hr />
<h3>3. Monitoring the Startup Process</h3>
<p>After the model download completes, it may take a few minutes for vLLM to become ready to serve. During this process, vLLM loads the model weights into GPU memory, compiles GPU kernels, and allocates memory for inference.</p>
<p>To monitor the vLLM logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-133 > .CodeMirror, .fusion-syntax-highlighter-133 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-133 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_133" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_133" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_133" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun logs ~/gpt-oss-120b-eugr-mxfp4.yaml</textarea></div><div class="fusion-text fusion-text-208"><p>&nbsp;</p>
<p>You will see the following in the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-134 > .CodeMirror, .fusion-syntax-highlighter-134 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-134 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_134" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_134" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_134" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">(APIServer pid=107) vLLM API server version 0.1.dev12774+g04f641e53.d20260720
(APIServer pid=107) non-default args: {&#8216;quantization&#8217;: &#8216;mxfp4&#8217;, &#8216;mxfp4_backend&#8217;: &#8216;CUTLASS&#8217;,
&#8216;mxfp4_layers&#8217;: &#8216;moe,qkv,o,lm_head&#8217;, &#8216;attention_backend&#8217;: &#8216;FLASHINFER&#8217;,
&#8216;distributed_executor_backend&#8217;: &#8216;ray&#8217;, &#8216;tensor_parallel_size&#8217;: 1, &#8230;}</p>
<p>(EngineCore_DP0 pid=512) Started a local Ray instance.</p>
<p>(RayWorkerWrapper pid=1347) SM12x detected &#8211; using native FlashInfer CUTLASS attention
instead of TRT-LLM attention (cubins not available for SM12x)
(RayWorkerWrapper pid=1347) Using AttentionBackendEnum.FLASHINFER backend.
(RayWorkerWrapper pid=1347) [MXFP4] Using backend: CUTLASS (&#8211;mxfp4-backend)
(RayWorkerWrapper pid=1347) Loading safetensors: 100%|██████████| 15/15 [00:41&lt;00:00] (RayWorkerWrapper pid=1347) Loading weights took 41.43 seconds (RayWorkerWrapper pid=1347) [MXFP4] lm_head quantized: torch.Size([201088, 2880]) BF16 -&gt; torch.Size([201088, 1440]) FP4 (4x smaller)
(RayWorkerWrapper pid=1347) Model loading took 61.33 GiB memory</p>
<p>(RayWorkerWrapper pid=1347) torch.compile takes 112.10 s in total
(RayWorkerWrapper pid=1347) Available KV cache memory: 8.460000 GiB
(EngineCore_DP0 pid=512) GPU KV cache size: 246,416 tokens
(EngineCore_DP0 pid=512) Maximum concurrency for 131,072 tokens per request: 3.54x</p>
<p>(APIServer pid=107) Application startup complete.</textarea></div><div class="fusion-text fusion-text-209"><p>&nbsp;</p>
<p>If you see the <code>Application startup complete.</code> line, the model server is ready. Press <code>Ctrl+C</code> to stop watching the logs. The server will continue running in the background.</p>
<hr />
<h3>4. Testing the Model</h3>
<p>First, verify that the server is running:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-135 > .CodeMirror, .fusion-syntax-highlighter-135 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-135 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_135" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_135" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_135" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s -o /dev/null -w &#8220;HTTP %{http_code}&#8221; http://:8000/health</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-136 > .CodeMirror, .fusion-syntax-highlighter-136 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-136 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_136" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_136" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_136" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">HTTP 200</textarea></div><div class="fusion-text fusion-text-210"><p>&nbsp;</p>
<p>An &#8216;HTTP 200&#8217; response indicates the server is healthy. Now check the sparkrun container status:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-137 > .CodeMirror, .fusion-syntax-highlighter-137 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-137 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_137" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_137" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_137" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun status</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-138 > .CodeMirror, .fusion-syntax-highlighter-138 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-138 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_138" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_138" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_138" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Job: /home/nvidia/gpt-oss-120b-eugr-mxfp4.yaml (tp=1) [531d9a939d23] (1 container(s))
node_0 127.0.0.1 Up 2 minutes vllm-node-mxfp4</p>
<p>Total: 1 container(s) across 1 host(s)</textarea></div><div class="fusion-text fusion-text-211"><p>&nbsp;</p>
<p>List the registered models:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-139 > .CodeMirror, .fusion-syntax-highlighter-139 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-139 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_139" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_139" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_139" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s http://localhost:8000/v1/models | python3 -m json.tool</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-140 > .CodeMirror, .fusion-syntax-highlighter-140 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-140 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_140" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_140" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_140" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="application/json">{
&#8220;object&#8221;: &#8220;list&#8221;,
&#8220;data&#8221;: [
{
&#8220;id&#8221;: &#8220;openai/gpt-oss-120b&#8221;,
&#8220;object&#8221;: &#8220;model&#8221;,
&#8220;created&#8221;: 1784286672,
&#8220;owned_by&#8221;: &#8220;vllm&#8221;,
&#8220;root&#8221;: &#8220;openai/gpt-oss-120b&#8221;,
&#8220;parent&#8221;: null,
&#8220;max_model_len&#8221;: 131072
}
]
}</textarea></div><div class="fusion-text fusion-text-212"><p>&nbsp;</p>
<p>The model <code>openai/gpt-oss-120b</code> is registered.</p>
<p>Now test the model with the prompt &#8220;What is 12*17?&#8221;:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-141 > .CodeMirror, .fusion-syntax-highlighter-141 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-141 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_141" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_141" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_141" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s http://localhost:8000/v1/chat/completions
-H &#8220;Content-Type: application/json&#8221;
-d &#8216;{
&#8220;model&#8221;: &#8220;openai/gpt-oss-120b&#8221;,
&#8220;messages&#8221;: [{&#8220;role&#8221;: &#8220;user&#8221;, &#8220;content&#8221;: &#8220;What is 12*17?&#8221;}],
&#8220;max_tokens&#8221;: 200
}&#8217; | python3 -m json.tool</textarea></div><div class="fusion-text fusion-text-213"><p>&nbsp;</p>
<p>A simplified version of the response is below. In the <code>content</code> field (the model&#8217;s answer) you will see <code>204</code>. From this, you can tell the model performed the multiplication correctly. Additionally, the response has a <code>reasoning</code> field. This field contains the model&#8217;s reasoning process. GPT-OSS&#8217;s Harmony format is working: the reasoning process and the final answer are presented in separate fields:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-142 > .CodeMirror, .fusion-syntax-highlighter-142 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-142 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_142" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_142" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_142" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="application/json">{
&#8220;id&#8221;: &#8220;chatcmpl-b863d6c512722c38&#8221;,
&#8220;model&#8221;: &#8220;openai/gpt-oss-120b&#8221;,
&#8220;choices&#8221;: [
{
&#8220;index&#8221;: 0,
&#8220;message&#8221;: {
&#8220;role&#8221;: &#8220;assistant&#8221;,
&#8220;content&#8221;: &#8220;(12 times 17 = 204)&#8221;,
&#8220;reasoning&#8221;: &#8220;User asks a simple multiplication: 12*17 = 204. Provide answer.&#8221;,
&#8220;reasoning_content&#8221;: &#8220;User asks a simple multiplication: 12*17 = 204. Provide answer.&#8221;
},
&#8220;finish_reason&#8221;: &#8220;stop&#8221;
}
],
&#8220;usage&#8221;: {
&#8220;prompt_tokens&#8221;: 72,
&#8220;total_tokens&#8221;: 110,
&#8220;completion_tokens&#8221;: 38
}
}</textarea></div><div class="fusion-text fusion-text-214"><p>&nbsp;</p>
<p>vLLM is now running and serving on the Spark&#8217;s port 8000. We can ask GPT-OSS 120B questions and get answers.</p>
<hr />
<h3>5. Shutdown</h3>
<p>When you are done, stop the model:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-143 > .CodeMirror, .fusion-syntax-highlighter-143 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-143 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_143" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_143" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_143" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun stop ~/gpt-oss-120b-eugr-mxfp4.yaml</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-144 > .CodeMirror, .fusion-syntax-highlighter-144 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-144 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_144" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_144" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_144" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Workload stopped on 1 host(s).</textarea></div><div class="fusion-text fusion-text-215"><p>&nbsp;</p>
<p>This command stops the container and frees up memory. However, the container, the Docker image, and the model files remain on disk. This means you don&#8217;t need to re-download to start again. Simply re-run the <code>sparkrun run</code> command from step 2.</p>
<hr />
<h2>Running with Two Sparks</h2>
<h3>1. Clearing the Cache</h3>
<p>Before starting vLLM, clear the filesystem cache on both Sparks. The main reason for doing this is the DGX Spark&#8217;s unified memory architecture (UMA): The operating system caches the model files it reads from disk in RAM. vLLM loads the model weights from here into GPU memory. After loading is complete, the data remaining in the cache is not used again but is not cleaned up immediately. In systems with separate memory, this is not significant. Since inference runs in GPU memory, RAM fullness does not affect performance. On the Spark, however, since the CPU and GPU share the same RAM, the cache reduces the space available to the GPU. The command clears this cache, providing maximum memory for vLLM.</p>
<p>Clear the cache on the Main Spark:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-145 > .CodeMirror, .fusion-syntax-highlighter-145 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-145 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_145" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_145" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_145" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sudo sh -c &#8216;sync; echo 3 &gt; /proc/sys/vm/drop_caches&#8217;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-146 > .CodeMirror, .fusion-syntax-highlighter-146 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-146 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_146" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_146" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_146" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[sudo] password for nvidia:</textarea></div><div class="fusion-text fusion-text-216"><p>&nbsp;</p>
<p>Apply the same cleanup on the Worker Spark via SSH from the Main Spark:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-147 > .CodeMirror, .fusion-syntax-highlighter-147 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-147 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_147" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_147" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_147" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ssh nvidia@ &#8220;sudo sh -c &#8216;sync; echo 3 &gt; /proc/sys/vm/drop_caches'&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-148 > .CodeMirror, .fusion-syntax-highlighter-148 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-148 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_148" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_148" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_148" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[sudo] password for nvidia:</textarea></div><div class="fusion-text fusion-text-217"><hr />
<h3>2. Launching the Model</h3>
<p>Now, let&#8217;s launch the model on two Sparks:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-149 > .CodeMirror, .fusion-syntax-highlighter-149 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-149 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_149" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_149" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_149" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun run ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;rootful &#8211;tp 2 &#8211;no-follow</textarea></div><div class="fusion-text fusion-text-218"><p>&nbsp;</p>
<p>The <code>--tp 2</code> flag tells sparkrun to run the model split (tensor parallel) across two Sparks. The <code>--rootful</code> flag runs the container with root privileges — FlashInfer compiles CUTLASS attention kernels at runtime and requires root privileges for this compilation. The <code>--no-follow</code> flag causes sparkrun to return to the command line after launching the containers. sparkrun automatically synchronizes the image to the Worker (skips if the same ID), copies the model to the Worker (skips if already present), sets up the Ray cluster, configures NCCL for the CX-7 interfaces, and launches containers on both Sparks.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-150 > .CodeMirror, .fusion-syntax-highlighter-150 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-150 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_150" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_150" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_150" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/yaml">sparkrun v0.2.40&lt;/p&gt;
&lt;p&gt;Runtime: vllm-ray&lt;br /&gt;
Image: vllm-node-mxfp4&lt;br /&gt;
Model: openai/gpt-oss-120b&lt;br /&gt;
Mode: cluster (2 nodes)&lt;br /&gt;
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.&lt;br /&gt;
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.&lt;/p&gt;
&lt;p&gt;VRAM Estimation:&lt;br /&gt;
Model dtype: mxfp4&lt;br /&gt;
KV cache dtype: bfloat16&lt;br /&gt;
Architecture: 36 layers, 8 KV heads, 64 head_dim&lt;br /&gt;
Model weights: 60.77 GB&lt;br /&gt;
Tensor parallel: 2&lt;br /&gt;
Per-GPU total: 30.38 GB&lt;br /&gt;
DGX Spark fit: YES&lt;/p&gt;
&lt;p&gt;GPU Memory Budget:&lt;br /&gt;
gpu_memory_utilization: 70%&lt;br /&gt;
Usable GPU memory: 84.7 GB (121 GB x 70%)&lt;br /&gt;
Available for KV: 54.3 GB&lt;br /&gt;
Max context tokens: 1,582,071&lt;/p&gt;
&lt;p&gt;Hosts: default cluster &#8216;dualspark'&lt;br /&gt;
Head: 127.0.0.1&lt;br /&gt;
Workers: 192.168.1.163&lt;/p&gt;[1/6] Preparing&lt;br /&gt;
done (0.0s)[2/6] Building image — skipped (no builder)[3/6] Distributing resources&lt;br /&gt;
Distributing image vllm-node-mxfp4 to 2 host(s)&lt;br /&gt;
Container image stale on 1 of 2 host(s), syncing&lt;br /&gt;
Distributing model openai/gpt-oss-120b to 2 host(s)&lt;br /&gt;
Fetching 37 files: 100%|██████████| 37/37 [00:00&amp;lt;00:00, 7261.00it/s]
Model synced to 2 host(s)&lt;br /&gt;
done (187.3s)[4/6] Syncing tuning configs&lt;br /&gt;
done (0.0s)[5/6] Launching vllm runtime&lt;br /&gt;
Step 1/5: Cleaning up existing containers&lt;br /&gt;
Step 2/5: Detecting InfiniBand&lt;br /&gt;
Step 3/5: Launching Ray head&lt;br /&gt;
Step 4/5: Launching Ray workers&lt;br /&gt;
Step 5/5: Executing serve command on head&lt;br /&gt;
done (17.9s)&lt;br /&gt;
Cluster: sparkrun_631038688bb2&lt;/p&gt;
&lt;p&gt;Serve command:&lt;br /&gt;
vllm serve openai/gpt-oss-120b &lt;br /&gt;
&#8211;tool-call-parser openai &lt;br /&gt;
&#8211;reasoning-parser openai_gptoss &lt;br /&gt;
&#8211;enable-auto-tool-choice &lt;br /&gt;
&#8211;tensor-parallel-size 2 &lt;br /&gt;
&#8211;distributed-executor-backend ray &lt;br /&gt;
&#8211;gpu-memory-utilization 0.7 &lt;br /&gt;
&#8211;enable-prefix-caching &lt;br /&gt;
&#8211;load-format fastsafetensors &lt;br /&gt;
&#8211;quantization mxfp4 &lt;br /&gt;
&#8211;mxfp4-backend CUTLASS &lt;br /&gt;
&#8211;mxfp4-layers moe,qkv,o,lm_head &lt;br /&gt;
&#8211;attention-backend FLASHINFER &lt;br /&gt;
&#8211;kv-cache-dtype fp8 &lt;br /&gt;
&#8211;max-num-batched-tokens 8192 &lt;br /&gt;
&#8211;host 0.0.0.0 &lt;br /&gt;
&#8211;port 8000&lt;/p&gt;
&lt;p&gt;Runtime versions:&lt;br /&gt;
cuda: 13.1&lt;br /&gt;
nccl: (2, 29, 2)&lt;br /&gt;
python: 3.12.3&lt;br /&gt;
torch: 2.10.0a0+a36e1d39eb.nv26.01.42222806&lt;br /&gt;
vllm: 0.1.dev12774+g04f641e53.d20260720&lt;/p&gt;[6/6] Post-launch hooks — skipped</textarea></div><div class="fusion-text fusion-text-219"><p>&nbsp;</p>
<p>sparkrun successfully completed all 6 steps. The &#8216;Mode: cluster (2 nodes)&#8217; section was seen. The image was synchronized to the Worker over CX-7. <code>--tensor-parallel-size 2</code> appears in the serve command. sparkrun detected the <code>--distributed-executor-backend ray</code> flag, selected the <code>vllm-ray</code> runtime, and automatically set up the Ray cluster.</p>
<p>At TP=2, the model weights are split in two — each Spark loads ~30 GB (vs. 61 GB at TP=1). This leaves 35.97 GiB of memory for KV cache on each Spark (vs. 8.46 GiB at TP=1) — meaning more concurrent requests.</p>
<hr />
<h3>3. Monitoring the Startup Process</h3>
<p>After the model download completes, it may take a few minutes for vLLM to become ready to serve. During this process, vLLM loads the model weights into GPU memory, compiles GPU kernels, and allocates memory for inference.</p>
<p>Monitor the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-151 > .CodeMirror, .fusion-syntax-highlighter-151 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-151 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_151" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_151" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_151" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun logs ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;tp 2</textarea></div><div class="fusion-text fusion-text-220"><p>&nbsp;</p>
<p>You will see the following in the logs:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-152 > .CodeMirror, .fusion-syntax-highlighter-152 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-152 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_152" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_152" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_152" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/yaml">(APIServer pid=1093) vLLM API server version 0.1.dev12774+g04f641e53.d20260720
(APIServer pid=1093) non-default args: {&#8216;quantization&#8217;: &#8216;mxfp4&#8217;, &#8216;mxfp4_backend&#8217;: &#8216;CUTLASS&#8217;,
&#8216;mxfp4_layers&#8217;: &#8216;moe,qkv,o,lm_head&#8217;, &#8216;attention_backend&#8217;: &#8216;FLASHINFER&#8217;,
&#8216;distributed_executor_backend&#8217;: &#8216;ray&#8217;, &#8216;tensor_parallel_size&#8217;: 2, &#8230;}</p>
<p>(EngineCore_DP0 pid=1407) Connecting to existing Ray cluster at address: 192.168.1.148:46379&#8230;
(EngineCore_DP0 pid=1407) Connected to Ray cluster.
(EngineCore_DP0 pid=1407) Creating a new placement group.</p>
<p>(RayWorkerWrapper pid=1510) Loading safetensors using Fastsafetensor loader: 100%|██████████| 8/8 [00:24&lt;00:00]
(RayWorkerWrapper pid=372, ip=192.168.1.163) Loading safetensors using Fastsafetensor loader: 100%|██████████| 8/8 [00:23&lt;00:00] (RayWorkerWrapper pid=372, ip=192.168.1.163) [MXFP4] lm_head quantized: torch.Size([100608, 2880]) BF16 -&gt; torch.Size([100608, 1440]) FP4 (4x smaller)
(RayWorkerWrapper pid=372, ip=192.168.1.163) Model loading took 31.99 GiB memory and 38.80 seconds</p>
<p>(RayWorkerWrapper pid=1510) torch.compile takes 116.55 s in total
(RayWorkerWrapper pid=372, ip=192.168.1.163) Available KV cache memory: 35.97 GiB
(EngineCore_DP0 pid=1407) GPU KV cache size: 2,095,472 tokens
(EngineCore_DP0 pid=1407) Maximum concurrency for 131,072 tokens per request: 30.06x</p>
<p>(APIServer pid=1093) Application startup complete.</textarea></div><div class="fusion-text fusion-text-221"><p>&nbsp;</p>
<p>If you see the <code>Application startup complete.</code> line, the model server is ready. Press <code>Ctrl+C</code> to stop watching the logs. The server will continue running in the background.</p>
<hr />
<h3>4. Testing the Model</h3>
<p>Check the container status. Containers should be running on both Sparks:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-153 > .CodeMirror, .fusion-syntax-highlighter-153 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-153 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_153" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_153" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_153" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun status</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-154 > .CodeMirror, .fusion-syntax-highlighter-154 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-154 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_154" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_154" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_154" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Job: /home/nvidia/gpt-oss-120b-eugr-mxfp4.yaml (tp=2) [631038688bb2] (2 container(s))
head 127.0.0.1 Up 4 minutes vllm-node-mxfp4
worker 192.168.1.163 Up 4 minutes vllm-node-mxfp4</p>
<p>Total: 2 container(s) across 2 host(s)</textarea></div><div class="fusion-text fusion-text-222"><p>&nbsp;</p>
<p>Both containers are running.</p>
<p>Let&#8217;s run a health check on the model:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-155 > .CodeMirror, .fusion-syntax-highlighter-155 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-155 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_155" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_155" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_155" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s -o /dev/null -w &#8220;HTTP %{http_code}&#8221; http://:8000/health</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-156 > .CodeMirror, .fusion-syntax-highlighter-156 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-156 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_156" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_156" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_156" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">HTTP 200</textarea></div><div class="fusion-text fusion-text-223"><p>&nbsp;</p>
<p>The model is healthy. Now let&#8217;s test the model with a prompt:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-157 > .CodeMirror, .fusion-syntax-highlighter-157 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-157 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_157" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_157" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_157" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">curl -s http://:8000/v1/chat/completions
-H &#8220;Content-Type: application/json&#8221;
-d &#8216;{
&#8220;model&#8221;: &#8220;openai/gpt-oss-120b&#8221;,
&#8220;messages&#8221;: [{&#8220;role&#8221;: &#8220;user&#8221;, &#8220;content&#8221;: &#8220;What is 12*17?&#8221;}],
&#8220;max_tokens&#8221;: 200
}&#8217; | python3 -m json.tool</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-158 > .CodeMirror, .fusion-syntax-highlighter-158 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-158 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_158" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_158" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_158" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="application/json">{
&#8220;id&#8221;: &#8220;chatcmpl-a2bdd35cbf300d46&#8221;,
&#8220;model&#8221;: &#8220;openai/gpt-oss-120b&#8221;,
&#8220;choices&#8221;: [
{
&#8220;index&#8221;: 0,
&#8220;message&#8221;: {
&#8220;role&#8221;: &#8220;assistant&#8221;,
&#8220;content&#8221;: &#8220;(12 times 17 = 204)&#8221;,
&#8220;reasoning&#8221;: &#8220;User asks a simple multiplication: 12*17 = 204. Provide answer.&#8221;,
&#8220;reasoning_content&#8221;: &#8220;User asks a simple multiplication: 12*17 = 204. Provide answer.&#8221;
},
&#8220;finish_reason&#8221;: &#8220;stop&#8221;
}
],
&#8220;usage&#8221;: {
&#8220;prompt_tokens&#8221;: 72,
&#8220;total_tokens&#8221;: 110,
&#8220;completion_tokens&#8221;: 38
}
}</textarea></div><div class="fusion-text fusion-text-224"><p>&nbsp;</p>
<p>The model answered <code>12 × 17 = 204</code>. Additionally, the <code>reasoning</code> and <code>reasoning_content</code> fields contain the model&#8217;s reasoning processes.</p>
<p>The API is served only through the head node (Main Spark). That is, we communicate with the model via the Main Spark. The Worker Spark only participates in the inference computation.</p>
<hr />
<h3>5. Shutdown</h3>
<p>When you are done, stop the model. Since you used <code>--tp 2</code> when launching, you need to specify <code>--tp 2</code> when stopping as well:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-159 > .CodeMirror, .fusion-syntax-highlighter-159 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-159 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_159" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_159" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_159" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sparkrun stop ~/gpt-oss-120b-eugr-mxfp4.yaml &#8211;tp 2</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-160 > .CodeMirror, .fusion-syntax-highlighter-160 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-160 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_160" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_160" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_160" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Workload stopped on 2 host(s).</textarea></div><div class="fusion-text fusion-text-225"><p>&nbsp;</p>
<p>This command stops the container and frees up memory. However, the container, the Docker image, and the model files remain on disk. This means you don&#8217;t need to re-download to start again. Simply re-run the <code>sparkrun run</code> command from step 2.</p>
<hr />
<h2>Using with Open WebUI</h2>
<p>In the previous tutorial (<a href="tutorial-vllm-openwebui-spark.md">Local LLM Serving with vLLM on DGX Spark</a>), we set up Open WebUI and connected it to vLLM&#8217;s port 8000. The GPT-OSS 120B model is also served from the same port 8000. If Open WebUI is running, it automatically detects the model and lists it as <code>openai/gpt-oss-120b</code> in the model selection menu on the chat screen. You can select the model and start using it from the browser.</p>
<hr />
<h2>Single vs Dual Spark Performance Comparison</h2>
<p>We compared the single Spark (TP=1) and dual Spark (TP=2) configurations at different concurrency levels. The measurements recorded average TTFT (Time to First Token — the time until the first token starts being generated) and TPS (Tokens Per Second — the number of tokens generated per second) values.</p>
<p>&nbsp;</p>
<table>
<thead>
<tr>
<th>Concurrency</th>
<th>Single Spark TTFT (ms)</th>
<th>Dual Spark TTFT (ms)</th>
<th>Single Spark TPS (tok/s)</th>
<th>Dual Spark TPS (tok/s)</th>
<th>TPS Improvement</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>219.57</td>
<td>167.69</td>
<td>55.65</td>
<td>69.86</td>
<td>+26%</td>
</tr>
<tr>
<td>2</td>
<td>294.41</td>
<td>227.76</td>
<td>37.05</td>
<td>51.52</td>
<td>+39%</td>
</tr>
<tr>
<td>4</td>
<td>320.61</td>
<td>255.41</td>
<td>25.20</td>
<td>37.14</td>
<td>+47%</td>
</tr>
<tr>
<td>8</td>
<td>395.31</td>
<td>291.94</td>
<td>16.88</td>
<td>26.92</td>
<td>+59%</td>
</tr>
<tr>
<td>16</td>
<td>444.10</td>
<td>319.99</td>
<td>11.67</td>
<td>19.08</td>
<td>+63%</td>
</tr>
</tbody>
</table>
<p>&nbsp;</p>
<p>The dual Spark setup provides 26% higher token throughput at concurrency 1 (single user), and as concurrency increases, this gap grows, exceeding 60% at concurrency 16, as shown.</p>
<p>The measurements were taken using the <a href="https://github.com/CordatusAI/llm-benchmark">CordatusAI LLM Benchmark Tool</a>. This tool is a benchmarking application developed by CordatusAI that tests LLM servers with OpenAI-compatible APIs. Below are screenshots of the benchmark results the application produced for our model. As can be seen, the application can test at concurrency levels from 1 to 64, and at each level it measures TTFT, ITL, TPS, latency, and throughput metrics, presenting the results as tables and graphs. It also calculates the recommended number of users the system can support. This calculation is a general estimate based on certain assumptions and does not reflect all usage scenarios. As seen in the image, the tool calculated a recommended user count of 55 for a single Spark and 120 for dual Sparks.</p>
<p><strong>Single Spark benchmark results:</strong></p>
<p>&nbsp;</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-31 hover-type-none"><img decoding="async" width="1024" height="579" title="oss_single_final_benchmark_1" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1024x579.webp" alt class="img-responsive wp-image-1811" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-32 hover-type-none"><img decoding="async" width="1024" height="579" title="oss_single_final_benchmark_2" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1024x579.webp" alt class="img-responsive wp-image-1812" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-226"><p>&nbsp;</p>
<p><strong>Double Spark benchmark results:</strong></p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-33 hover-type-none"><img decoding="async" width="1024" height="579" title="oss_single_final_benchmark_1" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1024x579.webp" alt class="img-responsive wp-image-1811" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_1.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-34 hover-type-none"><img decoding="async" width="1024" height="579" title="oss_single_final_benchmark_2" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1024x579.webp" alt class="img-responsive wp-image-1812" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-300x170.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-400x226.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-600x339.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-768x434.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-800x452.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1024x579.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1200x678.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2-1536x868.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/oss_single_final_benchmark_2.webp 1850w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/local-gpt-oss-120b-serving-on-dgx-spark-with-sparkrun/">Local GPT-OSS 120B Serving on DGX Spark with sparkrun</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>GPU-Accelerated Data Science and Machine Learning on DGX Spark</title>
		<link>https://blog.openzeka.com/en/gpu-accelerated-data-science-and-machine-learning-on-dgx-spark/</link>
		
		<dc:creator><![CDATA[Enhar]]></dc:creator>
		<pubDate>Wed, 19 Aug 2026 13:08:05 +0000</pubDate>
				<category><![CDATA[Generative AI]]></category>
		<guid isPermaLink="false">https://blog.openzeka.com/en/?p=1786</guid>

					<description><![CDATA[<p>In this tutorial, you will accelerate data science and  ... Continue Reading→</p>
<p>The post <a href="https://blog.openzeka.com/en/gpu-accelerated-data-science-and-machine-learning-on-dgx-spark/">GPU-Accelerated Data Science and Machine Learning on DGX Spark</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></description>
										<content:encoded><![CDATA[<div class="fusion-fullwidth fullwidth-box fusion-builder-row-12 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling" style="--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;" ><div class="fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap" style="max-width:1331.2px;margin-left: calc(-4% / 2 );margin-right: calc(-4% / 2 );"><div class="fusion-layout-column fusion_builder_column fusion-builder-column-11 fusion_builder_column_1_1 1_1 fusion-flex-column" style="--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;"><div class="fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column"><div class="fusion-text fusion-text-227"><p>In this tutorial, you will accelerate data science and machine learning workflows on a DGX Spark using its GPU.</p>
<p>The Spark can be used as a standalone computer by connecting a monitor and keyboard, or as a remote server accessed from another computer. In this tutorial, we will connect to the Spark remotely and install the necessary software to run our workloads.</p>
<p>We will use pandas as our data processing library and scikit-learn as our machine learning library. These two libraries are foundational to the data science and machine learning ecosystem. In this tutorial, we will accelerate their workflows on the GPU using CUDA-X.</p>
<p>CUDA-X is the umbrella name for NVIDIA&#8217;s collection of GPU acceleration libraries. Within this collection, the set containing data science libraries is called RAPIDS. In this tutorial, we will use cuDF from the RAPIDS library suite to accelerate pandas, and cuML to accelerate scikit-learn.</p>
<p>This tutorial consists of three parts:</p>
<ul>
<li><strong>Setup:</strong> Installing Miniconda, creating conda environments, and cloning the playbook repository</li>
<li><strong>Running the Notebooks:</strong> Running two notebooks in both CPU and GPU environments</li>
<li><strong>Comparison:</strong> Comparing CPU and GPU performance cell by cell</li>
</ul>
<hr />
<h2>Setup</h2>
<h3>Connecting to the Spark</h3>
<p>If you are connecting to the Spark remotely for the first time, you need to find its IP address. Connect a monitor and keyboard to the Spark, log in, and run the following command in the terminal:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-161 > .CodeMirror, .fusion-syntax-highlighter-161 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-161 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_161" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_161" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_161" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ip route get 1.1.1.1 | grep -oP &#8216;src KS+&#8217;</textarea></div><div class="fusion-text fusion-text-228"><p>&nbsp;</p>
<p>The command outputs the IP address of the Spark&#8217;s default network interface:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-162 > .CodeMirror, .fusion-syntax-highlighter-162 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-162 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_162" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_162" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_162" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">192.168.1.162</textarea></div><div class="fusion-text fusion-text-229"><p>&nbsp;</p>
<p>Note this address; you will use it in place of <code></code> throughout this tutorial. Alternatively, you can find the IP address by checking the NVIDIA Sync application.</p>
<p>Make sure your computer is connected to the same network as the Spark. Then, open a terminal on your computer and connect to the Spark via SSH:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-163 > .CodeMirror, .fusion-syntax-highlighter-163 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-163 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_163" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_163" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_163" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ssh nvidia@</textarea></div><div class="fusion-text fusion-text-230"><p>&nbsp;</p>
<p>On the first connection, you will see a fingerprint warning. Type <code>yes</code> and press Enter. Then, when prompted for a password, enter the Spark&#8217;s password:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-164 > .CodeMirror, .fusion-syntax-highlighter-164 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-164 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_164" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_164" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_164" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">The authenticity of host &#8216;192.168.1.162 (192.168.1.162)&#8217; can&#8217;t be established.
ED25519 key fingerprint is SHA256:S6EECYc6Pw2aLoLmhblFZ0QEoeVtJP41jJ5IYsdOmMM.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added &#8216;192.168.1.162&#8217; (ED25519) to the list of known hosts.
nvidia@192.168.1.162&#8217;s password:
Welcome to NVIDIA DGX Spark Version 7.5.0 (GNU/Linux 6.17.0-1026-nvidia aarch64)</p>
<p>System information as of Mon Aug 3 07:38:29 2026 UTC</p>
<p>System load: 0.50 Temperature: 35.0 C
Usage of /: 27% of 3.67TB Processes: 575
Memory usage: 3% Users logged in: 1
Swap usage: 0% IPv4 address for enP7s7: 192.168.1.162</textarea></div><div class="fusion-text fusion-text-231"><p>&nbsp;</p>
<p>Once connected, the Spark will start accepting commands from this terminal. Throughout this tutorial, you will enter all commands in this terminal on your computer.</p>
<hr />
<h3>Installing Miniconda</h3>
<p>Conda is a package manager that manages Python packages and environments. If conda is not installed on the Spark, you need to install Miniconda. Run the following commands:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-165 > .CodeMirror, .fusion-syntax-highlighter-165 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-165 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_165" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_165" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_165" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~
wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O miniconda.sh
bash miniconda.sh -b -p ~/miniconda3
rm miniconda.sh</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-166 > .CodeMirror, .fusion-syntax-highlighter-166 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-166 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_166" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_166" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_166" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">PREFIX=/home/nvidia/miniconda3
Unpacking bootstrapper&#8230;
Unpacking payload&#8230;</p>
<p>Installing base environment&#8230;</p>
<p>Preparing transaction: &#8230;working&#8230; done
Executing transaction: &#8230;working&#8230; done
installation finished.</textarea></div><div class="fusion-text fusion-text-232"><p>&nbsp;</p>
<p>Verify the installation:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-167 > .CodeMirror, .fusion-syntax-highlighter-167 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-167 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_167" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_167" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_167" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">~/miniconda3/bin/conda &#8211;version</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-168 > .CodeMirror, .fusion-syntax-highlighter-168 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-168 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_168" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_168" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_168" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">conda 26.5.3</textarea></div><div class="fusion-text fusion-text-233"><p>&nbsp;</p>
<p>Add conda to your PATH:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-169 > .CodeMirror, .fusion-syntax-highlighter-169 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-169 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_169" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_169" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_169" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">echo &#8216;export PATH=&#8221;$HOME/miniconda3/bin:$PATH&#8221;&#8216; &gt;&gt; ~/.bashrc
source ~/.bashrc</textarea></div><div class="fusion-text fusion-text-234"><hr />
<h3>Creating Conda Environments</h3>
<p>In this tutorial, we will use two separate conda environments on the Spark. The first contains standard data science packages — reflecting a typical setup you may already have. Let&#8217;s call this the CPU environment. The second contains the RAPIDS libraries. Let&#8217;s call this the GPU environment.</p>
<p><strong>Create the CPU environment</strong> (standard data science packages, no RAPIDS):</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-170 > .CodeMirror, .fusion-syntax-highlighter-170 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-170 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_170" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_170" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_170" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">conda create -n cpu-baseline -c conda-forge python=3.12 jupyter pandas numpy scikit-learn hdbscan umap-learn kaggle matplotlib -y</textarea></div><div class="fusion-text fusion-text-235"><p>&nbsp;</p>
<p><strong>Create the GPU environment</strong> (includes RAPIDS libraries):</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-171 > .CodeMirror, .fusion-syntax-highlighter-171 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-171 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_171" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_171" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_171" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">conda create -n rapids-test -c rapidsai -c conda-forge -c nvidia rapids=26.06 python=3.12 &#8216;cuda-version=13.0&#8217; jupyter hdbscan umap-learn -y</textarea></div><div class="fusion-text fusion-text-236"><p>&nbsp;</p>
<p>This command installs the RAPIDS 26.06 package from the <code>rapidsai</code> channel, CUDA 13.0 runtime from the <code>nvidia</code> channel, and other packages from the <code>conda-forge</code> channel.</p>
<p>After installation completes, verify the package versions:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-172 > .CodeMirror, .fusion-syntax-highlighter-172 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-172 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_172" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_172" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_172" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">conda run -n cpu-baseline python -c &#8220;import pandas, sklearn, hdbscan, umap; print(f&#8217;pandas {pandas.__version__}, scikit-learn {sklearn.__version__}, hdbscan OK, umap {umap.__version__}&#8217;)&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-173 > .CodeMirror, .fusion-syntax-highlighter-173 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-173 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_173" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_173" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_173" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">pandas 3.0.5, scikit-learn 1.9.0, hdbscan OK, umap 0.5.12</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-174 > .CodeMirror, .fusion-syntax-highlighter-174 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-174 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_174" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_174" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_174" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">conda run -n rapids-test python -c &#8220;import cudf, cuml; print(f&#8217;cudf {cudf.__version__}, cuml {cuml.__version__}&#8217;)&#8221;</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-175 > .CodeMirror, .fusion-syntax-highlighter-175 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-175 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_175" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_175" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_175" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">cudf 26.06.01, cuml 26.06.00</textarea></div><div class="fusion-text fusion-text-237"><hr />
<h3>Cloning the Playbook Repository</h3>
<p>NVIDIA shares its DGX Spark playbooks on its <strong><a style="color: #00bf5c;" href="https://build.nvidia.com/spark/cuda-x-data-science">website</a></strong> and <a href="https://github.com/NVIDIA/dgx-spark-playbooks/tree/main/nvidia/cuda-x-data-science"><strong style="color: #00bf5c;">GitHub repository</strong></a> . The CUDA-X Data Science playbook there contains the two notebooks we will use.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-176 > .CodeMirror, .fusion-syntax-highlighter-176 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-176 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_176" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_176" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_176" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~
git clone https://github.com/NVIDIA/dgx-spark-playbooks.git</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-177 > .CodeMirror, .fusion-syntax-highlighter-177 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-177 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_177" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_177" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_177" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">Cloning into &#8216;dgx-spark-playbooks&#8217;&#8230;</textarea></div><div class="fusion-text fusion-text-238"><p>&nbsp;</p>
<p>Let&#8217;s look at the directory containing the notebooks:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-178 > .CodeMirror, .fusion-syntax-highlighter-178 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-178 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_178" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_178" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_178" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">ls -la ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets/</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-179 > .CodeMirror, .fusion-syntax-highlighter-179 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-179 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_179" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_179" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_179" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">total 284
drwxrwxr-x 2 nvidia nvidia 4096 Aug 3 07:41 .
drwxrwxr-x 3 nvidia nvidia 4096 Aug 3 07:41 ..
-rw-rw-r&#8211; 1 nvidia nvidia 36324 Aug 3 07:41 cudf_pandas_demo.ipynb
-rw-rw-r&#8211; 1 nvidia nvidia 242100 Aug 3 07:41 cuml_sklearn_demo.ipynb</textarea></div><div class="fusion-text fusion-text-239"><p>Two notebooks are available:</p>
<ul>
<li><strong><code>cudf_pandas_demo.ipynb</code></strong> — Accelerates pandas workflows with cuDF</li>
<li><strong><code>cuml_sklearn_demo.ipynb</code> </strong>— Accelerates scikit-learn workflows with cuML</li>
</ul>
<hr />
<h3>Kaggle API Key</h3>
<p>The first notebook downloads a 4.76 GB dataset from Kaggle. To do this, you need a Kaggle API key (<code>kaggle.json</code>). If you don&#8217;t have a Kaggle account, sign up at <a style="color: #00bf5c;" href="https://www.kaggle.com">kaggle.com</a> and create an API key.</p>
<p>Copy the <code>kaggle.json</code> file from your computer to the Spark. Open a <strong>separate terminal</strong> on your computer and run the following command:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-180 > .CodeMirror, .fusion-syntax-highlighter-180 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-180 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_180" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_180" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_180" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">scp ~/Downloads/kaggle.json nvidia@:~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets/</textarea></div><div class="fusion-text fusion-text-240"><p>&nbsp;</p>
<p>Then return to your SSH terminal and set the file permissions:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-181 > .CodeMirror, .fusion-syntax-highlighter-181 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-181 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_181" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_181" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_181" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">chmod 600 ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets/kaggle.json</textarea></div><div class="fusion-text fusion-text-241"><hr />
<h3>Preparing CPU Copies</h3>
<p>To enable comparison, we will create a CPU copy of each notebook. The only difference in these copies is that the acceleration commands are commented out. All other code remains the same.</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-182 > .CodeMirror, .fusion-syntax-highlighter-182 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-182 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_182" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_182" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_182" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
cp cudf_pandas_demo.ipynb cudf_pandas_demo_cpu.ipynb
cp cuml_sklearn_demo.ipynb cuml_sklearn_demo_cpu.ipynb</textarea></div><div class="fusion-text fusion-text-242"><p>&nbsp;</p>
<p>Now comment out the <code>%load_ext</code> line in each copy:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-183 > .CodeMirror, .fusion-syntax-highlighter-183 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-183 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_183" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_183" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_183" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">sed -i &#8216;s/%load_ext cudf.pandas/# %load_ext cudf.pandas/&#8217; cudf_pandas_demo_cpu.ipynb
sed -i &#8216;s/%load_ext cuml.accel/# %load_ext cuml.accel/&#8217; cuml_sklearn_demo_cpu.ipynb</textarea></div><div class="fusion-text fusion-text-243"><hr />
<h2>What Are cuDF and cuML?</h2>
<p>In this section, before diving into the practical work, we will explain cuDF and cuML.</p>
<h3>cuDF — pandas Acceleration</h3>
<p>cuDF offers a &#8220;pandas accelerator mode&#8221; called <code>cudf.pandas</code>. This mode is activated by adding a single acceleration command to a Jupyter notebook:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-184 > .CodeMirror, .fusion-syntax-highlighter-184 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-184 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_184" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_184" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_184" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">%load_ext cudf.pandas</textarea></div><div class="fusion-text fusion-text-244"><p>&nbsp;</p>
<p>When this command is run, the <code>import pandas as pd</code> statement loads a &#8220;proxy module&#8221; instead of the normal pandas. This proxy attempts to run every pandas operation on the GPU (via cuDF) first; if the operation is not supported on the GPU, it automatically falls back to the CPU (pandas) and synchronizes the data in the background.</p>
<p><strong>Important:</strong> This mechanism requires no changes to user code. All pandas commands such as <code>pd.read_csv()</code>, <code>df.merge()</code>, and <code>df.groupby()</code> work as-is — they just run on the GPU.</p>
<h3>cuML — scikit-learn Acceleration</h3>
<p>cuML offers a &#8220;scikit-learn accelerator mode&#8221; called <code>cuml.accel</code>. This mode is also activated with a single acceleration command:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-185 > .CodeMirror, .fusion-syntax-highlighter-185 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-185 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_185" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_185" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_185" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">%load_ext cuml.accel</textarea></div><div class="fusion-text fusion-text-245"><p>&nbsp;</p>
<p>This command intercepts the import operations of the <code>sklearn</code>, <code>umap-learn</code>, and <code>hdbscan</code> modules. When you create an estimator (e.g., <code>LinearSVC</code>), a &#8220;proxy estimator&#8221; is created instead of the actual estimator. If cuML has a GPU implementation for that algorithm, the operation is dispatched to the GPU; otherwise, it automatically falls back to the CPU (scikit-learn).</p>
<hr />
<h2>Notebook 1: Accelerating pandas with cuDF</h2>
<p>&nbsp;</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-35 hover-type-none"><img decoding="async" width="1024" height="576" title="nb1-overview" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-1024x576.webp" alt class="img-responsive wp-image-1824" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-300x169.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-400x225.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-600x338.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-768x432.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-800x450.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-1024x576.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-1200x675.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview-1536x864.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-overview.webp 1920w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-246"><p>&nbsp;</p>
<p>This notebook contains pandas data processing operations: loading, string operations, and grouping.</p>
<p>The cuDF accelerator mode (cudf.pandas) routes these operations to the GPU without requiring any code changes.</p>
<h3>Running the Notebooks</h3>
<p>We will run the notebooks in headless mode using jupyter nbconvert &#8211;execute. This method automatically runs all cells in the notebook and saves the outputs into the notebook file.</p>
<p>Running on GPU (RAPIDS environment, acceleration command active):</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-186 > .CodeMirror, .fusion-syntax-highlighter-186 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-186 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_186" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_186" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_186" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
conda run -n rapids-test jupyter nbconvert &#8211;execute &#8211;to notebook
&#8211;output cudf_pandas_demo_gpu_executed.ipynb cudf_pandas_demo.ipynb</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-187 > .CodeMirror, .fusion-syntax-highlighter-187 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-187 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_187" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_187" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_187" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[NbConvertApp] Converting notebook cudf_pandas_demo.ipynb to notebook[NbConvertApp] Writing 136623 bytes to cudf_pandas_demo_gpu_executed.ipynb</textarea></div><div class="fusion-text fusion-text-247"><p>&nbsp;</p>
<p><strong>Running on CPU</strong> (standard environment, acceleration command disabled):</p>
<p>We are running the CPU copy we prepared in the &#8220;Preparing CPU Copies&#8221; section:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-188 > .CodeMirror, .fusion-syntax-highlighter-188 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-188 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_188" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_188" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_188" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
conda run -n cpu-baseline jupyter nbconvert &#8211;execute &#8211;to notebook
&#8211;output cudf_pandas_demo_cpu_executed.ipynb cudf_pandas_demo_cpu.ipynb</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-189 > .CodeMirror, .fusion-syntax-highlighter-189 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-189 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_189" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_189" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_189" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[NbConvertApp] Converting notebook cudf_pandas_demo_cpu.ipynb to notebook[NbConvertApp] Writing 127995 bytes to cudf_pandas_demo_cpu_executed.ipynb</textarea></div><div class="fusion-text fusion-text-248"><p>&nbsp;</p>
<hr />
<p>&nbsp;</p>
<h2>Notebook 2: Accelerating scikit-learn with cuML</h2>
<p>&nbsp;</p>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-36 hover-type-none"><img decoding="async" width="1024" height="576" title="nb2-overview" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-1024x576.webp" alt class="img-responsive wp-image-1830" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-200x113.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-300x169.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-400x225.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-600x338.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-768x432.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-800x450.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-1024x576.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-1200x675.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview-1536x864.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-overview.webp 1920w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-249"><p>&nbsp;</p>
<p>This notebook contains various machine learning operations with scikit-learn: classification, clustering, and dimensionality reduction.</p>
<p>The cuML accelerator mode (cuml.accel) routes these operations to the GPU without requiring any code changes.</p>
<h3>Running</h3>
<p><strong>Running on GPU:</strong></p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-190 > .CodeMirror, .fusion-syntax-highlighter-190 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-190 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_190" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_190" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_190" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
conda run -n rapids-test jupyter nbconvert &#8211;execute &#8211;to notebook
&#8211;output cuml_sklearn_demo_gpu_executed.ipynb cuml_sklearn_demo.ipynb</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-191 > .CodeMirror, .fusion-syntax-highlighter-191 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-191 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_191" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_191" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_191" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[NbConvertApp] Converting notebook cuml_sklearn_demo.ipynb to notebook[NbConvertApp] Writing 358291 bytes to cuml_sklearn_demo_gpu_executed.ipynb</textarea></div><div class="fusion-text fusion-text-250"><p><strong>Running on CPU:</strong></p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-192 > .CodeMirror, .fusion-syntax-highlighter-192 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-192 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_192" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_192" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_192" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
conda run -n cpu-baseline jupyter nbconvert &#8211;execute &#8211;to notebook
&#8211;output cuml_sklearn_demo_cpu_executed.ipynb cuml_sklearn_demo_cpu.ipynb</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-193 > .CodeMirror, .fusion-syntax-highlighter-193 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-193 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_193" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_193" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_193" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[NbConvertApp] Converting notebook cuml_sklearn_demo_cpu.ipynb to notebook[NbConvertApp] Writing 353326 bytes to cuml_sklearn_demo_cpu_executed.ipynb</textarea></div><div class="fusion-text fusion-text-251"><hr />
<h2>Viewing Executed Notebooks with Jupyter</h2>
<p>Executed notebooks have all cell outputs (timings, tables, charts) embedded within them. You can view them in your browser through a Jupyter server.</p>
<p>Start a Jupyter server on the Spark:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-194 > .CodeMirror, .fusion-syntax-highlighter-194 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-194 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_194" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_194" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_194" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
conda run -n cpu-baseline jupyter notebook &#8211;no-browser &#8211;ip=0.0.0.0 &#8211;port=8888</textarea></div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-195 > .CodeMirror, .fusion-syntax-highlighter-195 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-195 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_195" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_195" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_195" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/txt">[I 2026-08-03 07:56:01.000 ServerApp] jupyter_serverterminals | extension was successfully linked.[I 2026-08-03 07:56:01.000 ServerApp] jupyter_events | extension was successfully linked.[I 2026-08-03 07:56:01.000 ServerApp] jupyter_server | extension was successfully linked.[I 2026-08-03 07:56:02.000 ServerApp] Serving notebooks from local directory: /home/nvidia/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets[I 2026-08-03 07:56:02.000 ServerApp] Jupyter Server 2.16.0 is running at:[I 2026-08-03 07:56:02.000 ServerApp] http://192.168.1.162:8888/tree?token=7c7ea2a440905886912cf036999d93409a8401990257ac97</textarea></div><div class="fusion-text fusion-text-252"><p>Open the URL shown in the output (including the token) in your computer&#8217;s browser. In the file list on the left, you will see four executed notebooks:</p>
<ul>
<li><code>cudf_pandas_demo_gpu_executed.ipynb</code> — GPU run (cuDF)</li>
<li><code>cudf_pandas_demo_cpu_executed.ipynb</code> — CPU run (cuDF)</li>
<li><code>cuml_sklearn_demo_gpu_executed.ipynb</code> — GPU run (cuML)</li>
<li><code>cuml_sklearn_demo_cpu_executed.ipynb</code> — CPU run (cuML)</li>
</ul>
<p>Click on any of them to view cell outputs, <code>%%time</code> timings, tables, and charts. You can open the CPU and GPU notebooks side by side in two browser tabs for comparison.</p>
<hr />
<h2>Comparison</h2>
<p>In this section, we will compare important cells side by side and evaluate the results. In the screenshots, the CPU run is on the left and the GPU run is on the right.</p>
<hr />
<h3>Notebook 1: cuDF / pandas</h3>
<h4>In[4] — Acceleration Command</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-37 hover-type-none"><img decoding="async" width="1024" height="248" title="nb1-cell-in4" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-1024x248.webp" alt class="img-responsive wp-image-1820" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-200x49.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-300x73.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-400x97.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-600x146.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-768x186.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-800x194.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-1024x248.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-1200x291.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4-1536x372.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in4.webp 1835w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-253"><p>&nbsp;</p>
<p>This cell is the only difference between the CPU and GPU runs. In the GPU notebook, the <strong><code>%load_ext cudf.pandas</code> </strong>command replaces pandas with a proxy module. In the CPU notebook, this line is commented out (<code># <strong>%load_ext cudf.pandas</strong></code>), so normal pandas is used.</p>
<hr />
<h4>In[6] — Data Loading (4.76 GB)</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-38 hover-type-none"><img decoding="async" width="1024" height="122" title="nb1-cell-in6" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-1024x122.webp" alt class="img-responsive wp-image-1821" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-200x24.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-300x36.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-400x48.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-600x72.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-768x92.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-800x96.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-1024x122.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-1200x143.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6-1536x184.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in6.webp 1833w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-254"><p>&nbsp;</p>
<p>Loads the 4.76 GB <code>job_summary.csv</code> file. This file contains the text summaries of 1.1 million LinkedIn job postings.</p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>25.7s</td>
<td>3.87s</td>
<td><strong>6.6x</strong></td>
</tr>
</tbody>
</table>
<p>CPU pandas reads the dataset with a single thread, which is why it is slow. cuDF&#8217;s GPU CSV reader, on the other hand, loads the data in parallel into GPU memory. For a 4.76 GB file, the time difference is very significant.</p>
<hr />
<h4>In[7-8] — Data Loading + String Operation</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-39 hover-type-none"><img decoding="async" width="1024" height="258" title="nb1-cell-in7-8" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-1024x258.webp" alt class="img-responsive wp-image-1822" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-200x50.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-300x76.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-400x101.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-600x151.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-768x193.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-800x201.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-1024x258.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-1200x302.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8-1536x387.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in7-8.webp 1847w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-255"><p>&nbsp;</p>
<p>Loads two smaller CSV files (<code>job_skills.csv</code> 642 MB, <code>linkedin_job_postings.csv</code> 397 MB, totaling ~1 GB) and then calculates the length of 1.1 million strings. Calculating string length is a common operation when working with text data.</p>
<p><strong>In[7] — Data loading (2 CSVs, ~1 GB):</strong></p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>6.46s</td>
<td>252ms</td>
<td><strong>25.6x</strong></td>
</tr>
</tbody>
</table>
<p><strong>In[8] — String operation (<code>str.len()</code> on 1.1M strings):</strong></p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>246ms</td>
<td>37.2ms</td>
<td><strong>6.6x</strong></td>
</tr>
</tbody>
</table>
<p>In string operations, the GPU processes thousands of strings simultaneously rather than calculating each string&#8217;s length one by one.</p>
<hr />
<h4>In[11] — Aggregation (groupby + sort)</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-40 hover-type-none"><img decoding="async" width="1024" height="434" title="nb1-cell-in11" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-1024x434.webp" alt class="img-responsive wp-image-1823" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-200x85.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-300x127.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-400x169.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-600x254.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-768x325.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-800x339.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-1024x434.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-1200x508.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11-1536x650.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb1-cell-in11.webp 1835w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-256"><p>&nbsp;</p>
<p>Groups 1.1 million rows by <code>job_title</code> and <code>job_location</code> columns, calculates the mean, and sorts by three columns.</p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>1.59s</td>
<td>199ms</td>
<td><strong>8.0x</strong></td>
</tr>
</tbody>
</table>
<p>&nbsp;</p>
<p>The GPU is significantly faster in this operation.</p>
<hr />
<h3>Notebook 2: cuML / scikit-learn</h3>
<h4>In[2] — Acceleration Command</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-41 hover-type-none"><img decoding="async" width="1024" height="202" title="nb2-cell-in2" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-1024x202.webp" alt class="img-responsive wp-image-1825" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-200x39.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-300x59.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-400x79.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-600x118.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-768x151.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-800x158.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-1024x202.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-1200x237.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2-1536x303.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in2.webp 1825w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-257"><p>&nbsp;</p>
<p>As in the previous notebook, this cell is the only difference between the CPU and GPU runs. In the GPU notebook, the <code>%load_ext cuml.accel</code> command intercepts scikit-learn/umap-learn/hdbscan imports.</p>
<hr />
<h4>In[7] — Classification: LinearSVC Training</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-42 hover-type-none"><img decoding="async" width="1024" height="222" title="nb2-cell-in7" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-1024x222.webp" alt class="img-responsive wp-image-1826" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-200x43.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-300x65.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-400x87.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-600x130.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-768x167.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-800x174.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-1024x222.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-1200x260.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7-1536x333.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in7.webp 1830w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-258"><p>&nbsp;</p>
<p>Trains a Linear Support Vector Classifier on the UCI Covertype dataset. The dataset contains 581 thousand samples and 55 features.</p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>31.9s</td>
<td>3.65s</td>
<td><strong>8.7x</strong></td>
</tr>
</tbody>
</table>
<p>&nbsp;</p>
<p>Model training is the most time-consuming step in machine learning. The GPU performs LinearSVC&#8217;s matrix operations in parallel, significantly reducing training time. This is of great importance in scenarios where you need to train multiple models, such as hyperparameter search, which involve heavy computational workloads.</p>
<hr />
<h4>In[12] — CPU Fallback: KernelDensity</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-43 hover-type-none"><img decoding="async" width="878" height="719" title="nb2-cell-in12" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12.webp" alt class="img-responsive wp-image-1827" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12-200x164.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12-300x246.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12-400x328.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12-600x491.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12-768x629.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12-800x655.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in12.webp 878w" sizes="(max-width: 640px) 100vw, 878px" /></span></div><div class="fusion-text fusion-text-259"><blockquote>
<p>&nbsp;</p>
<p><strong>Note:</strong> This screenshot is taken from the GPU notebook only (no side-by-side comparison).</p>
</blockquote>
<p>This cell is used to demonstrate cuML&#8217;s CPU fallback mechanism. The KernelDensity algorithm does not have a GPU implementation in cuML. The notebook deliberately calls this algorithm to show that unsupported operations gracefully fall back to the CPU.</p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>1.65ms</td>
<td>9.74ms</td>
<td>0.2x</td>
</tr>
</tbody>
</table>
<hr />
<h4>In[16] — Clustering: HDBSCAN</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-44 hover-type-none"><img decoding="async" width="1024" height="237" title="nb2-cell-in16" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-1024x237.webp" alt class="img-responsive wp-image-1828" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-200x46.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-300x69.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-400x93.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-600x139.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-768x178.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-800x185.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-1024x237.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-1200x278.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16-1536x356.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in16.webp 1827w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-260"><p>&nbsp;</p>
<p>HDBSCAN is a density-based clustering algorithm. Here it runs on synthetic data (20 thousand samples, 100 features, 5 clusters).</p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>15.9s</td>
<td>216ms</td>
<td><strong>73.6x</strong></td>
</tr>
</tbody>
</table>
<p>This is the largest speedup recorded during this tutorial. HDBSCAN involves computationally intensive steps such as density calculations and minimum spanning tree construction. These steps can be accelerated dozens of times by leveraging the GPU&#8217;s parallel processing power.</p>
<hr />
<h4>In[25] — Dimensionality Reduction: UMAP</h4>
</div><div class="fusion-image-element " style="--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);"><span class=" fusion-imageframe imageframe-none imageframe-45 hover-type-none"><img decoding="async" width="1024" height="179" title="nb2-cell-in25" src="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-1024x179.webp" alt class="img-responsive wp-image-1829" srcset="https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-200x35.webp 200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-300x53.webp 300w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-400x70.webp 400w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-600x105.webp 600w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-768x135.webp 768w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-800x140.webp 800w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-1024x179.webp 1024w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-1200x210.webp 1200w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25-1536x269.webp 1536w, https://blog.openzeka.com/en/wp-content/uploads/2026/08/nb2-cell-in25.webp 1827w" sizes="(max-width: 640px) 100vw, 1024px" /></span></div><div class="fusion-text fusion-text-261"><p>UMAP reduces high-dimensional data to a lower dimension (here, 561D → 2D). The UCI HAR dataset (7,352 samples, 561 features) contains human activities recorded from smartphone sensors.</p>
<table>
<thead>
<tr>
<th>CPU</th>
<th>GPU</th>
<th>Speedup</th>
</tr>
</thead>
<tbody>
<tr>
<td>13.9s</td>
<td>367ms</td>
<td><strong>37.9x</strong></td>
</tr>
</tbody>
</table>
<p>UMAP involves building a k-NN graph and various optimization steps. These steps provide significant parallelization advantages on the GPU.</p>
<hr />
<h2>Conclusion</h2>
<p>In this tutorial, we accelerated pandas and scikit-learn workflows on the DGX Spark using CUDA-X libraries (cuDF, cuML) on the GPU. We ran two notebooks in both CPU and GPU environments to make a real performance comparison.</p>
<p>The results obtained:</p>
<table>
<thead>
<tr>
<th><strong>Operation Category</strong></th>
<th><strong>Typical Speedup</strong></th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>CSV loading</strong></td>
<td><strong>6-25x</strong></td>
</tr>
<tr>
<td><strong>String operations</strong></td>
<td><strong>6x</strong></td>
</tr>
<tr>
<td><strong>groupby + sort</strong></td>
<td><strong>8x</strong></td>
</tr>
<tr>
<td><strong>LinearSVC training</strong></td>
<td><strong>8.7x</strong></td>
</tr>
<tr>
<td><strong>HDBSCAN clustering</strong></td>
<td><strong>73.6x</strong></td>
</tr>
<tr>
<td><strong>UMAP dimensionality reduction</strong></td>
<td><strong>37.9x</strong></td>
</tr>
</tbody>
</table>
<p>&nbsp;</p>
<p>The most striking results were achieved with the HDBSCAN (73.6x) and UMAP (37.9x) algorithms. This is because these algorithms involve intensive mathematical computations, which is where the GPU&#8217;s parallel processing power provides a significant advantage.</p>
<p>These results show that by adding a single command to your pandas and scikit-learn code running on the Spark, you can achieve significant speedups. We also recommend trying these notebooks on your own computer. This is because whether GPU acceleration is present or not, the performance of the DGX Spark&#8217;s 20-core ARM CPU may surprise you!</p>
<hr />
<h2>Shutdown</h2>
<p>When you are done, stop the Jupyter server with <code>Ctrl+C</code>. The conda environments you created on the Spark (<code>cpu-baseline</code> and <code>rapids-test</code>), the executed notebooks, and the downloaded datasets remain on disk — nothing is deleted.</p>
<p>To view the notebooks again, restart the Jupyter server:</p>
</div><style type="text/css" scopped="scopped">.fusion-syntax-highlighter-196 > .CodeMirror, .fusion-syntax-highlighter-196 > .CodeMirror .CodeMirror-gutters {background-color:#000000;}</style><div class="fusion-syntax-highlighter-container fusion-syntax-highlighter-196 fusion-syntax-highlighter-theme-dark" style="opacity:0;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:14px;border-width:1px;border-style:solid;border-color:rgba(242,243,245,0);"><div class="syntax-highlighter-copy-code"><span class="syntax-highlighter-copy-code-title" data-id="fusion_syntax_highlighter_196" style="font-size:14px;">Copy to Clipboard</span></div><label for="fusion_syntax_highlighter_196" class="screen-reader-text">Syntax Highlighter</label><textarea class="fusion-syntax-highlighter-textarea" id="fusion_syntax_highlighter_196" data-readOnly="nocursor" data-lineNumbers="" data-lineWrapping="" data-theme="oceanic-next" data-mode="text/x-sh">cd ~/dgx-spark-playbooks/nvidia/cuda-x-data-science/assets
conda run -n cpu-baseline jupyter notebook &#8211;no-browser &#8211;ip=0.0.0.0 &#8211;port=8888</textarea></div><div class="fusion-text fusion-text-262"><p>&nbsp;</p>
<p>When the server starts, open the URL containing the token shown in the terminal in your browser.</p>
</div></div></div></div></div>
<p>The post <a href="https://blog.openzeka.com/en/gpu-accelerated-data-science-and-machine-learning-on-dgx-spark/">GPU-Accelerated Data Science and Machine Learning on DGX Spark</a> appeared first on <a href="https://blog.openzeka.com/en">OpenZeka EN Blog</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
