What a DGX Spark is actually for
Post 1 said rent the GPUs. The asterisk: renting is for inference. For finetuning, a $3k box with 128GB beats the math - and the meter changes what you try.
The ideas here are mine - AI helps edit and sharpen the writing, and I do the final pass.
Post 1 said rent the GPUs. So why is there a DGX Spark sitting next to my server?
Because renting is for inference, and the thing I actually needed a GPU in the house for is finetuning. Different job, different math, and the Spark is weirdly good at exactly one of them.
I got the ASUS GX10 version when it was $2,999 - same GB10 chip and 128GB of unified memory as NVIDIA’s $3,999 Founders box, just without the badge and 1TB ssd instead of 4TB. That 128GB is the entire reason to buy one. It’s the most VRAM on a CUDA GPU you can get for the money right now, by a wide margin - a 5090 gives you 32GB at the same $4k street price, and the 96GB RTX Pro 6000 is $12k before you build a machine around it.
What mine does in production: the voice side of Era - Qwen TTS for voice cloning, plus Whisper turbo for word-level timestamps so the app can highlight each word as it reads it. Always-on duty. Not glamorous, doesn’t need to be fast, needs to be there.
The real value though is finetuning. Lekton’s voice and correction models get finetuned here, then deployed to the edge. Lekton is a Wispr Flow with a privacy focus - everything stays on your device, your voice never gets sent off for training - which means its models have to be small enough to run on a Macbook. Same box also finetunes some of Crucible’s mesh and rig generation models. This is the workload the Spark’s shape is right for: training wants full-precision weights plus optimizer state plus activations, so memory is the wall, not speed.
Also nice: the Spark has a Mellanox ConnectX-7 built in. I put a ConnectX-5 in my main Xeon server and cabled the two directly. The CX-5 can’t make full use of the CX-7, but it’s still 25Gb against normal gigabit ethernet, and you feel it, because every training run starts with copying a dataset over and ends with copying checkpoints back. It also makes the GX10’s one real spec cut a non-issue: it ships with a 1TB SSD where the $3,999 Founders box has 4TB, but all my storage lives on the Xeon server anyway.
What didn’t work: LLMs, mostly. I tried finetuning gpt-oss-120b to write children’s stories for Era. Mechanically it works - eats most of the 128GB - and after about five iterations at 8-18 hours each, the verdict was the output just isn’t intelligent enough to ship. Inference depends on the architecture: MoE runs fine - gpt-oss-120b does ~64 tok/s here, since only ~5B of its params are active per token, and for a while it actually served as Era’s pipeline orchestrator from this box - but dense Qwen models crawl at 10-20 tok/s on this bandwidth. I took gpt-oss down in the end anyway: calling DeepSeek, Grok or Muse Spark over an API is cheaper, faster and more intelligent for that work. Post 1 rule holding: if the job needs a big general model, don’t own it, call it.
That’s also my answer to the most common take on the Spark - not worth it, too slow for local LLMs. Half true: dense models crawl, MoE is actually usable. But both versions grade the box on the wrong job. The value was never local inference. The Spark is 128GB you can train against for $3,000 - a training machine that happens to also run my voice models on the side.
One caveat though: it’s ARM. The GB10 is aarch64 with CUDA 13, and most of the Python ML ecosystem ships x86 wheels built against CUDA 12. Microsoft’s TRELLIS ships an amd64-only image - you can get it running on ARM but it’s real effort (there’s a community fork now). torchaudio isn’t even in NVIDIA’s own PyTorch container for this box, and building it from source fails on missing headers until you patch things. flash-attn needs a community wheel. None of it is a dealbreaker, but it’s a lot of extra work that won’t work out of the box.
So was the gpt-oss experiment a waste? This is the part that’s hard to put in a spreadsheet. Those five failed runs were 40-90 hours of GPU time at Spark speeds - an H200 would get through each pass a lot faster, so rented it’d cost less than the $150-300 that time naively works out to, but it’s still a bill. I might not have even done it if I think about the cost. And the Spark being slow doesn’t actually cost me anything - I’m not in a hurry, the runs continue while I sleep. Every training run on RunPod is additional money spent, and every run on the Spark is money already well spent. The meter changes what you’re willing to try - same friction argument as post 1, pointed the other way. Some of the experiments I only ran because they were free turned into the models now in production. So part of that is also psychological where owning the box does unlock a lot more experimentations.
Would I buy it again? Yes, and probably even at the $3,999 Founders price. Not as fast as a 5090, but the 5090 can’t fit most of these runs at all, and an overnight pass beats a run you can’t start.
Buy the server, rent the GPUs - and if you finetune, own one cheap slow box with a lot of VRAM. The Spark is that box.
Discuss on X.
Sources: TRELLIS amd64-only on DGX Spark, torchaudio missing from the NGC container on GB10, awesome-dgx-spark, RunPod H200 dedicated rates as of July 2026.