Knowing Where to Spend AI's Time: Building the DGX Spark Control Plane

NVIDIA ships a dashboard with the DGX Spark, and it's fine as far as it goes: GPU and memory utilization, Tailscale for remote access, a button to install software updates, a Jupyter surface if you want one. What it doesn't do is help you actually download and install models. Which model is loaded, whether it's safe to change that, whether the one you just loaded is even healthy — none of that is a question the dashboard has an opinion on. So I was having Claude Code do the work, session after session which was ok while I was learning.

The real problem wasn't the dashboard

I had Claude keeping notes in a git repo to avoid needing to rediscover the same steps every time — which model was compatible with what, how to bring one down and another one up cleanly, what to check before trusting it. That helped. But it didn't change the shape of the work. Every swap still meant opening an agent, pointing it at those notes, and having it re-derive and re-execute the same sequence through natural language and shell commands. The documentation solved “don't relearn this.” It didn't solve “stop doing this by hand.”

I needed to do this a lot, because swapping models wasn't incidental — it was the actual job. I test my own MCP work by running it against different models to see whether it holds up regardless of which one's driving it, and separately, I was testing many different models to find out which models are actually good tool callers in practice, not just on paper. That's a real, recurring need, not a one-off chore.

Eventually I reached the point where I realized: this problem wasn't going away, and I already knew exactly what it looked like, because I'd been living it for weeks. Having Claude repeat the same labor intensive task every session was me not using Claude properly. Having Claude solve the actual problem — once — was.

Writing the spec instead of re-deriving it

So I laid out everything I already knew. Models come from Hugging Face, and that means an API-key storage problem to solve. Getting a model to actually perform on the GB10 means configuring a lot of parameters correctly, not just pointing at a checkpoint. I wanted the dashboard reachable from my existing Tailnet, the way NVIDIA's tool already was. I wanted to keep the log-viewing and update-installing conveniences that tool had, rather than lose them. And model weights are large enough that I didn't want to hoard them — delete what I'm not using, redownload it when I need it again.

That was the spec. The one piece I genuinely didn't know going in was what the dashboard's own web stack should be to get something clean and responsive rather than another crusty admin panel — that's the one part I had Claude actually research, as opposed to just build. Everything else, I already knew. I handed it over and let Claude get to work.

The constraint that made “safe” the whole point

The GB10 chip in a DGX Spark has no MIG — no Multi-Instance GPU support, and not because of a driver gap. NVIDIA is explicit that it's architectural, a consequence of the chip's unified CPU/GPU memory pool. There's no hardware-isolated way to run more than one model at a time with guaranteed, separated resources.

Given that, one model at a time wasn't a limitation to work around. It was the only sane design. The job was making the swap itself safe: check what's loaded, stop it, drain the GPU, bring up the target with its own validated recipe, wait for a real health check before calling it done. Every model serves on the same port, so nothing downstream has to know a swap even happened.

What “not just works, but works well” actually took

Getting a model loaded is one thing. Getting it to be fast, reliable, and actually good at tool-calling is another, and that took real trial and error across the model and engine stack. A swap to one Gemma checkpoint quietly broke tool-calling outright — any request with a tools array failed mid-stream, because the engine's default tool-choice handling didn't match that model's actual chat format, and needed an explicit, model-specific parser flag before it worked at all. A dense model I wanted to use for its quality turned out to be nearly unusable in practice — a few tokens a second, because on this hardware decode speed tracks how much memory traffic each token costs, and a large, unquantized model is bandwidth-bound before it's anything else. Quantizing it, or trading it for a sparse-expert sibling in the same family, is what actually got it to a usable speed. And a backend flag that speeds up one quantized model will silently misbehave on another that isn't quantized the same way.

None of that is exotic. It's just real, and it's exactly the kind of thing that doesn't show up until you've actually tried to run several different models on the same box and made each one work. The dashboard was designed to allow for easy configuration and validation, which is a requirement for every new model you pull.

Fail cheap before you fail expensive

Some of that trial and error surfaced a plainer problem: certain models would just hang mid-request. That's what the stability battery is actually for — nine checks covering health, generation, coherence, long-form output, streaming, tool-calling, large-context handling, concurrency, and a sustained run. A model doesn't get promoted out of Experimental until it passes cleanly.

That's not testing for its own sake. It's a cost gate. Running a real evaluation against a model — the kind of work the dashboard exists to support in the first place — takes real time. The stability battery is cheap by comparison, and its whole job is catching a broken config before you spend the expensive time finding out the hard way during a long running MCP evaluation.

The takeaway

The lesson underneath all of it isn't really about GPUs or vLLM flags. It's about knowing where AI actually creates leverage. I was having Claude perform the same task over and over — and it was still a hassle, notes or not — instead of having it solve the actual problem once. Effectively using AI means first identifying where to focus the effort. Firing off prompts to handle tasks feels simple, but real problems require real solutions. It’s our job to be solution architects and actually problem solve. Creating my new dashboard was the right answer, and I haven't had to ask an agent to swap a model for me since.

❯ deploy(code) ✓