← BACK TO INSIGHTS
AI ENGINEERING2026-07-26•7 min read

The Homelab Grand Prix: Racing 14 Quantized LLMs on Two DGX Sparks

I put fourteen quantized LLMs on the grid of my two-node DGX Spark cluster and timed every one, top speed measured in tokens per second. A 35B model took pole at 57.1 tok/s, but the real finding sits behind it: a 122B model lands within 29 percent of that, and speculative decoding nearly doubled a 120B into the same class. Total size predicts very little. Here is the full grid, the configuration each car ran, and what actually moves the needle.


The Homelab Grand Prix: Racing 14 Quantized LLMs on Two DGX Sparks


Most benchmark posts read like a spreadsheet. Mine reads like a race, because that is what it felt like. Over the past weeks I put fourteen quantized large language models on the grid of my two-node NVIDIA DGX Spark cluster and timed every one of them. Top speed is measured in tokens per second (tok/s). The results reshuffled my whole intuition about which models are worth running at home.


Here is the full grid, what configuration each car ran, and which ones took the podium.


The circuit


I run two DGX Spark GB10 boxes, "Jean-Luc" and "Kathryn", each with 128 GB of unified memory (about 121.7 GiB usable), joined by a 200 GbE RoCE fabric. Think of that fabric as the pit lane connecting two garages: very fast, but crossing it still costs you lap time.


Four levers decide how quick a model is:


  • NVFP4 4-bit quantization is the aero package. It shrinks each model so a big engine fits under the memory weight limit at all. Most cars here run NVFP4; a few run AWQ int4 or a mixed AutoRound scheme.
  • Tensor-parallel (TP=2) is a twin-engine car spanning both garages. It is mandatory for the heavyweights, but the cross-fabric all-reduce is a chicane that bleeds top speed.
  • Multi-Token Prediction (MTP), or speculative decoding, is the DRS. The model drafts several tokens ahead and verifies them in one pass. When the draft fits in memory it is free speed; when it does not, you leave the flap closed.
  • llama-swap is the pit crew. It hot-swaps models in and out of memory so two mutually exclusive heavyweights never collide and crash the box.

    • The whole grid is served by a GB10-tuned vLLM, with a memory pre-check on both nodes before any weights load, so a car that will not fit is waved off cleanly instead of blowing the engine.


    Qualifying: the podium


    The sprint classes took every podium place. These are small-active-parameter Mixture-of-Experts (MoE) models: light, nimble, and startlingly quick.


    PosModelSizeConfigtok/s
    PoleQwen3.6-35B-A3B21.8 GiBSingle node, NVFP4, MTP spec-457.1
    P2Nemotron-Cascade-2-30B-A3B18.0 GiBSingle node, NVFP4 W4A449.1
    P3gemma-4-26B-A4B16.7 GiBTP=2 across both nodes48.2

    Qwen3.6-35B-A3B is the car to beat: only about 3B active parameters per token, DRS wide open, and it never even needs the second garage. It actively does not want it, which turns out to be one of the more useful findings of the whole meet, and I come back to it in the tuning section. None of these three is larger than 22 GiB on disk.


    The full grid


    Below the podium sit the workhorses and the heavyweights, and this is where the genuinely interesting result hides. Qwen3.5-122B-A10B laps at 40.3 tok/s. That is only 29 percent off pole, from a model with three and a half times the total parameters that also has to cross the fabric on every token.


    ModelSize on diskActiveConfigtok/s
    Qwen3.5-122B-A10B77.8 GiB~10BTP=2, MTP spec-340.3
    MiniMax-M2.7 (AWQ int4)121.5 GiBMoE 8-activeTP=236.8
    Nemotron-3-Super-120B-A12B74.8 GiB~12BTP=2, MTP spec-336.1
    HunYuan Hy3168.4 GiBMoE 8-activeTP=2, MTP spec-220.8
    Qwen3-VL-32B (vision)20.4 GiBdenseTP=218.9
    Qwen3-235B-A22B124.9 GiB~22BTP=218.6
    gemma-4-31B21.7 GiBdenseTP=218.6
    DeepSeek-V4-Flash156.7 GiBMoE 6-activeTP=2 (MTP hurts)18.5
    Qwen3-VL-235B (vision)126.0 GiB~22BTP=216.7

    MiniMax-M2.7 is the car that surprised me most. It is 121.5 GiB of weights, by far the largest model on my grid that still clears 30 tok/s, and it laps at 36.8. It is also my reminder that the quantization scheme is not a rounding error: the NVFP4 build of the same model managed only 25.4 tok/s on the same hardware, so switching to AWQ int4 bought 45 percent for free. That is why it became my default MiniMax and the model I reach for most days.


    HunYuan Hy3 is the physically largest car in the regular field at 168.4 GiB, and it still turns 20.8 tok/s. DeepSeek-V4-Flash is the interesting outlier: here the DRS closes, speculative decoding makes it slower rather than faster, so I run it plain.


    The exotics and the DNFs


    Not every entry finished at a useful speed.


  • Qwen3.5-397B, "the 400B", 12.3 tok/s. A genuine milestone: the first 400B-class car to complete a lap on just two Sparks, on a knife-edge fit with only 2 to 4 GiB of headroom. It finished. Then I retired it, there is no room to race it regularly.
  • MiniMax-M3 AutoRound 3.2-bit, 5.7 tok/s. The most exotic build of the meet, a 428B reasoning car I got running only through the checkpoint's own bespoke serving stack after every standard path stalled. It crossed the line, but at 5.7 tok/s it is a show car, not a race car.
  • Mechanical DNFs. A REAP-pruned 397B whose non-uniform expert layout the loader could not assemble, and two more MiniMax-M3 quantizations felled by a missing activation kernel and a broken upload. Clean failures, logged, moved on.

    • I kept the finish-line data and the recipes, then cleared the exotics off the grid to make room for the cars worth racing.


    Racing lessons


  • Displacement is not lap time. Active parameters are. Total parameter count barely predicts speed. What fires per token does. Qwen3.5-122B-A10B (40.3 tok/s) sits within 29 percent of the 35B pole-sitter despite being three and a half times larger; the 235B and 397B cars are slow because they activate 22B and 17B per token, not because of the number on the box.
  • The DRS is the big lever for big engines. Speculative decoding took Nemotron-3-Super-120B from 19 tok/s to 36.1, very close to doubling it. That puts a 120B model squarely in 30B-class territory. If a heavyweight feels slow, the draft setting is the first thing to reach for.
  • But no lever is universal. On DeepSeek-V4-Flash speculative decoding makes it slower, so I run that one plain. Tune every setting per car; never assume the one that helped carries over.
  • A desperate fit costs more than size does. The 397B activates fewer parameters than the 235B and still runs slower (12.3 against 18.6), because squeezing it in forced a single sequence, a 2048-token window and a memory utilisation on the edge. Comfortable fit beats clever fit.
  • A second garage helps one car and hurts another. gemma-4-26B gains from the twin-engine setup: 46.2 tok/s on one node, 48.2 across two. Qwen3.6-35B does the opposite, dropping from 27.1 to 23.6 without speculative decoding, and from 51.9 to 48.2 with it. When a model is already small and sparse there is little to split, and the fabric crossing costs more than the extra compute returns. Splitting a model across machines is a fit decision, not a speed decision.
  • 4-bit is what lets the big engines race at all. NVFP4 is the difference between a 235B or 400B model fitting under the weight limit and never starting.
  • Finishing is not winning. The 400B and the 428B reasoning model both crossed the line and both got retired. Capability you cannot serve at a usable speed is a trophy, not a tool.

    • What this says about the hardware


    The DGX Spark is not a bandwidth monster, and it is worth being blunt about that. Its unified memory runs LPDDR5X at 8533 MT/s on a 256-bit bus, roughly 273 GB/s per node. A Blackwell workstation graphics card clears well over 1 TB/s, six to seven times more. Token generation is memory-bound, so on paper that gap is the most important number on the whole spec sheet.


    And yet. Three models north of 100 billion parameters run at more than 30 tok/s on this machine: Qwen3.5-122B at 40.3, MiniMax-M2.7 at 36.8 from 121.5 GiB of weights, and Nemotron-3-Super-120B at 36.1. Above roughly 30 tok/s a model produces text faster than I read it, which is the point where I stop watching the speed and start reading the answer. That is genuinely usable, not merely impressive for the form factor.


    The trade is easy to state. I do not get workstation-card bandwidth. I get 128 GB of unified memory per node, which is what allows a 121.5 GiB model to exist on the machine at all, and sparse Mixture-of-Experts architectures, 4-bit quantization and speculative decoding win back enough of the speed to make large models practical. For running big models locally, capacity plus the right levers beats raw bandwidth with nowhere to put the weights.


    Where I landed


    The fleet I actually run day to day is the fast half of this grid, hot-swapped by llama-swap: Qwen3.6-35B for raw speed, the 120B-class Nemotron and Qwen3.5-122B when I need more depth, MiniMax-M2.7 as the dependable GT, and DeepSeek-V4-Flash or the 235B-class cars when a job demands the biggest engine on hand.


    Honestly, I did not expect models above 200B to be usable on this hardware at all. I had DeepSeek-V4-Flash running early on and it was quite slow, until speculative decoding made it genuinely usable. And MiniMax-M2.7 remains my biggest surprise of the whole exercise: rather large, rather fast, and seriously capable, all at the same time. That combination is what makes running local open-weight models both fun and quite effective.


    Two Sparks, a 200 GbE pit lane, 4-bit aero, and a good pit crew. For a homelab, that is a surprisingly competitive grid.


    Paul Oesterwitz

    Paul Oesterwitz

    AI & SAP Consultant · PhD Researcher

    More Insights

    AI ENGINEERING

    Soofi S: A Sovereign AI Model from Germany, and Why the Details Matter

    2026-07-14 • 10 min read

    AI ENGINEERING

    Standing Orders for Your Coding Agent: AGENTS.md Done Right

    2026-07-10 • 8 min read