DGX Spark vs H100: memory capacity against memory bandwidth

Published specifications

The H100 figures are for the GPU alone. A DGX Spark is a complete computer, so its power figure covers the CPU, memory and storage as well.

What the numbers mean

Using the published figures, the H100 SXM's memory bandwidth is about 12 times that of the DGX Spark (3,350 divided by 273). That is our own arithmetic, not an NVIDIA claim. For work that reads the whole model for every output token, such as generating text from a large dense model, speed scales closely with bandwidth, so an H100 will be much faster.

The DGX Spark has 128GB where the H100 has 80GB. A model that needs more than 80GB for weights and cache has to be split across several H100s, or heavily quantised, but fits on one DGX Spark.

spark, h100_sxm = 273, 3350          # GB/s, NVIDIA published
print(round(h100_sxm / spark, 1))     # 12.3, our arithmetic

# A dense 70B model at FP8 is ~70 GB of weights read once per generated token:
print(round(spark / 70, 1))           # 3.9 tokens/s upper bound on a DGX Spark

When the DGX Spark is the better fit

• The model is larger than 80GB and you want to avoid multi-GPU complexity.
• You are prototyping, experimenting or fine-tuning with parameter-efficient methods, and total time matters less than getting started.
• You want a clean, dedicated machine with root access billed by the minute.

When an H100 is the better fit

• Throughput is the goal: serving many users, or training from scratch.
• Your model already fits in 80GB and you need it as fast as possible.
• You need the datacenter software and interconnect options built around H100 systems.

NVIDIA positions the DGX Spark for prototyping, fine-tuning and local inference, with work then moved to DGX Cloud or a datacenter. We do not claim it replaces datacenter GPUs.

• NVIDIA DGX Spark product page
• NVIDIA DGX Spark hardware overview
• NVIDIA H100 Tensor Core GPU
• NVIDIA H100 PCIe product brief (PDF)