H100 vs B200 GPU Cost Comparison

Side-by-Side Provider Price Comparison

The table below shows providers that carry both H100 SXM5 and B200 on-demand, ranked by B200 price. Use this to compare your actual cost delta at the provider you already use.

Data pulled from GridStackHub's real-time pricing database. Prices shown are cheapest on-demand per GPU at each provider. See all 396+ pricing records →

H100 SXM5 Pricing — All Providers

As of May 2026, 27 providers offer H100 SXM5 on-demand in GridStackHub's database. Specialist GPU clouds (Lambda, CoreWeave, RunPod) are consistently 30–50% cheaper than hyperscalers (AWS, GCP, Azure) for equivalent H100 hardware.

H100 spot pricing from providers like Vast.ai and Shadeform can drop to $0.79–$1.40/hr — 40–60% below on-demand. Spot is preemptible but viable for fault-tolerant training jobs. Calculate total cost for your workload →

B200 GPU Pricing — All Providers

B200 availability is more limited than H100. As of May 2026, 6 providers carry B200 on-demand in GridStackHub's database, compared to 27 for H100. Supply constraints keep B200 pricing higher and less volatile than H100.

Hyperscalers (AWS p6.48xlarge, GCP a3ultra-megagpu) offer B200 in 8-GPU cluster configurations only — per-GPU pricing is higher than specialist clouds but includes managed networking, storage, and SLAs. Full B200 provider guide →

Monthly Cost Breakdown

Monthly estimates at cheapest available on-demand rates (Shadeform H100 at $1.79/hr, Lambda B200 at $5.29/hr). 720 hours = 30 days continuous use.

Note: hyperscaler pricing (AWS, GCP, Azure) is typically 40–80% higher than specialist GPU clouds for equivalent hardware. The table above uses cheapest available specialist cloud pricing. Reserved/committed pricing discounts vary by provider and term length (1-year vs 3-year commitments).

Training Cost Math: H100 vs B200

For LLM pre-training and fine-tuning, what matters is cost per token processed — not cost per hour. The B200's higher throughput changes the calculus significantly.

Key Training Specs

Cost Per Million Tokens Trained (7B Model Estimate)

Using cheapest available on-demand pricing. Throughput assumes mixed FP8/BF16 training on a 7B-parameter model. Actual throughput varies by batch size, context length, and framework.

The training cost advantage compounds at scale. A 30-billion-token training run costs approximately $16781.25 on H100 vs $22041.67 on B200 — a saving of $-5260.42 at current cheapest-available pricing. At 8 GPUs this advantage multiplies accordingly.

Important caveat: these estimates assume the workload can saturate the GPU's compute and memory bandwidth. Smaller batches, short context lengths, or CPU-bottlenecked data pipelines will reduce the throughput advantage. Always benchmark your specific workload before committing to hardware.

Inference Cost Math: Why B200 Wins on Tokens/Sec

For LLM inference, token generation is memory-bandwidth-bound — the GPU must load model weights from HBM for every forward pass. B200's 8.0 TB/s bandwidth (vs H100's 3.35 TB/s) translates directly into higher tokens/sec throughput, which is what drives cost per token.

Cost Per Million Tokens Generated (70B Model Inference)

At production inference volumes (billions of tokens/day), this cost advantage is substantial. An API serving 1 billion tokens/day costs approximately $41.44/day on H100 vs $52.48/day on B200 — a saving of $-11.04/day, or over $-4031.42/year.

When H100 Beats B200 on Inference

H100 remains cheaper in the following inference scenarios:

• Low-traffic APIs (<1M tokens/day): At low request rates, the GPU is idle most of the time. A cheaper H100 sitting idle is cheaper than a more expensive B200 sitting idle. The bandwidth advantage only materializes when the GPU is fully loaded.
• Small models (7B–13B): Smaller models load faster even on H100's bandwidth. The B200 throughput advantage narrows to ~1.8× for 7B models, which may not justify the 3.0× hourly cost.
• Spot pricing on H100: H100 spot from Vast.ai, Shadeform, and RunPod can drop to $0.79–$1.40/hr — 40–60% below H100 on-demand. At $1.00/hr spot, H100 cost per million inference tokens drops to $0.023, significantly undercutting B200 on-demand.
• Short context requests: For short inputs/outputs (under 512 tokens), batching efficiency matters more than raw bandwidth. H100 with optimized batching can match B200 cost-per-token in many production request distributions.

When to Choose H100 vs B200

The break-even point: if B200's throughput advantage reduces your required GPU-count or wall-clock time by more than 3.0×, B200 wins on total cost. For most memory-bandwidth-bound 70B+ workloads, this threshold is met. For anything under 13B with low concurrency, H100 wins on simplicity and price.