NVIDIA Hopper
NVIDIA H100 GPU Cloud & Dedicated Infrastructure
The general-purpose workhorse for production LLM inference and training.
The NVIDIA H100 is the most widely deployed data-center GPU for AI workloads. It offers a strong balance of memory bandwidth, compute throughput, and provider availability, making it a reasonable default for teams that have not yet outgrown its 80GB memory envelope.
H100 specifications
- GPU Memory
- 80GB HBM3
- Memory Bandwidth
- Up to ~3.35 TB/s
- Architecture
- NVIDIA Hopper
- Interconnect
- NVLink / NVSwitch, PCIe Gen5
- Form Factors
- SXM and PCIe variants
- Typical Server Configs
- 1x, 2x, 4x, 8x GPU nodes
Deployment options
- On-demand cloud GPU instances
- Reserved capacity for predictable monthly workloads
- Dedicated single-tenant H100 servers
- Multi-node H100 clusters for distributed training
Best suited for
- Production inference for 7B–70B parameter LLMs
- Fine-tuning open-weight models with LoRA/QLoRA
- Multi-GPU training runs with NVLink-connected nodes
- Mixed inference and training environments
Less ideal for
- Serving the largest frontier-scale models without model parallelism
- Workloads that specifically require 141GB+ memory per GPU
Related solutions
Managed software options
How it compares
- vs. H200
H200 offers ~1.76x the memory for teams serving larger models or longer context windows.
- vs. A100
A100 is typically more cost-efficient for workloads that do not require Hopper-generation throughput.
- vs. B200
B200 provides a further generational step up in training throughput for teams scaling beyond Hopper.
H100 frequently asked questions
Is the H100 still a good choice for new deployments?
Yes. The H100 remains the most broadly available data-center GPU across our provider network, and 80GB of HBM3 is sufficient for the majority of production inference and fine-tuning workloads. It is a sensible default unless your model or context-length requirements specifically call for more memory.
What is the difference between SXM and PCIe H100 variants?
SXM variants offer higher power limits and NVLink bandwidth, which benefits multi-GPU training and tensor-parallel inference. PCIe variants are more broadly compatible with standard server chassis and are often more available for single- or dual-GPU deployments. We can help you choose based on your workload.
Can I get a dedicated H100 server instead of shared cloud capacity?
Yes. GPUMarket can source dedicated, single-tenant H100 servers in addition to on-demand and reserved cloud instances, depending on your compliance, performance isolation, and budget requirements.
Do you provide managed infrastructure on top of H100 capacity?
Yes. We can deploy and operate the full stack on top of the GPU — Kubernetes, GPU drivers, inference runtimes, monitoring, and support — as part of our managed GPU infrastructure service.