How to choose a cloud GPU for an LLM.

1. Start with the workload mode

Inference stores weights and runtime state; training adds gradients, optimizer states and activations; parameter-efficient tuning sits between them. Write down model revision, precision, context or sequence length, batch and concurrency before comparing hardware.

2. Build a memory budget

A rough weight estimate is parameters multiplied by bytes per parameter, but kernels, quantization metadata, KV cache, activations and fragmentation add overhead. Treat any calculator as a shortlist, then validate peak allocation.

• Add 10–20% operational headroom as a starting assumption
• Do not mix decimal GB and binary GiB silently
• Retest after changing context, batch or runtime

3. Compare outcomes, not specifications

For inference compare latency and dollars per million accepted tokens. For training compare time and dollars per accepted checkpoint. For visual work compare cost per accepted image, clip or frame.