One work session
$5.52
1 GPU × 8h · compute only
Production and batch inference
Rent GPUs for LLM serving by context, concurrency and token cost.
Direct answer
Size LLM inference from weight precision, runtime overhead, KV cache, context length and simultaneous sequences. Compare time to first token, inter-token latency, p95 latency and cost per million accepted tokens before selecting RTX 5090, L40S, H100 or H200.
Reviewed by the GPURento infrastructure team · September 1, 2026
Transparent planning windows
Example rental cost on one RTX 5090.
These are compute rental windows, not predictions of job duration or performance. Replace the hours with a measured pilot and add storage in the calculator.
Forty-hour window
$27.60
1 GPU × 40h · compute only
One-hundred-twenty hours
$82.80
1 GPU × 120h · compute only
| Memory drivers | Weights + KV cache | Context and concurrency can dominate after weights. |
|---|---|---|
| Latency metrics | TTFT · TPOT · p95 | Average latency hides queue and tail behavior. |
| Economic metric | $ / 1M tokens | Hold model, quality and traffic shape constant. |
| Execution choice | Dedicated or serverless | Depends on utilization and cold-start tolerance. |
Best for
- Private model endpoints
- Batch generation and evaluation
- Long-context or high-concurrency serving
- Teams measuring latency and cost together
Not the best fit when
- Traffic with no known quality or latency target
- Models that have not been profiled at the intended precision
- Sensitive data flows without a reviewed region and retention policy
Decision method
What to verify before you deploy capacity.
The content below separates published specifications, GPURento catalog references and decisions that still require a workload benchmark.
01
Fit the model and the traffic
Weight memory is only the starting point. Add quantization metadata, runtime workspace and KV cache for the desired context length, batch size and simultaneous sequences. Leave headroom so a brief peak does not cause an out-of-memory restart.
02
Choose dedicated or serverless from utilization
A dedicated GPU suits steady utilization and predictable warm latency. Serverless fits bursty traffic when scale-to-zero savings exceed cold-start and per-request overhead. Evaluate both with the same traffic trace.
- Record time to first token
- Record inter-token latency and p95
- Divide total metered cost by accepted output tokens
03
Use the smallest GPU that meets the service level
A larger GPU is justified when it reduces queueing, raises concurrency or avoids multi-GPU sharding enough to lower total cost. Otherwise, a 24–48 GB card can be a better economic fit.
Catalog shortlist
Relevant GPU options.
BlackwellAvailable
NVIDIA RTX 5090
32 GB GDDR7 · $0.69/hr
Fast single-GPU inference and media
EU West · ParisEU Central · Frankfurt
AdaAvailable
NVIDIA L40S
48 GB GDDR6 ECC · $0.79/hr
Production inference, fine-tuning and video
EU West · ParisEU Central · Frankfurt
HopperAvailable
NVIDIA H100 SXM
80 GB HBM3 · $2.69/hr
Intensive training and FP8 inference
EU West · ParisEU Central · Frankfurt
HopperAvailable
NVIDIA H200 SXM
141 GB HBM3e · $3.59/hr
Large-model inference and HPC
EU West · ParisEU Central · Frankfurt
Questions
Clear answers, including the limits.
How much VRAM do I need for LLM inference?+
It depends on parameter count, precision, runtime, context and concurrency. Calculate model weights first, then add KV cache and at least a practical runtime margin.
Is serverless GPU cheaper than renting an instance?+
It can be for intermittent traffic because idle workers can scale down. A continuously busy endpoint may be cheaper and more predictable on a dedicated instance.
Which metric should I compare?+
Use cost per million accepted tokens together with time to first token and p95 latency. Throughput alone can hide poor user experience.
Sources and scope
- NVIDIA H100 product pageArchitecture and memory specifications from the manufacturer.
- NVIDIA H200 product pageH200 memory capacity and bandwidth from the manufacturer.
Last reviewed September 1, 2026. GPURento rates are current catalog references; provisioning state remains visible in the workspace, and manufacturer specifications do not substitute for workload benchmarks.
Continue the research
Deployment plan
Size the endpoint before requesting capacity.
Bring the model, precision, context, concurrency and latency target to the workspace configuration.