RunPod — AI Glossary

Reference details

Related terms

• Modal
• Replicate
• Serverless Inference

How it works

1. Accelerators are rented as pods with near-bare-metal access, so whatever runs on them is the renter's choice.
2. Billing is per second of pod lifetime, which means an idle pod costs the same as a busy one — utilisation is the renter's problem.
3. Persistent volumes survive pod termination, so weights and datasets need not be re-downloaded each session.
4. Serverless endpoints wrap a custom container with autoscaling, which is the managed end of the same platform.
5. A community tier offers consumer-grade accelerators at lower prices with correspondingly weaker availability and reliability guarantees.

Example: Per-second billing only helps if something stops the clock

A pod at a notional £1.50 an hour left running for a month costs about £1,080 regardless of use. Used for 3 hours a day it is 90 hours of value against 720 billed — 12.5% utilisation, so the effective rate is eight times the headline. Per-second granularity is only an advantage if something tears the pod down when idle; without automation it bills like a reservation. This is the most common way raw rental ends up costing more than a managed service.

Commonly confused with

Raw GPU rental rather than a managed inference service: you get the machine and run whatever you like on it. That is the opposite end of the spectrum from a hosted API, and the practical difference is who handles serving, scaling and failure — here, you do. Spot and on-demand pricing is the other axis, and spot capacity can be reclaimed mid-run, which matters for training more than for inference.

When to use it

Reach for it when:

• You need full control of the machine — custom kernels, unusual frameworks, direct profiling.
• Sustained utilisation makes raw rental cheaper than per-token inference pricing.
• Experiments require specific accelerator types on demand.

Reach for something else when:

• You want a model endpoint rather than a machine.
• Nothing automates teardown, in which case per-second billing behaves like a reservation.
• Production reliability guarantees are required, especially on the community tier.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to RunPod — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path