GPU VRAM & Deployment Estimator
Estimate the VRAM required to serve an LLM and get GPU recommendations with ready-to-copy vLLM / Ollama commands.
About
Estimate the GPU VRAM (GB) required to serve a large language model. Inputs: model parameter count in billions, weight quantization precision (fp16/int8/int4), context length in tokens, batch size (concurrent sequences), and KV-cache precision. Returns a memory breakdown (weights, KV cache, activations, runtime overhead), a recommended GPU configuration, and copy-paste vLLM and Ollama launch commands. Use this when a user asks how much GPU memory a model needs or which GPU to buy/rent.
Parameters
| Name | Type | Description |
|---|---|---|
modelParamsB * | number | Model size in billions of parameters (e.g. 7, 13, 70). (default: 7) |
precision | fp16 | int8 | int4 | Weight quantization precision. (default: "fp16") |
contextLength | integer | Context window length in tokens. (default: 8192) |
batchSize | integer | Number of concurrent sequences (batch size). (default: 1) |
kvPrecision | fp16 | int8 | KV-cache precision. (default: "fp16") |
tensorParallel | integer | Number of GPUs to shard the model across (tensor parallel size). (default: 1) |
locale | en | zh | UI language for the rendered card. Match the language of the end user. (default: "en") |
Examples
- How much VRAM does Llama 3 70B need at INT4 with a 32k context?
- Which GPU should I rent to serve a 13B model for 16 concurrent users?
Use it
In the browser: https://mcp.gholl.com/embed/vram-calc
As an AI tool: call vram-calc on the MCP endpoint https://mcp.gholl.com/mcp.
{
"mcpServers": {
"gholl": {
"type": "http",
"url": "https://mcp.gholl.com/mcp"
}
}
}