mcp.gholl.com — Interactive micro-tools

GPU VRAM & Deployment Estimator

Estimate the VRAM required to serve an LLM and get GPU recommendations with ready-to-copy vLLM / Ollama commands.

  • Infrastructure
  • vram
  • gpu
  • llm
  • vllm
  • ollama

About

Estimate the GPU VRAM (GB) required to serve a large language model. Inputs: model parameter count in billions, weight quantization precision (fp16/int8/int4), context length in tokens, batch size (concurrent sequences), and KV-cache precision. Returns a memory breakdown (weights, KV cache, activations, runtime overhead), a recommended GPU configuration, and copy-paste vLLM and Ollama launch commands. Use this when a user asks how much GPU memory a model needs or which GPU to buy/rent.

Parameters

NameTypeDescription
modelParamsB *numberModel size in billions of parameters (e.g. 7, 13, 70). (default: 7)
precisionfp16 | int8 | int4Weight quantization precision. (default: "fp16")
contextLengthintegerContext window length in tokens. (default: 8192)
batchSizeintegerNumber of concurrent sequences (batch size). (default: 1)
kvPrecisionfp16 | int8KV-cache precision. (default: "fp16")
tensorParallelintegerNumber of GPUs to shard the model across (tensor parallel size). (default: 1)
localeen | zhUI language for the rendered card. Match the language of the end user. (default: "en")

Examples

  • How much VRAM does Llama 3 70B need at INT4 with a 32k context?
  • Which GPU should I rent to serve a 13B model for 16 concurrent users?

Use it

In the browser: https://mcp.gholl.com/embed/vram-calc

As an AI tool: call vram-calc on the MCP endpoint https://mcp.gholl.com/mcp.

{
  "mcpServers": {
    "gholl": {
      "type": "http",
      "url": "https://mcp.gholl.com/mcp"
    }
  }
}