The Physics of AI Memory: Sizing Model Weights and Parameter Precision
Deploying and training Large Language Models (LLMs) requires massive computational resources, with the primary bottleneck being Graphics Double Data Rate (GDDR) memory, commonly known as VRAM. Unlike CPU RAM, GPU VRAM must store the entire model state and activation parameters during execution to enable fast parallel matrix multiplication. Sizing this footprint is critical to prevent Out-Of-Memory (OOM) crashes.
The baseline memory footprint is occupied by the model weights. An LLM's size is measured in parameters (e.g., 7 billion, 70 billion). The memory required to store these weights is directly proportional to the numerical precision used. The weight VRAM is modeled as: $$V_{\text{weights}} (\text{GB}) = P × R_{\text{precision}}$$ where P is the number of parameters in billions, and R_{\text{precision}} is the byte footprint of the precision format: FP32 requires 4 bytes, FP16/BF16 requires 2 bytes, INT8 quantization requires 1 byte, and INT4 quantization requires 0.5 bytes.
For a broader analysis of machine learning workflows, you can estimate vector database memory with the vector DB storage calculator or plan host clustering with the Kubernetes capacity planner. Sizing weights is the first step in AI cluster planning.
Let's calculate the weight footprint of a 70 billion parameter model (like LLaMA-3-70B). In full 32-bit floating point precision (FP32), the weights require $70 × 4 = 280\text{ GB}$ of VRAM. Quantizing the model to 4-bit precision (INT4) using formats like AWQ or GPTQ slashes the weight footprint to: $$V_{\text{weights}} = 70 × 0.5 = 35.0\text{ GB}$$ allowing this massive model to run on a single workstation equipped with consumer-grade GPUs.
Furthermore, model size configurations are key variables during deployment. While a 7B model can fit in 14 GB of FP16 memory, hosting standard commercial models at scale requires allocating massive distributed VRAM configurations. Sizing your weights footprint correctly ensures that your GPU servers have enough capacity to support serving pipelines without latency degradation.