Memory-First Sizing: GPU count determined by total memory requirements (model weights + KV cache), not compute requirements.

KV Cache Calculation: Uses actual model architecture details (layers, hidden size, attention heads) with support for GQA and MoE models.

Memory-Bandwidth Bound: Throughput estimates assume memory bandwidth is the bottleneck. Uses average memory traffic (sequence_length/2) during autoregressive decode.

Real-World Efficiency: Throughput includes configurable efficiency modifier (default 70%) to account for batching and scheduling overhead.

TTFT (Prefill) - Per Request: Compute-bound using formula: (2 × parameters × input_tokens) / (TFLOPs × GPUs_per_instance × efficiency). Uses per-instance GPUs.

TPOT (Decode) - Per Request: Memory-bound based on cluster token throughput divided by concurrent requests: 1 / (cluster_tokens_per_sec / concurrent_requests).

End-to-End Time - Per Request: Total request completion time = TTFT + (TPOT × output_tokens)

Cluster Throughput: Requests/sec = (concurrent_requests × 1000ms) / time_per_request. Effective output tokens/sec = requests/sec × output_tokens/request.

Concurrent Requests: Input value represents total cluster-wide concurrent load, not per-instance load.

Model Parameters: Extracted directly from HuggingFace model configs when available, with manual input fallback.

Precision Support: Supports 4-bit through 32-bit precision for both model weights and KV cache independently.

Model Instance Deployment: Small models use 1 instance per GPU. Models using >80% of GPU memory span 2 GPUs. Large models can span up to 4 servers per instance.

Server Configuration: Default 8 GPUs per server. Large models span multiple servers (2-4) per instance. Smaller models achieve better utilization with multiple instances per server.