GenAI Sizing Calculator
Calculate the optimal GPU configuration for your GenAI workloads. Input your requirements and get recommendations for hardware sizing, expected throughput, and cost estimates.
Use Case Profiles
Add and compare multiple workload profiles. Sizing results on the right are aggregated across all configured profiles.
Use Case Configuration
Use Case 1Model Configuration
Used for memory footprint calculation
For future optimization features
Memory Requirements
Workload Requirements
Peak or p95 simultaneously active requests across the deployment. Queued or merely connected users should not be included.
This is an active-sequence admission ceiling, not a scheduler token batch size. Auto keeps hard HBM capacity separate from the recommended per-replica operating load; a manual value is retained even when current cluster demand is lower.
GPU Selection
NVLink domain, scale-out interconnect, and GPUs per node for sizing
NVIDIA B300 SXM (Blackwell Ultra) is sized in HGX-8 (8-GPU NVLink island): 8-GPU NVLink domain, 8 GPUs per server/node, and 50 GB/s scale-out per GPU.
Set by topology preset
Derived from total GPUs ÷ GPUs per server / node
Performance Specifications
Per-Use-Case Performance
Aggregate Sizing Results
Select and configure at least one use case profile to see aggregate GPU requirements.
Key Assumptions
Constraint-Based Sizing: GPU count starts with model-weight and KV-cache fit, then increases for explicit throughput and latency targets.
KV Cache Calculation: Uses architecture details from the loaded config, including GQA/MHA and MLA-style cache fields when available.
Roofline Performance: TTFT, TPOT, and throughput use compute, memory bandwidth, NVLink, scale-out bandwidth, pipeline efficiency, and calibration defaults.
Utilization Headroom: Request-rate targets are sized against the selected max utilization so interactive workloads keep latency headroom.
Latency SLA Targets: Response-time, TTFT, and TPOT targets only affect sizing when their requirement toggles are enabled.
Uncertainty Bands: Current latency and throughput estimates are theoretical, not benchmark-validated, so ranges are intentionally wide.
End-to-End Time: Request time is TTFT plus decode time across average output tokens, with queue-delay risk surfaced when target utilization is high.
Cluster Throughput: Effective output tokens/sec scales by data-parallel replicas; requests/sec divides token throughput by average output tokens.
Peak Active Requests: Input value is peak or p95 simultaneously active cluster-wide demand. Replica admission policy, HBM capacity, throughput-efficient operating load, and final modeled load are reported separately.
Model Parameters: Extracted directly from HuggingFace model configs when available, with manual input fallback.
Precision Support: Supports MXFP4, NVFP4, generic 4-bit, 8-bit, 16-bit, and 32-bit model weights. KV cache precision is configured separately.
Model Instance Deployment: Small models use 1 instance per GPU. Models using >80% of GPU memory span 2 GPUs. Large models can span up to 4 servers per instance.
Server Configuration: Default 8 GPUs per server. Large models span multiple servers (2-4) per instance. Smaller models achieve better utilization with multiple instances per server.
Roadmap
Launch v1.0!
Multiple model instance support
Add Time to First Token (TTFT)
Add Time Per Output Token (TPOT)
Differentiate between prefill and decode timing
Add additional GPU options
Add use case templates
Mixed LLM workloads