The MI300X's 192GB HBM3 capacity allows running 70B+ parameter models without tensor parallelism. Combined with 5.3 TB/s bandwidth, it delivers industry-leading inference throughput for memory-bound LLM serving.
Fully supported by ROCm and the PyTorch ecosystem, the MI300X is the GPU of choice for organizations training models on open-source frameworks. Leading labs have adopted MI300X for training runs of Llama, Mistral, and other open models.
Successor to the Frontier supercomputer lineage, MI300X excels at traditional HPC workloads including climate simulation, molecular dynamics, and computational physics with native FP64 performance and massive memory bandwidth.
With 192GB of memory, a single MI300X can host multiple AI models simultaneously — serving a routing model, embedding model, and multiple LLMs from one GPU. This consolidation dramatically reduces infrastructure costs for AI platforms.
| GPU Architecture | AMD CDNA 3 |
| Process Node | TSMC 5nm / 6nm (3D chiplet) |
| Compute Units | 304 |
| Stream Processors | 19,456 |
| Matrix Cores | 1,216 (AI Accelerators) |
| Memory Capacity | 192 GB HBM3 |
| Memory Interface | 8192-bit |
| Memory Bandwidth | 5.3 TB/s |
| Infinity Cache | 256 MB |
| Form Factor | OAM (OCP Accelerator Module) |
| Thermal Design Power | 750W |