Quick comparison why Nvidia is preferred over its (only) competitor AMD for LLM work.
| Aspect | NVIDIA GPUs | AMD Radeon Pro |
|---|---|---|
| Framework Support | Extensive support via CUDA for PyTorch, TensorFlow, and most ML frameworks | Limited support, fewer frameworks optimized for AMD |
| ML Ecosystem | Mature CUDA ecosystem with wide adoption | Less developed ROCm ecosystem with limited compatibility |
| Software Integration | Well-established pipelines and tools | More restricted options, may require additional setup |
| Raw Computing Power | Strong performance with direct ML optimization | Good raw power but harder to leverage for ML tasks |
| Memory Options | Various models with sufficient VRAM (16GB+) | Competitive VRAM options but harder to utilize for ML |
| Primary Use Case | Strong in both ML/AI and professional graphics | Better suited for professional graphics work |
| LLM Specific Support | De facto standard for LLM deployment | Limited practical application for LLMs |
Towards end of CY24:
Upcoming: RTX 4080 Super?
- Expected Advantages:
- Better performance than 4080
- Same MSRP as current 4080
- Improved efficiency
- Note: Wait for release if considering 4080 for a middle ground GPU.