How to Choose a GPU for AI? A Practical Guide Based on Requirements
In AI-driven projects, GPU selection often starts with choosing a specific card model. In practice, this is the wrong starting point. AI infrastructure should be defined by project requirements: use case, model size, number of users, expected TPS, context length, and quantization method. Only then should you determine the appropriate class of solution.
Starting Point: Not the GPU Model, but the Requirements
The same GPU may be optimal in one scenario and completely insufficient in another. That’s why GPU selection should begin with workload analysis rather than a product list.
1) Define the AI use case
The first step is answering: what will AI be used for? This determines GPU requirements.
Chatbots and RAG
Q&A systems, knowledge retrieval, user support, integration with internal documentation.
Document and data analysis
Processing contracts, reports, and internal datasets with larger context requirements.
AI agents and automation
Multi-step workflows, tool usage, system integrations, complex processes.
Expert models and AI-as-a-service
70B+ models, multi-session environments, SLA-driven infrastructure.
2) Model size: 7B / 13B / 70B
| Model class | Characteristics | Typical use |
|---|---|---|
| 7–8B | Lightweight, fast, low requirements | Chatbots, RAG, Q&A |
| 13B | Balanced quality/performance | Enterprise AI, document analysis |
| 70B | High requirements, advanced reasoning | Expert systems, enterprise |
3) TPS and number of users
TPS (tokens per second) defines generation speed, but must be evaluated alongside concurrent users.
4) Context length
- 4k–8k – standard
- 16k–32k – documents
- 32k+ – advanced workloads
5) Choosing the GPU class
Entry-level
Testing, small deployments
Production
Stable multi-user environments (e.g. RTX 6000 Ada)
Enterprise
Large-scale AI (e.g. RTX PRO 6000 Blackwell)
Summary
- Use case
- Model size
- Users
- TPS
- Context
- Quantization
