The following table lists the recommended instance types required for AI Gateway configurations.
| AWS Instance Type | GCP | Azure | ESXi VM | Concurrent Requests | Minimum Disk Size | |
|---|---|---|---|---|---|---|
| small | c6a.4xlarge (16 cores/32GB) | c2d-highcpu-16 | D16ls_v7 (16 cores/32GB) | 16 cores/32GB | CPU ML: 8 RPS GPU LLM: 40 RPS DLPoD: 380 RPS | 300GB |
| medium | c6a.8xlarge (32 cores/64GB) | c2d-highcpu-32 | D32ls_v7 (32 cores/64GB) | 32 cores/64GB | CPU ML: 15 RPS GPU LLM: 40 RPS DLPoD: 380 RPS TSS: 1500 RPS | 300GB |
Newer instance generations usually offer better performance. Customers can choose newer families, such as c6i, c7i, c7a, c8i, or c8a, for improved throughput.
Requirement: The AI Gateway VM must be deployed on an AVX2-capable host.
Warning: The AI Gateway deployment will fail if the host does not meet the minimum specifications listed above.
Warning: The AI Gateway deployment will fail if the host does not meet the minimum specifications listed above.
DLP Appliance Sizing
The following table lists the recommended instance types for cloud deployments and the corresponding size configurations for on-premises environments of the DLP On Demand appliance. For more information, see /en/dlpondemandconfig.
| AWS | GCP | Azure | VM | Concurrent Requests | Minimum Disk Size | |
|---|---|---|---|---|---|---|
| Small | c5ad.4xlarge | n2-standard-16 | Standard-F16s_v2 | 16 cores/32GB | 480 rps with burst of 576 rps. Max throughput limit: 786 MB per min | 351GB |
| Medium | c5ad.8xlarge | n2-standard-32 | Standard-F32s_v2 | 32 cores/64GB | 1280 rps with burst of 1536 rps. Max throughput limit: 2048 MB per min | 351GB |
| Large | c5ad.16xlarge | n2-standard-64 | Standard-F64fs_v2 | 64 cores/128GB | 2880 rps with burst of 3456 rps. Max throughput limit: 4608 MB per min | 351GB |

