Rate Limit

To configure rate limits for all or specific gateway, follow the steps below.

1.  Log in to the Netskope tenant UI and go to Settings > Security Cloud Platform > AI Gateway > Rate Limit.

2. Click New Override.

3. In the New Rate Limit Override page, enter the values for the following parameters.

   • Name: Enter a name for the rate limit override. This is a required field.
   • Gateway: Select an AI Gateway to which you want rate limit override to be applied.
   • Subset of Gateways/All Gateways: Specify if you want to apply the rate limits to a subset of gateways or to all of the gateways.
   • Match Criteria: Specify the match criteria for the rate limit to be applied. As match criteria, you can select an AI provider, AI model name and Token groups. You can select multiple token groups. If the value is left empty, it will be applied to all.

4. Under Rate Limit, specify a rate limit in the form of number/per requests/day or Hour/gateway.

   

5. Add a custom response message in the Response Message text box. When the defined rate limit reaches the maximum limit, AI Gateway sends this custom message with the response code 429.