AI Red Teaming

The AI Red Teaming service provides vulnerability assessments to secure your LLMs pre-deployment and identify production drifts to meet compliance and ensure quality continuity.

You can deploy the AI Red Teaming service to:

• Simulate Adversarial Attacks: Test your LLMs against a wide range of attack scenarios.

• Uncover Weaknesses: Pinpoint flaws in your LLM’s alignment, robustness, and content moderation capabilities.

• Assess Safety Risks: Proactively identifying potential vulnerabilities, prompt injections, jailbreaking, data leakage, privacy violations, unethical responses, harmful content, and safety gaps.

  The goal is to systematically uncover weaknesses in the LLM’s alignment, robustness, and content moderation capabilities.

  Netskope sends adversarial prompt testing requests to your LLM models and apps and assess the security risks based on the responses from the LLM under testing. In the Netskope UI, you can use register target LLMs, launch red teaming tests and access results. AI Red Teaming supports testing custom LLM models hosted locally or on public cloud and foundation models offered by:

  • OpenAI

  • Azure OpenAI

  • Amazon Bedrock

  Following are a few terms and concepts specific to AI Red Teaming:

  Term	Definition
  Target	It refers to the specific AI system or model that is being tested for vulnerabilities, biases, and potential safety issues. It’s the AI you are trying to “break” or find flaws in.
  Test	A test is a collection of probes or attacks executed at a single point in time to evaluate a target AI’s performance and vulnerabilities.
  Prompt	A question or instruction given to the LLM to elicit a response.

AI Guardrails is supported in FedRAMP and PBMM tenants.

Overview

In the Overview tab (AI Red Teaming > Overview), you can:

1. Filter the overview results from 30 to 120 days. By default, it shows data for 30 days.

2. Filter the overview results by specific AI targets you’ve registered. By default, it shows data for all targets.

3. View the overall attack success rate (ASR) percentage, which measures the percentage of attack attempts that successfully attack your AI targets. You can also see:

   • Test Result: A summary of results from the total cases that were used to measure the ASR.

     • Attacks Blocked: The total attacks blocked from attacking the target, indicating that Netskope found no vulnerability or weaknesses.

     • Attacks Successful: The total attacks that were was successful attacking the target, exposing a vulnerability or weakness.

     • Errored: The total red team tests that errored because Netskope wasn’t able to send an attack prompt to the AI target.

   • Total Cases: The total number of test cases, which is all the probes or attack prompts that Netskope sends to the AI target for evaluation.

4. View derived distributions of attack types used across tests, including:

   • Sensitive Data Leakage: The percentage of test cases associated with unauthorized disclosure of sensitive data in LLM responses.

   • Prompt Injection & Jailbreaking: The percentage of test cases using manipulation techniques designed to bypass AI safety guardrails, including DAN (Do Anything Now), Evil Confident, Switch Mode, Refusal Suppression, Wiki Attack, and Deceptive Delight methods.

   • Inappropriate Content (Categories): The percentage of test cases across all inappropriate content categories. To learn more: Prompt Library.

5. View the ASR percentage and distribution for the test cases that successfully attacked a target:

   • Prompt Sets: The classification used to group attack prompts based on their topic, intent, or type. To learn more: Prompt Library.

   • Attack Success Rate: The ASR percentage based on if the test cases successfully attacked an AI target.

6. Filter the Attack Success Rate Breakdown table by:

   • Prompt Sets: The Netskope prompt categories. To learn more: Prompt Library.

   • Techniques: The manipulation technique.

   • OWASP: The OWASP LLM categories.

   • Sensitive Data Leakage: Your configured DLP profiles.

7. View 5 bar graphs showing ASR trends from the most recent tests you’ve performed (up to 15 total). Each bar graph represents one test round with up to three targets, where each bar represents a different target tested in that round. This allows you to compare ASR performance across targets within the same test round and track trends over time.

   Click See All to go to the Test Rounds tab and see all your tests.

8. Go to the Target Registry tab where you can add a new AI target.

9. Go to the Test Rounds tab where you can conduct red team testing.

10. Go to the Model Drifting tab where you can manage the performance of your AI targets and mitigate drift.

11. Go to the Prompt Library tab to learn more about the Netskope prompt categories.