STIX/TAXII Plugin for Threat Exchange

This document explains how to configure the STIX/TAXII v3.2.0 plugin with the Threat Exchange module of the Netskope Cloud Exchange platform. This plugin fetches IoCs of type Domain, URL and Hash (MD5 and SHA256) for version 1.1 and IOCs of type Domain, URL, IPv4, IPv6, and Hash (MD5 and SHA256) for version 2.1/2.2 from the TAXII feeds and extracts observables from them. This plugin supports retraction of IoCs pulled from TAXII feeds. This plugin does not support sharing of indicators to TAXII feeds.

Prerequisites

To complete this configuration, you need:

• A Netskope tenant (or multiple, for example, production and development/test instances).
• A Netskope Cloud Exchange tenant with the Tenant plugin and Threat Exchange plugin already configured.
• A Secure Web Gateway subscription for URL sharing.
• A STIX/TAXII server to pull indicators.
• STIX/TAXII API credentials details. Contact STIX/TAXII support for an appropriate Discovery URL/API Root URL.
• Connectivity to STIX/TAXII Partner login URL.

STIX/TAXII Plugin Support

This plugin is used to fetch IoCs of type Domain, URL, and Hash (MD5 and SHA256) for version 1.1 and IoCs of type Domain, URL, IPv4, IPv6, and Hash (MD5 and SHA256) for version 2.1/2.2 from the TAXII feeds and extracts observables from them. This plugin supports retraction of IoCs pulled from TAXII feeds. This plugin does not support sharing of indicators to TAXII feeds.

Mappings for Pulled IoCs

For version 1.1

For version 2.x

Permissions

Permissions may vary from platform to platform. For the XSOAR platform, you will need an Administrator role.

1) Client creation:
from cabby import create_client
client = create_client(
                base,
                port=port,
                use_https=True,
                discovery_path=discovery_url,
            )

2) Fetch collections:
client.get_collections(uri=collection_uri)

3) Poll objects:
content_blocks = client.poll(
      collection_name=collection,
      begin_date=start_time,
)

1) ApiRoot object creation:
apiroot = ApiRoot21(
       configuration["discovery_url"].strip(),
       user=username,
       password=password,
       verify=self.ssl_validation,
       proxies=self.proxy,
       )

2) Fetch collections:
collections = apiroot.collections

3) Fetch objects:
pages = as_pages21(
            func,
            per_request=batch_size,
            added_after=start_time,
            next=next,
            with_header=True,
            headers=headers
      )

Performance Matrix

Here are the performance readings conducted for fetching 100K IoCs in each plugin lifecycle on a Large Cloud Exchange instance with the below specifications.

A XSOAR TAXII server was used to pull data.

User Agent

netskope-ce-6.0.0-cte-stix/taxii-v3.2.0

Workflow

1. Get your Discovery URL/API Root URL, username, and password.
2. Configure the STIX/TAXII plugin.
3. Configure a Business Rule.
4. Configure Sharing.
5. Validate the plugin.

A XSOAR TAXII server was used to pull data in these instructions.

Watch a Video

Click play to watch a video:

Configure the TAXII2 Server

1. Log in to the XSOAR server.
2. Click Settings > Integrations > Instances.

   

3. Search for Taxi and click Add Instance.

   

   

   

4. Enter the following information:
   • Name: Enter the STIX/TAXII name.
   • TAXII2 Server version: 2.1 or 2.0.
   • Listen Port: The port on which you want to send feeds.
   • Username: The user name for the STIX server(This will be required when configuring plugin).
   • Password: The password for stix server(This will be required when configuring plugin).
   • Collection JSON: Collection name for Stix.
   • Response Size:  The size of the response you want to receive from the server. Note that Netskope Cloud Exchange only supports pulling up to a size of 1000 ICs.

   Note

   Example for Collection JSON:

   {
     "url_collection": {
       "query": "type: URL",
       "description": "This is a test collection"
     },
    "hashes_collection": {
       "query": "type: File",
       "description": "This is a test collection"
     },
     "domain_collection": {
       "query": "type: Domain",
       "description": "This is a test collection"
     },
     "IPv4_collection": {
       "query": "type:IP",
       "description": "Collection of IPv4 indicators for internal network analysis"
     },
     "IPv6_collection": {
       "query": "type:IPv6",
       "description": "Collection of IPv6 indicators for external threat tracking"
     }
   }

5. Click Save. For more information, see: https://xsoar.pan.dev/docs/reference/integrations/taxii2-server.

Configure the STIX/TAXII Plugin

1. In Cloud Exchange, go to Settings > Plugins.
2. Search for and select the STIX/TAXII v3.2.0 (CTE) plugin box.

   

3. Enter the Basic Information:
   • Configuration Name: Plugin configuration name.
   • Sync Interval: Interval to fetch data from this plugin source.
   • Aging Criteria: Expire indicators after a specific time.
   • Override Reputation: Set value to override reputation of indicators received from this configuration. Leave empty to keep default(Default value will be 5).
   • Tags Aggregate Strategy: Choose whether to append new tags to existing IoC(s) or overwrite them. These configuration parameters determine how tags are stored for indicators pulled for this configuration.
   • Enable SSL Validation: Enable/Disable SSL Certificate validation.
   • Use System Proxy: Use system proxy configured in Settings.

   

4. Click Next and enter the Configuration Parameters:
   • STIX/TAXII Version: Select STIX/TAXII Version as per your instance.
   • Discovery URL/API Root URL: Discovery/Feed URL of TAXII server for version 1.x and API Root URL for version 2.x. Contact your STIX/TAXII support for appropriate Discovery URL/API Root URL.
   • Username: Username required for authentication, if any.
   • Password: Password required for authentication, if any.
   • Collection Names: Comma separated collection names from which data needs to be fetched. Leave empty to fetch data from all of the collections.
     Format: “<collection1>,<collection2>,<collection3>”
   • Initial Range: Number of days to pull the data for the initial run.
   • Look Back (in minutes): Number of minutes to backdate the start time for pulling the data. Valid value is anything between 0 to 1440.
   • Type of Threat data to pull: Type of Threat data to pull. Note: IPv4/IPv6 is only supported for STIX/TAXII version 2.x.
   • Severity: Only indicators with matching severity will be fetched. STIX/TAXII version 2.x only supports Unknown severity, thus to pull data using STIX/TAXII version 2.x it is mandatory to select ‘Unknown’ in the severity field. Refer to Troubleshooting if you face any issue while pulling IOCs.
   • Reputation: Only indicators with reputation equal to or greater than this will be saved in Netskope Cloud Exchange.
   • Pagination Method: Pagination Method to use while pulling the indicators. Contact your STIX/TAXII support to choose the appropriate option.
   • Batch Size: Number of indicators to fetch per bundle. Must be an integer in range 1 to 1000.
   • Retraction Interval (in days): Number of days to look back for retraction checks. Leave empty to disable retraction. Must be an integer in range 0 to 365.

   

   

   Note

   • There will be only the Unknown severity available in the configuration parameters for 2.x version.
   • The range for Batch Size is 1 to 1000 for 2.1 version and 2 to 1000 for 2.0 version.
   • For 1.1 version, the Batch Size configuration parameter will not be there.

   

5. Click Save.

   

Add a Threat Exchange Business Rule for STIX/TAXII

The business rule is used to filter out the indicators that are to be shared. To share IOCs with any third-party plugin, you need to create a business rule:

1. In Threat Exchange, go to Business Rules and click Create New Rule.
2. Add the rule name and select the fields through which you want to filter the IoCs.

   

3. Click Save.

   

Add Threat Exchange Sharing for STIX/TAXII

To configure the sharing:

1. In Threat Exchange, go to Sharing and click Add Sharing Configuration.
2. Select a Source configuration (source from which you want to share data), a Business Rule, and a Destination configuration.
3. Select the target value and action type.
4. Click Save.

   

Note that the STIX/TAXII plugin does not support sharing of indicators to TAXII feeds.

Validate the STIX/TAXII Plugin

Validate the Pull

Indicators stored in Cloud Exchange can be verified in Threat Exchange at Threat IoCs. Search the STIX/TAXII IOCs by filtering indicators from STIX TAXII plugin.

Example: Add a query on the Threat IoCs page like “sources.source Is equal “CTE STIX TAXII” && type IN (“<IOC_TYPE>”)”

You can also verify the indicators pulled in Cloud Exchange from the logs available at Logging.

To verify whether IoCs are present on XSOAR platform, log in to your instance and go to Indicators.

Troubleshooting the STIX/TAXII Plugin

Unable to configure the STIX/TAXII plugin

If you are unable to configure the plugin, it might be due to one of these reasons:

• Invalid Discovery URL/API Root URL.
• Invalid Username or password.
• Getting the below error while configuring the plugin:
  CTE STIX/TAXII [CTE STIX TAXII]: Could not fetch the collection list from the server. Error: ('Connection broken: IncompleteRead(586 bytes read)', IncompleteRead(586 bytes read))
• Getting error while configuring the plugin with on-premise instance for STIX/TAXII version 1.1.

What to do:

1. Verify the Discovery URL/API Root URL that you are using is correct. If you are not sure about the Discovery URL/API Root URL, then contact your STIX/TAXII support team for respective platforms.
2. Verify the username and password. They must be the same as the one used while configuring the TAXII server.
3. Verify the collection JSON provided in the STIX server. It must be as provided in the plugin configuration.
4. Might be due to the host not added in the container. To add the host, follow the below steps:
   • SSH to the VM where your Cloud Exchange is installed and stop all the containers using “sudo ./stop” script.
   • Edit your docker-compose file using command “vi docker-compose.yml”.
   • Add extra hosts under the core section.
   • Format:
     extra_hosts: "<your domain>:<your ip>"
   • Save the file.
   • Start the Netskope Cloud Exchange using command sudo ./start.
   • Configure the plugin with your valid credentials for an on-premises setup.

Note

Use domain and IP of the server where your STIX/TAXII instance is hosted.

Unable to pull data with the STIX/TAXII plugin

If you are facing an issue while pulling the data via the STIX/TAXII plugin, it might be due to one of these reasons:

• No data present on the 3rd party platform.
• It may be due to the filters selected while configuring the plugin.

What to do:

1. Verify whether the IoCs are present on your platform. In the case of the XSOAR platform, you can verify it on the Indicators page.
2. Data matching the filter provided in the configuration parameters will be stored. For example for the Severity filter, only indicators with matching severity will be fetched. STIX/TAXII version 2.x only supports the Unknown severity, so to pull data using STIX/TAXII version 2.x, it is mandatory to select Unknown in the severity field. If the severity filter is having all values except Unknown severity, then no indicators will be stored in Netskope Cloud Exchange, and you will be able to see the logs for pulling at Logging.

Getting an error while enabling the plugin after a plugin upgrade

When you upgrade the STIX/TAXII plugin to the latest version, and you use the skip button while upgrading, then by default the plugin will be in disabled state, and Type of Threat data to pull field will have an older value, so you will face this error while enabling that plugin.

What to do:

If you are unable to enable the plugin after plugin upgrade, then edit the plugin and make sure that Type of Threat data to pull is has a valid value. After the plugin upgrade, you need to manually remove the old values for Type of Threat data to pull the field, and add new values from the available list.

Supported types are as below:

Known Behaviors

• We have observed that if the user upgrades the freshly configured plugin from an older version to STIX/TAXII v3.1.0, and if the initial run is not executed, then the plugin will not pull any IoCs for the range provided in the initial run.
• Only the data matching the filter provided in the configuration parameters will be stored. For example, if Severity filter is having all values except the Unknown severity, then no indicators will be stored in Cloud Exchange still you will be able to see the logs for pulling in the Logging page.

Limitation

The Last Run At plugin configuration parameter displayed while editing the plugin configuration will be not useful for STIX/TAXII v2.x (As we have implemented a circuit breaker).