ThreatConnect Plugin for Threat Exchange

ThreatConnect Plugin for Threat Exchange

This document explains how to configure ThreatConnect with the Threat Exchange module of the Netskope Cloud Exchange platform. This integration allows for sharing of file hashes and URLs from ThreatConnect to Netskope. This plugin also supports sharing indicators with the ThreatConnect platform.

Prerequisites

To complete this configuration, you need:

  • A Netskope tenant (or multiple, for example, production and development/test instances) that is already configured in Cloud Exchange.
  • A Secure Web Gateway subscription for URL sharing.
  • A Threat Prevention subscription for malicious file hash sharing.
  • A Netskope Cloud Exchange tenant with the Threat Exchange module already configured.
  • A ThreatConnect instance.
  • Connectivity to the following host: http://partnerstage.threatconnect.com/.
Compatibility

Netskope CE: v4.1.0, v4.2.0

ThreatConnect Plugin Support
Fetched indicator typesSHA256, MD5, URL (URL, Address, IPV4, IPV6, Host)
Shared indicator typesSHA256, MD5, URL (URL, Address, IPV4, IPV6, Host)
Permissions

To create an API user make sure you have the organization administrator role of the Organization Administrator for creating an API User for getting the Access ID and the Secret Key.

To pull and push data from/to ThreatConnect make sure your user has the below rights

FunctionalityAPI EndpointRequest TypePermissions
Pull Indicators/api/v3/indicators?GETSelect any one of the following:
– App Developer
– Organization Administrator
– Read Only Commenter
– Read Only User
– Sharing User
– Standard User
List Groups/api/v3/groups/GETSelect any one of the following:
– App Developer
– Organization Administrator
– Read Only Commenter
– Read Only User
– Sharing User
– Standard User
Create Group/api/v3/groups/POSTSelect any one of the following:
– App Developer
– Organization Administrator
– Sharing User
– Standard User
Push Indicators/api/v3/indicatorsPOSTSelect any one of the following:
– App Developer
– Organization Administrator
– Sharing User
– Standard User
Get Owner/api/v2/owners/mineGETSelect any one of the following:
– App Developer
– Organization Administrator
– Read Only Commenter
– Read Only User
– Sharing User
– Standard User
API Details

List of APIs used

API NameMethodAPI Endpoint
Pull IndicatorsGET/api/v3/indicators?
List GroupsGET/api/v3/groups/
Create GroupPOST/api/v3/groups/
Push IndicatorsPOST/api/v3/indicators
Get OwnerGET/api/v2/owners/mine
Performance Matrix

Below is the performance reading conducted for fetching and sharing 100K IOCs in each plugin lifecycle on a Large CE instance with the below specifications.

Stack detailsSize: Large
RAM: 32 GB
CPU: 16 Cores
Indicators fetched from ThreatConnect~6K per minute
Indicators shared to ThreatConnect~4.5K per minute
User Agent

The user agent added for this plugin is in the following format: netskope-ce-<CE VERSION>-<MODULE NAME>-<PLUGIN NAME>-v<PLUGIN VERSION>.

For example: netskope-ce-4.2.0-cte-threatconnect-v1.1.0.

Mappings

Default tags mapping for the Indicators pulled from the ThreatConnect platform.

Type of Indicator pulled from ThreatConnect Tag added in Netskope CE for each type of indicator
URL ThreatConnect-URL
Address(IPV4) ThreatConnect-Address-IPV4
Address(IPV6) ThreatConnect-Address-IPV6
Host ThreatConnect-Host
File(MD5) ThreatConnect-File-MD5
File(SHA256) ThreatConnect-File-SHA256
Mappings for Pull (Netskope field – ThreatConnect fields)
Netskope CE Fields ThreatConnect Fields
Value md5, sha256, text(URL), ip(Address), hostName(host)
Active active
Severity rating
Reputation confidence
Comments description
Firstseen dateAdded
Lastseen lastModified
Tags Tags
Mappings for Push (Netskope field – Third Party field)
Netskope CE Fields ThreatConnect Fields
value md5, sha256, text(URL), ip(Address), hostName(host)
type url, File, Host, Address
Fetched existing_group_id associatedGroups
Netskope CE tags
severity rating
reputation confidence

Workflow

  1. Create a custom File Profile.
  2. Create a Malware Detection Profile.
  3. Create a Real-time Protection Policy.
  4. Get ThreatConnect credentials.
  5. Configure a ThreatConnect Plugin.
  6. Configure sharing between Netskope and ThreatConnect.
  7. Validate the ThreatConnect Plugin.

Click play to watch a video.

 

Create a Secure Web Gateway Custom File Profile for ThreatConnect

  1. In the Netskope UI, go to Policies , select File , and click New File Profile.
    image3.jpeg
  2. Click File Hash in the left panel, select SHA256 from the File Hash dropdown list.
    image4.jpeg
  3. Enter a temporary value in the text field. Netskope does not support progressing without having a value in this field, and recommends entering a string of 64 characters that consists of the character f. For example, ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff. This will have a very low possibility of matching a valid file format.
    image5.jpeg
  4. Click Next.
  5. Enter a Profile Name and a Description. We recommend not having blank spaces in your profile name; use underscores for spaces.
    image6.jpeg
  6. Click Save.
  7. To publish this profile into the tenant, click Apply Changes in the top right.

Create a Malware Detection Profile for ThreatConnect

  1. In the Netskope UI, go to Policies, select Threat Protection , and click New Malware Detection Profile.
  2. Click Next.

    Note

    For this configuration example, we will be using the intelligence for this list as a block list. Netskope does support inclusion of both allow and block lists in the threat profiles.

  3. Click Next again.
  4. Select the File Profile you created in the previous section and click Next.
  5. Enter a Malware Detection Profile name and click Save Malware Detection Profile.
  6. To publish this profile in the tenant, click Apply Changes in the top right.

Create a Real-time Threat Protection Policy for ThreatConnect

  1. In the Netskope UI, go to Policies > Real-time Protection.

    Note

    The policy configured here is just an example. Modify as appropriate for your organization.

  2. Click New Policy and select Threat Protection.
  3. For Source, leave the default (User = All Users)
  4. For Destination: select Category
  5. The Category section expands and allows you to search and select categories. Click Select All.

    When finished, click outside of the Category section.

  6. When the Activities & Constraints section opens, click Edit.
  7. Select Upload and Download, and then click Save.
  8. For Profile & Action, click in the text field.
  9. Select the Malware Detection profile you created in the previous section.
  10. For the Severity Levels, change all of the Actions settings from Action: Alert to Action: Block.
  11. Select a template to choose which block message is sent to the user.
  12. For Set Policy, enter a descriptive Policy Name.
  13. Click Save in the top right to save the policy.
  14. Choose the To the top option when it appear. (Or appropriate location in your security policy)
  15. To publish this policy into the tenant, select Apply Changes in the top right.

Get your ThreatConnect Access ID and Secret Key

  1. Go to https://partnerstage.threatconnect.com and log in.
  2. On the top navigation bar, hover the cursor over Settings and select Org Settings.
  3. Go to the Organization Settings screen and click Create API User on the Membership tab.
  4. The API User Administration window will be displayed. Enter these parameters:
    • First Name: Enter your first name.
    • Last Name: Enter your last name.
    • Organization Role: Select an Organization role.
      • App Developer.
      • Organization Administrator
      • Sharing User
      • Standard User
    • Include in Observations and False Positives: Select the checkbox to allow data provided by the API user to be included in observation and false-positive counts.
    • Allow User to Exceed API Link Limit: Select the checkbox to override the system-level limit on the number of association levels that can be retrieved at one time for intelligence items using v3 of the ThreatConnect API.
    • Custom TQL Timeout: Select the checkbox to override the system-level ThreatConnect Query Language (TQL) query timeout for the user. In the field to the right of the checkbox, enter the maximum amount of time, in milliseconds, that TQL queries made by the user will be allowed to run before timing out.

      Note

      This checkbox will be available only when the user creating the account has a System role of Operations Administrator or Administrator.

    • Disabled: Select the checkbox to disable an API user’s account in the event that the Administrator wishes to retain log integrity when the API user no longer requires ThreatConnect access.
  5. Copy the Access ID and Secret Key, as these will not be accessible after the window is closed. These are required to configure the ThreatConnect plugin.
  6. Click Save.

Configure the ThreatConnect Plugin

  1. In Cloud Exchange, go to Settings and click Plugins.
  2. Search for and select the ThreatConnect plugin tile.
  3. Enter and select the Basic Information on the first page:
    • Configuration Name: Unique name for the configuration.
    • Sync Interval: Leave default.
    • Aging Criteria: Expiry time of the plugin in days. (Default: 90)
    • Override Reputation: Set a value to override the reputation of indicators received from this configuration.
    • Enable SSL Validation: Enable SSL Certificate validation.
    • Use System Proxy: Enable if proxy is required for communication.
  4. Click Next.
  5. Enter the Configuration Parameters on the second page:
    • Base URL: Base URL for ThreatConnect API Endpoints – https://<yourcompany>.threatconnect.com
    • Access ID: Access ID of ThreatConnect API.
    • Secret Key: Secret Key of ThreatConnect API.
    • Type of Threat IoC: Type of threat IoC you want to pull from ThreatConnect.
    • Enable Tagging: Enable/Disable tagging functionality.
    • Enable Polling: Enable/Disable polling data from ThreatConnect.
    • Initial Range: Number of days to pull the data for the initial run.
  6. Click Save.

Create a Business Rule for Sharing of Indicators

To share indicators from Cloud Exchange to ThreatConnect, and ThreatConnect’s indicators to Netskope, or any ThirdParty plugin, you need to have a business rule that will filter out the indicators that you need to share. To configure a business rule, follow these steps:

  1. In Threat Exchange, click Business Rules and then Create New Rule.
  2. Add your required filters for the IoCs you want to share.
  3. Click Save.

Configure Sharing for Netskope and ThreatConnect

The ThreatConnect plugin supports sharing of URLs, MD5, and SHA256 types of IoCs. The plugin has an Add to Group action that will add indicators to available groups in the organization. It will list all the available groups to which you can add your IoCs. To share IOCs to ThreatConnect follow these steps:

  1. Go to Threat Exchange and select Sharing. The Sharing page displays the existing relationships for each sharing configuration in grid view as shown below. The Sharing page also has inputs to configure new sharing from one plugin to another.
  2. Click Add Sharing Configuration, and in the Source Configuration dropdown list, select Netskope CTE.
  3. Select a Business Rule, and then select ThreatConnect for the Destination Configuration. Sharing configurations are unidirectional. data obtained from one plugin is shared with another plugin. To achieve bi- or multi-directional sharing, configure each separately.
  4. Select a Target. Each plugin will have a different target or destination for the IoC.
  5. For Action, select Add to Group. The Add to Group action will add indicators to available groups in the organization. It will list out all the available groups in which you can add your IoCs. It will also show one last option as “Create New Group” which will create a new group in ThreatConnect and add all the indicators to that group.
  6. Click Save.
  7. Repeat steps 2-6, but select ThreatConnect as the Source Configuration and Netskope CTE as the Destination Configuration.
  8. Click Save.

Adding a new sharing configuration on the active source poll will share the existing IoCs of the source configuration to the destination configuration. Whenever a new sharing configuration is built, all the active IoCs will also be considered for sharing if they match the source/destination combination.

Note

Plugins that do not have API for ingesting data cannot receive threat data. This is true of the installed plugin API Source, which provides a bucket associated with an API endpoint for remote 3rd-party systems to push data to. Once a Sharing policy has been added, it takes effect.

After a sharing configuration has been created, the sharing table will show the rule being invoked, the source system providing the potential IoC matches, the destination system that will receive matching IoC, and the target applicable to that rule. Multiple Sharing configurations can be made to support mapping certain IoC to multiple targets even on the system destination system.

Modify, Test, or Delete a Sharing Configuration

Each configuration supports 3 actions:

  • Edit the rule by clicking on the pencil icon.
  • Test the rule by clicking on the synchronization icon. This tests how many IoC will actually be sent to the destination system based on the timeframe and the rule.
  • Delete the rule by clicking on the garbage can icon.

Validate the ThreatConnect Plugin

Validate the Pull

We pull Addresses, Files, URLs, and Hosts from Browse > Indicators in the ThreatConnect plugin.

  1. Based on the Plugin configuration, Indicators will pull from ThreatConnect. Go to Threat Exchange > Threat IoCs to view the received IoCs.
    image4.png
  2. If data is not being fetched from the platform, you can look at the logs in Cloud Exchange. Go to Cloud Exchange > Logging and look through the logs for errors.
    image11.png

Validate the Push

  1. To verify pushed IoCs in ThreatConnect, go to ThreatConnect Platform > Browse > Indicators.
  2. IoCs pushed from Netskope are tagged with Netskope CE.
  3. To validate the pushed indicator in Cloud Exchange, go to Threat IOCs and search for IOCs that are shared with ThreatConnect.

Troubleshooting

Receiving the 400 Client Error in logs while executing the plugin life cycle

If you are receiving the above-mentioned error in logs while pulling indicators from the ThreatConnect platform, try increasing the value of the Custom TQL Timeout.

You can increase the TQL Timeout value using the following steps:

  1. Log in to your ThreatConnect account.
  2. Go to Settings > Org Settings.
  3. Edit the account that you use for generating the configuration parameters and increase the value for the Custom TQL Timeout.
Receiving an error while updating the plugin using the plugin repository

If you are facing an issue updating the configured ThreatConnect plugin, follow these steps:

  1. Close the plugin repo page once pulling and downloading the plugin updates.
  2. Go to Threat Exchange > Plugins.
  3. Edit the plugin, on the Configuration Parameters page. Remove the selected value from the Type of Threat Indicator field, and reselect the IoC type that you want to pull.
  4. Click Save.
  5. Click on the enable plugin icon and enable the plugin. The plugin will be updated with the latest changes and should start working as expected.
Share this Doc

ThreatConnect Plugin for Threat Exchange

Or copy link

In this topic ...