External Website Plugin for Threat Exchange

This document explains how to configure the External Website v1.0.0 plugin with the Threat Exchange module of the Netskope Cloud Exchange platform.

Prerequisites

• A Netskope tenant (or multiple, for example, production and development/test instances) that is already configured in Cloud Exchange.
• A Netskope Cloud Exchange tenant with the Threat Exchange module already configured.
• Connectivity to your external website.

Compatibility

This plugin is compatible with the below Netskope CE versions.

Netskope CE: v4.1.0, v4.2.0 and later.

Plugin Scope

This plugin supports pulling URLs (URLs, IPv4, Domains), SHA256, and MD5 types of indicators from any external websites. This plugin does not support pushing any indicators to the external websites.

External Website Plugin Support

Fetched indicator types	URL (URLs, IPv4, Domains), SHA256, MD5
Shared indicator types	Not Supported

Mappings

• NA

Permissions

• NA

API Details

We do not use any API to pull the data, but the file Endpoint/URL. You can use the File endpoint as your API Endpoint to pull the data.

List of APIs Used

Pull Data

Example

API Endpoint: <URL of your external website>

Method: GET

Parameters: None

API Request Endpoint

<URL used for pulling data from an External Website>, For example,

https://bitbucket.org/vrudesai/netskope_ce_vrunda/raw/0ee77838f1e1b0491c13e*********/ios.txt

Sample API Response

193.30.120.139

128.140.81.180

198.98.51.123

66.146.193.33

46.165.254.40

185.181.61.18

68.194.101.52

172.245.240.14

184.174.38.53

[2001:067c:089c:0702:01ce:01ce:babe:0009]

[2001:067c:089c:0702:01ce:01ce:babe:0008]

[2001:067c:089c:0702:01ce:01ce:babe:0007]

192.87.28.28

Performance Matrix

Below is the performance reading conducted after pulling 100K IOCs on a Large CE instance with the below specifications.

Stack details	Size: Large RAM: 32 GB CPU: 16 Cores
Indicators pulled from External Website	~20K per minute

User Agent

The user agent added for this plugin is in the following format:

netskope-ce-<CE VERSION>-<MODULE NAME>-<PLUGIN NAME>-v<PLUGIN VERSION>

Example:

Netskope-ce-4.2.0-cte-external-website-v1.0.0

Workflow

1. Configure the CTE External website plugin.
2. Configure Sharing between External Website and Netskope.
3. Validate the External Website plugin.

Click play to watch a video.

Configure the External Website Plugin

1. Go to Settings > Plugins, and search for and select the External Website plugin box.

   

2. Enter these Basic Information values:
   • Configuration Name: Enter a unique name for the configuration.
   • Sync Interval: Leave the default.
   • Aging Criteria: Enter an expiry time of the plugin in days. (Default: 90)
   • Override Reputation: Set a value to override the reputation of indicators received from this configuration.
   • Enable SSL Validation: Enable SSL Certificate validation.
   • Use System Proxy: Enable if a proxy is required for communication.

   

3. Click Next.
4. Enter these Configuration Parameter values:
   • External Website URL: Add the URL of any External website from where the data is to be pulled.
   • Type of Threat data to pull: Select the type of Threat data you want to pull.

   

5. Click Save.

Configure a Threat Exchange Business Rule for the External Website Plugin

To share indicators fetched from the External Website plugin to the Cloud Exchange, you need to have a business rule that will filter the indicators that you want to share. To configure a business rule, follow these steps:

1. Go to Threat Exchange > Business Rules > Create New Rule.
   

   

2. Add your required filter for the IoCs you want to share, and then click Save.

Configure Threat Exchange Sharing for the External Website Plugin

To share IoCs from the External Website plugin to Netskope, follow these steps:

1. Go to Threat Exchange > Sharing, and click Add Sharing Configuration.
2. Select your Source Configuration (External Website), Business Rule, Destination Configuration (Netskope), and Target, and then select the existing IoC List Name, or create a new IoC list on the platform.

   

3. Click Save.

Validate the External Website Plugin

In Cloud Exchange

Validate the Pull

1. You can verify the pulling of IoCs from the plugin by going to Logging and checking the pulled logs from the Threat Exchange External Website plugin.

   

2. You can check the pulled data stored in Cloud Exchange. Go to Threat Exchange > Threat IoCs and search for the IoCs pulled from the plugin.

   

Validate the Push

The external website plugin does not support the pushing of IoCs.

You can push the IOCs pulled from the External website to Netskope or any third-party plugin supported in Threat Exchange.

Follow these steps to verify the pushed IoCs to Netskope:

1. To validate the pushed indicator on Threat Exchange, go to Threat IoCs and search for IoCs that are shared with Netskope.
2. You can also verify the pushed IoCs from Logging in Cloud Exchange.
3. Filter the logs available from the Netskope plugin.

   

To validate the IOCs shared on Netskope, log in to your Netskope tenant and go to Policies > Web > URL Lists. Click on the URL List that you selected while configuring the sharing, and check the shared IOCs.

Troubleshooting

Unable to pull IoCs from the plugin

If you are not able to pull any IoCs from the plugin it might be due to one of the following reasons:

• IoCs are not available at all for pulling.
• The type of IoCs that are available on the External Website are either not supported for the plugin or are not selected when the plugin was configured.
• The file you are pointing to is incorrect.

What to do: If you are receiving the above issue it might be due to one of the above-mentioned points. In order to resolve this issue follow these steps respectively:

• ICs are not available at all for pulling.
  • Hit the External Website URL in any browser and check the content of the file. It should have supported Threat data to be pulled.

• The type of IoCs that are available on the External Website are either not supported for the plugin or are not selected when the plugin was configured.
  • Check the content of the file. The file should have IPv4, Domains, URLs, SHA256, and MD5 type of data.

• • Check the plugin configuration parameters and verify that the type of IoC you are trying to pull from the website is available on the website and selected in the plugin configuration.

• The file you are pointing to is incorrect.
  • Check the content of the file by hitting the URL(External Website URL) as a GET request in Postman. The response should be a valid regex for supported IOC types.