MISP Plugin for Threat Exchange - Netskope Technical Documentation

Release Notes

1.5.2 (Required minimum CE version for this is 7.0.0)

Added

Added support for IPv4 and IPv6 CIDR IOCs.

1.5.1

Fixed

Fixed an issue where excludeDecayed=1 was being sent even when the ‘Decaying Score Threshold’ parameter value was not provided in the configuration parameters.

1.5.0

Added

Added support for IoC(s) Retraction.
Added support for IoC Source Labelling.
Added support for pulling Hostname, Destination IP (ip-dst), Destination IP|Port (ip-dst|port), Domain|IP (domain|ip), Source IP|Port (ip-src|port), and Hostname|Port (hostname|port) from MISP.

1.4.1

Added

Added support for accepting multiple events in the “Exclude IOCs from Event” parameter.

1.4.0

Added

Added support for pulling indicators on the basis of Decaying Score Threshold, Decaying Model IDs, Published Events, IDS flag and Enforce Warning List IOCs flag.
Added support to divide the URL by types (Domain, IPv4, IPv6) starting from CE v5.0.1.
Added the “netskope-ce” tag when sharing IOCs to fix the issue creating the IOC cycle in Cloud Exchange.

1.3.0
Added

Added additional optimizations for a large number of indicators.

1.2.0
Added

Added batching of 2500 in push functionality.
Added new configuration parameters for defining the pulling mechanism (Incremental or Look Back) and the look back time in hours.

1.1.0
Added

Added new configuration parameters.

1.0.0
Added

Initial release.

This document explains how to configure the MISP v1.5.2 plugin with the Threat Exchange module of the Netskope Cloud Exchange platform. This plugin is used to fetch event attributes from MISP (Malware Information Sharing Platform) and extract indicators of type SHA256, MD5, URL, Domain, IP (IPv4 and IPv6) and Hostname from them. It can also share the indicators of type SHA256, MD5, URL, Domain (Domain and FQDN), Hostname, IP (IPv4 and IPv6) as attributes to MISP Custom Events. To get required details for creating a new configuration, go to https://<misp-url>/events/automation.

The Source IP (ip-src) and Destination IP (ip-dst) will be stored as either IPv4 or IPv6 in Cloud Exchange. Source IP|Port (ip-src|port), Destination IP|Port (ip-dst|port), and Hostname|Port (hostname|port) will be stored as URLs in Cloud Exchange. For Domain|IP, the domain and IP (either IPv4 or IPv6) will be split and stored as separate IoCs in Cloud Exchange. This plugin also supports pulling IPv4 and IPv6 CIDR IoCs.

Prerequisites

To complete this configuration, you need:

A Netskope tenant (or multiple, for example, production and development/test instances) that is already configured in Cloud Exchange.
A File Profile on your Netskope tenant.
A URL List on your Netskope tenant.
A Destination Profile on your Netskope tenant.
A Private App on your Netskope tenant.
A DNS Profile on your Netskope Tenant
A Netskope Cloud Exchange tenant with the Tenant plugin and Threat Exchange plugin already configured.
A MISP Instance.
Connectivity to the following host: the MISP platform (https://<misp-URL>)

MISP Plugin Support

This plugin is used to fetch event attributes from MISP (Malware Information Sharing Platform) and extract indicators of type SHA256, MD5, URL, Domain, IP (IPv4 and IPv6) and Hostname from them. It can also share the indicators of type SHA256, MD5, URL, Domain (Domain and FQDN), Hostname, IP (IPv4 and IPv6) as attributes to MISP Custom Events. To get required details for creating a new configuration, go to https://<misp-url>/events/automation.

Fetched Indicator Types	Shared Indicator Types
MD5, SHA256, URL (Source IP\|Port [ip-src\|port], Destination IP\|Port [ip-dst\|port], and Hostname\|Port [hostname\|port]), Domain, IPv4 and IPv6 (Source IP [ip-src] and Destination IP [ip-dst]), Domain ( Domain\|IP), Hostname, IPv4 and IPv6 CIDR	MD5, SHA256, URL, Domain (Domain, FQDN and Hostname), IP (IPv4 and IPv6)

IoC Retraction

IoC Retraction (Pull): IOCs that are deleted on the MISP or not under the retraction interval set in the plugin configuration or not under the selected MISP Attribute Type, MISP Attribute Category, MISP Attribute Tags, Event Names, Exclude IoCs from Events, IoC Event Type, Decaying Score Threshold, Decaying Model IDs, Filter on IDS flag, and Enforce Warning List IoCs will be marked as retraced in Cloud Exchange.

IoC Retraction (Push): Retracted indicators present on CE will be deleted from MISP during sharing.

Retraction Type	Supported Retraction Type
IOC Retraction (Pull)	Yes
IOC Retraction (Push)	Yes

Mappings

Pull Mapping

Cloud Exchange Fields	MISP API Response Fields
value	value
type	type: md5, sha256, url, domain, ip-src, ip-dst, ip-src\|port, ip-dst\|port, hostname\|port, domain\|ip ipv4_cidr and ipv6_cidr IOC types (MISP does not have these types; while pulling data from MISP, the plugin processes the CIDR values and stores them as ipv4_cidr and ipv6_cidr in CE).
first_seen	first_seen
last_seen	last_seen
Comment	Comment \| Decaying Score: <DecayingModel.score>, Decaying Model ID: <DecayingModel.id>, Decaying Model Name: <DecayingModel.name>
tags	Tag + MISPCATEGORY-<category>
extendedInformation	<Base URL>//events/view/<event_id>

Push Mapping

Cloud Exchange Fields	MISP API Response Fields
value	value
type	type: md5, sha256, ip-src, url, domain, hostname Note that the Domain, FQDN, Hostname in CE will be shared as domain to MISP Platform.
comment	comment
first_seen	firstSeen
last_seen	lastSeen
Tag	netskope-ce Netskope CE \| <Source Plugin Name>

Permissions

Admin permissions is required to generate an Authentication Key.

API Details

List of APIs used

API Endpoint	Method	Use Case
/events/restSearch	POST	Check event existence
/attributes/restSearch	POST	Pull attributes
/attributes/deleteSelected/<Event ID>	POST	Delete/retract previously-pushed attributes from MISP (push retraction)
/tags/search/<Tag name>	POST	Check tag existence
/tags/add	POST	Create tag
/events/edit/<Event ID>	POST	Share indicators to existing event
/events/add	POST	Share indicators to new event

Check Event Existence

API Endpoint: /events/restSearch
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value
returnFormat	json
limit	1
page	1
eventinfo	<Event Name>
metadata	true

Sample API Response

{
  "response": [
    {
      "Event": {
        "id": "1517",
        "orgc_id": "1",
        "org_id": "1",
        "date": "2024-06-25",
        "threat_level_id": "4",
        "info": "new",
        "published": false,
        "uuid": "09502561-a07f-41e9-8359-59394878e47d",
        "attribute_count": "11",
        "analysis": "0",
        "timestamp": "1719493608",
        "distribution": "1",
        "proposal_email_lock": false,
        "locked": false,
        "publish_timestamp": "0",
        "sharing_group_id": "0",
        "disable_correlation": false,
        "extends_uuid": "",
        "protected": null,
        "event_creator_email": "admin@admin.test",
        "Org": {
          "id": "1",
          "name": "ORGNAME",
          "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
          "local": true
        },
        "Orgc": {
          "id": "1",
          "name": "ORGNAME",
          "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
          "local": true
        },
        "RelatedEvent": [
          {
            "Event": {
              "id": "1325",
              "date": "2024-07-05",
              "threat_level_id": "4",
              "info": "new5",
              "published": false,
              "uuid": "3508388f-b6a6-4be2-9b99-de05825cc304",
              "analysis": "0",
              "timestamp": "1720177270",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1324",
              "date": "2024-07-02",
              "threat_level_id": "4",
              "info": "new4",
              "published": false,
              "uuid": "e8d24ff9-2955-4eb6-80f1-5f276fae2e42",
              "analysis": "0",
              "timestamp": "1720424260",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1319",
              "date": "2024-06-27",
              "threat_level_id": "4",
              "info": "420test",
              "published": false,
              "uuid": "1e1f4455-8d0e-43dd-95ba-0fd4f7eb6abf",
              "analysis": "0",
              "timestamp": "1720418522",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1315",
              "date": "2024-03-12",
              "threat_level_id": "4",
              "info": "test",
              "published": false,
              "uuid": "dceaa84e-156f-4a31-b0b2-a3ae0778d72a",
              "analysis": "0",
              "timestamp": "1720175425",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1308",
              "date": "2015-01-01",
              "threat_level_id": "1",
              "info": "testevent",
              "published": false,
              "uuid": "0c3d5017-8edf-483b-a829-426bb2c56c92",
              "analysis": "0",
              "timestamp": "1699523309",
              "distribution": "0",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          }
        ],
        "Galaxy": [],
        "CryptographicKey": [],
        "Tag": [
          {
            "id": "251",
            "name": "workflow:todo=\"create-missing-misp-galaxy-cluster\"",
            "colour": "#770040",
            "exportable": true,
            "user_id": "0",
            "hide_tag": false,
            "numerical_value": null,
            "is_galaxy": false,
            "is_custom_galaxy": false,
            "local_only": false,
            "local": 0,
            "relationship_type": null
          }
        ]
      }
    }
  ]
}

Pull Attributes

API Endpoint: /attributes/restSearch
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value	Comment
returnFormat	json	The format to return data in. e.g. json, csv, etc.
limit	1	Maximum records to return
page	1	Page number
eventid	< Event ID>	Event ID
timestamp	[ <Start Time>, <End Time>]	Restrict the results by the timestamp (last edit)
category	[<Categories Name>]	Categories of attributes to pull.
type	[“md5″,”sha256″,”ip-src”,”domain”,”ip-src\|port”, “ip-dst”,”ip-dst\|port”,”domain\|ip”,”hostname”,”hostname\|port”]	Type of indicators to pull
tags	[“!netskope-ce”]	Include or exclude attributes with certain tags.
includeDecayScore	1	If set to 1, decay score information will be included for attributes that are affected by decaying.
decayingModel	[1,2]	Allows you to set the decaying model(s) to use to calculate the decay score.
excludeDecayed	1	Filter out all expired IoCs.
modelOverrides.threshold	30	JSON that can be used to modify Model parameters on-the-fly.
published	1	Set whether published or unpublished events should be returned Accepted values 0 or 1.
to_ids	1	By default (0) all attributes are returned that match the other filter parameters, regardless of their to_ids setting.
enforceWarninglist	1	Remove any attributes from the result that would cause a hit on a warninglist entry.

Sample API Response

{
  "response": {
    "Attribute": [
      {
        "id": "347224",
        "event_id": "1315",
        "object_id": "0",
        "object_relation": null,
        "category": "Network activity",
        "type": "ip-src",
        "to_ids": true,
        "uuid": "567ac850-6337-4266-a646-9317661f8974",
        "timestamp": "1719987636",
        "distribution": "5",
        "sharing_group_id": "0",
        "comment": "",
        "deleted": false,
        "disable_correlation": false,
        "first_seen": null,
        "last_seen": null,
        "value": "efd:2dbe:3d3:b9ac:71e1:fd4f:5d2:2de7",
        "decay_score": [
          {
            "score": 30.478404654276954,
            "base_score": 90,
            "decayed": false,
            "DecayingModel": {
              "id": "2",
              "name": "NIDS Simple Decaying Model Updated"
            }
          }
        ],
        "Event": {
          "org_id": "1",
          "distribution": "1",
          "id": "1315",
          "info": "test",
          "orgc_id": "1",
          "uuid": "dceaa84e-156f-4a31-b0b2-a3ae0778d72a"
        }
      }
    ]
  }
}

Delete/retract previously-pushed attributes from MISP (push retraction)

API Endpoint: /attributes/deleteSelected/<Event ID>

Method: POST

Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Payload

Key	Value
id	[“attribute_id”]
event_id	“event_id”

Sample API Response

{
    "saved": true,
    "success": true,
    "name": "1 attribute deleted.",
    "message": "1 attribute deleted.",
    "url": "/attributes/deleteSelected/141",
    "id": "141"
}

Check a Tag’s Existence on MISP

API Endpoint: /tags/search/<Tag name>
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value
limit	5000
offset	“”

Sample API Response

  {
    "Tag": {
      "id": "1399",
      "name": "netskope-ce",
      "colour": "#ff0000",
      "exportable": true,
      "org_id": "0",
      "user_id": "0",
      "hide_tag": false,
      "numerical_value": null,
      "is_galaxy": false,
      "is_custom_galaxy": false,
      "local_only": false
    }
  }
]

Create a Tag on MISP

API Endpoint: /tags/add
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value
name	<Tag Name>
colour	#ff0000

Sample API Response

{
  "Tag": {
    "id": "1400",
    "name": "netskooe-ce",
    "colour": "#ff0000",
    "exportable": true,
    "org_id": "0",
    "user_id": "0",
    "hide_tag": false,
    "numerical_value": null,
    "is_galaxy": false,
    "is_custom_galaxy": false,
    "local_only": false
  }
}

Share Indicators to an Existing Event (Update Event)

API Endpoint: /events/edit/<Event ID>
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value	Comment
type	sha256	Indicator Type
value	56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a	SHA256 value
comment	Test IoC
first_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
last_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
Tag.name	netskope-ce \| <Source Plugin name>

Sample Payload

{
  "Attribute": [
    {
      "type": "sha256",
      "value": "56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a",
      "comment": "Test IoC.",
      "first_seen": "2024-07-05T09:48:51.585000",
      "last_seen": "2024-07-05T09:48:51.585000",
      "Tag": [
        {
          "name": "netskope-ce"
        }
      ]
    }
  ]
}

Sample API Response

{
  "Event": {
    "id": "1317",
    "orgc_id": "1",
    "org_id": "1",
    "date": "2024-06-25",
    "threat_level_id": "4",
    "info": "new",
    "published": false,
    "uuid": "09502561-a07f-41e9-8359-59394878e47d",
    "attribute_count": "12",
    "analysis": "0",
    "timestamp": "1720443083",
    "distribution": "1",
    "proposal_email_lock": false,
    "locked": false,
    "publish_timestamp": "0",
    "sharing_group_id": "0",
    "disable_correlation": false,
    "extends_uuid": "",
    "protected": null,
    "event_creator_email": "admin@admin.test",
    "Org": {
      "id": "1",
      "name": "ORGNAME",
      "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
      "local": true
    },
    "Orgc": {
      "id": "1",
      "name": "ORGNAME",
      "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
      "local": true
    },
    "Attribute": [
      {
        "id": "1488555",
        "type": "sha256",
        "category": "Payload delivery",
        "to_ids": true,
        "uuid": "343b7c36-beb7-4197-945e-4b35c8179426",
        "event_id": "1317",
        "distribution": "5",
        "timestamp": "1720443083",
        "comment": "Test IoC.",
        "sharing_group_id": "0",
        "deleted": false,
        "disable_correlation": false,
        "object_id": "0",
        "object_relation": null,
        "first_seen": "2024-07-05T09:48:51.585000+00:00",
        "last_seen": "2024-07-05T09:48:51.585000+00:00",
        "value": "56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a",
        "Galaxy": [
          
        ],
        "ShadowAttribute": [
          
        ],
        "Tag": [
          {
            "id": "1399",
            "name": "netskope-ce",
            "colour": "#ff0000",
            "exportable": true,
            "user_id": "0",
            "hide_tag": false,
            "numerical_value": null,
            "is_galaxy": false,
            "is_custom_galaxy": false,
            "local_only": false,
            "local": 0,
            "relationship_type": null
          }
        ]
      }
    ],
    "ShadowAttribute": [
      
    ],
    "RelatedEvent": [
      {
        "Event": {
          "id": "1325",
          "date": "2024-07-05",
          "threat_level_id": "4",
          "info": "new5",
          "published": false,
          "uuid": "3508388f-b6a6-4be2-9b99-de05825cc304",
          "analysis": "0",
          "timestamp": "1720177270",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1324",
          "date": "2024-07-02",
          "threat_level_id": "4",
          "info": "new4",
          "published": false,
          "uuid": "e8d24ff9-2955-4eb6-80f1-5f276fae2e42",
          "analysis": "0",
          "timestamp": "1720424260",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1319",
          "date": "2024-06-27",
          "threat_level_id": "4",
          "info": "420test",
          "published": false,
          "uuid": "1e1f4455-8d0e-43dd-95ba-0fd4f7eb6abf",
          "analysis": "0",
          "timestamp": "1720418522",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1315",
          "date": "2024-03-12",
          "threat_level_id": "4",
          "info": "test",
          "published": false,
          "uuid": "dceaa84e-156f-4a31-b0b2-a3ae0778d72a",
          "analysis": "0",
          "timestamp": "1720175425",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1308",
          "date": "2015-01-01",
          "threat_level_id": "1",
          "info": "testevent",
          "published": false,
          "uuid": "0c3d5017-8edf-483b-a829-426bb2c56c92",
          "analysis": "0",
          "timestamp": "1699523309",
          "distribution": "0",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      }
    ],
    "Galaxy": [
      
    ],
    "Object": [
      
    ],
    "EventReport": [
      
    ],
    "CryptographicKey": [
      
    ],
    "Tag": [
      {
        "id": "251",
        "name": "workflow:todo=\"create-missing-misp-galaxy-cluster\"",
        "colour": "#770040",
        "exportable": true,
        "user_id": "0",
        "hide_tag": false,
        "numerical_value": null,
        "is_galaxy": false,
        "is_custom_galaxy": false,
        "local_only": false,
        "local": 0,
        "relationship_type": null
      }
    ]
  }
}

Share Indicators to a New Event (Create Event)

API Endpoint: /events/add
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value	Comment
type	sha256	Indicator Type
value	56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a	SHA256 value
comment	Test IoC.
first_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
last_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
Tag.name	netskope-ce

Sample Payload

{
"info": "test"
  "Attribute": [
    {
      "type": "sha256",
      "value": "56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a",
      "comment": "Test IoC.",
      "first_seen": "2024-07-05T09:48:51.585000",
      "last_seen": "2024-07-05T09:48:51.585000",
      "Tag": [
        {
          "name": "netskope-ce"
        }
      ]
    }
  ]
}

Sample API Response

{
  "Tag": {
    "id": "1400",
    "name": "netskope-ce",
    "colour": "#ff0000",
    "exportable": true,
    "org_id": "0",
    "user_id": "0",
    "hide_tag": false,
    "numerical_value": null,
    "is_galaxy": false,
    "is_custom_galaxy": false,
    "local_only": false
  }
}

Performance Matrix

Here is the performance reading conducted by pulling and sharing 100K indicators from/to MISP on a Large CE Stack with these specifications.

Description	Specification
Stack details	Size: Large RAM: 32 GB CPU: 16 Cores
Indicators fetched from MISP Platform	~18K per minute
Indicators shared with MISP Platform	~8K per minute

User Agent

netskope-ce-7.0.0-cte-misp-v1.5.2

Workflow

Get your MISP URL and API key.
Configure the MISP Plugin.
Configure a Business Rule.
Configure Sharing between Netskope and MISP.
Validate the MISP Plugin.

Watch a Video

Click play to watch a video:

Get your Instance URL and Auth Key

Click Add authentication key, enter the required parameters, and click Submit.

Your MISP API key will be generated. Make a copy of this value for when you configure the Plugin.

Configure the MISP Plugin

In Cloud Exchange, go to Settings > Plugin Store. Search for and select the MISP v1.5.2 (CTE) plugin.
Enter and select the Basic Information:
- Configuration Name: Plugin configuration name
- Sync Interval: Interval to fetch data from this plugin source.
- Aging Criteria: Expire indicators after a specific time.
- Override Reputation: Set value to override reputation of indicators received from this configuration. Leave empty to keep default.
- Enable SSL Validation: Enable/Disable SSL Certificate validation based on your platform requirement.
- Use System Proxy: Use system proxy configured in Settings.
Click Next and enter the Configuration Parameters:
- MISP Base URL: Base URL of MISP instance. (https://<misp-url>)
- Authentication Key: API Key generated from the MISP platform.
- MISP Attribute Type: Select MISP attribute type(MD5, SHA256, URL (Source IP|Port (ip-src|port), Destination IP|Port (ip-dst|port), and Hostname|Port (hostname|port)), Domain, IPv4 and IPv6( Source IP(ip-src) and Destination IP (ip-dst)), Domain( Domain|IP), Hostname), Indicators from only specified attribute types will be fetched. Keep empty to fetch indicators of all types. Multiple Types are accepted.
- MISP attribute Category: Select MISP attribute Category, Indicators from only specified Attribute Categories will be fetched. Keep empty to fetch indicators of all Categories. Multiple Categories are accepted.
- MISP Attribute Tags: Enter MISP Attribute Tags, Indicators from only specified comma-separated Tags will be fetched. Keep empty to fetch indicators of all Tags. Dynamic values are accepted.
- Event Names: Leave Event Names blank to fetch indicators from all the events, or enter them separately by comma to fetch only those indicators which belong to that event. For now, we leave it blank.
- Exclude IoCs from Event: In Exclude IoCs from Event , enter the name of the event whose IoCs you want to exclude from being fetched. Indicators attached to the provided comma-separated events will be ignored while pulling data from MISP. Expected value is comma-separated event names or event IDs.
- Ioc Event Type: Indicators will be pulled based on the selected event type. Published, Unpublished or both types of events can be selected. Keep empty to fetch all types
- Decaying Score Threshold: Only indicators having Decaying Score greater than the provided value will be pulled. Value should be in the range of 0 to 100.
- Decaying Model IDs: Decaying score of only specified comma-separated decaying models will be tracked. Keep blank to fetch scores for all enabled decaying models that apply to the attribute type. Decaying model IDs can be found from Global Action > List Decaying Models.
- Filter on IDS Flag: Pull IoCs based on the Selected option for IDS. Enabled IDS flag, Disabled IDS flag or both types of indicators can be selected. Keep empty to fetch all indicators.
- Enforce Warning List IoCs: Select Yes to remove any IoC from the events that would cause a hit on a warning list entry. Warning List can be found from Input Filters > Warninglists.
- Pulling Mechanism: Select a Pulling Mechanism
  - Incremental (Default): Plugin will fetch data using stored checkpoint, like last_run_at of the plugin.
  - Look Back: Plugin will fetch data subtracting lookback value from the plugin run time on every sync. For ex: If the provided value is 72 Hrs then it will fetch 72 Hrs of IoCs from now on every sync.
- Look Back (in hours): Enter Look Back (in hours) to pull the indicators of historical time from now on every time the plugin syncs. Default value is set to empty and it will only be used when you select Look Back in Pulling Mechanism.
- Retraction Interval (in days): Retraction Interval days to run IoC(s) retraction for MISP indicators. Note: This parameter will only be considered if IoC(s) Retraction is enabled in Threat Exchange Settings. The minimum value expected for the retraction will be 1. If the retraction value is not added, and IoC(s) Retraction is enabled in the global setting, retraction will not take place for the plugin.
- Enable Tagging: Plugin will pull tags associated with the IoCs if Enable tagging is Yes. If you do not want to pull tags with the IoCs, keep Enable tagging as No.
- Initial Range (in days): Number of days to pull the data for the initial run. Note that this parameter will only be considered if Pulling Mechanism is set to Incremental.
Click Save.

Note

IoCs stored in Netskope CE will have the current date and time as Last_Seen rather than the MISP’s last seen.

Configure a Threat Exchange Business Rule for MISP

A Business Rule is used to filter out the indicators that are to be shared. In order to share IoCs with MISP, create a business rule using the following steps:

In Threat Exchange, go to Business Rules and click Create New Rule.
Enter a Rule name and select the fields for which you want to filter the IoCs as described in the next two steps.
Click Create New Rule. Enter a Rule name and select the fields for which you want to filter the IoCs. For example, IoCs by Sources is equal to CTE Netskope. Click Save.
Click Create New Rule. Enter a Rule name and select the fields for which you want to filter the IoCs. IoCs by Sources is equal to CTE MISP. Click Save.

Configure Threat Exchange Sharing for MISP

In Threat Exchange, go to Sharing and click Add Sharing Configuration.
Select the Source configuration (Source from which you want to share data to MISP), select a Business Rule, and select the Destination (MISP Plugin).
Select the Target value Add to Event.
Enter the Event Name that you want to use for pushing IoCs on the MISP Platform.
Select the type for sharing the IPv4/IPv6/CIDR IoCs to MISP.
Click Save.

Note

Refer to the Netskope Threat Exchange plugin guide for more information related to sharing on Netskope Tenant. As the MISP plugin supports pulling SHA256, MD5, URL, Domain, IPv4, IPv6 and Hostname, we can perform below actions on Netskope using these indicators:

For URL, Domain, IPv4, IPv6 and Hostname

Add to URL List
Add to Private App
Add to Destination Profile
Add to DNS Profile

For SHA256 and MD5

Add to File Hash List

Validate the MISP Plugin

Validate the Pull

Indicators from MISP are pulled from: Event Actions > List Attributes.

Indicators stored in Cloud Exchange can be verified from the Threat Exchange > Threat IoCs page.

Search for the MISP IoCs by filtering indicators on the Threat IoCs page with source name as Configured MISP Plugin.

Example: Add a query on the Threat IoCs page like “sources.source Is equal “CTE MISP” && type IN (“<IOC_TYPE>”)”

You can also verify the indicators pulled in Cloud Exchange from the logs available at Logging.

For verifying the Retracted IoCs from MISP, check the logs for IoC Retraction. For example: message Like “CTE MISP [CTE MISP] [Retraction]:”

When the IoCs shared from MISP to 3rd-Party will be retracted, it will be marked as “<plugin-config-name>: retracted” in the Retraction Result. If they are not deleted from the 3rd-party, the Retraction Result will be pending.

Note

The IoCs that are deleted on the MISP and fall under the Retraction Interval will be marked as Retracted in Cloud Exchange.

Validate the Push

Shared IoCs to MISP can be verified from logs available at Logging in Cloud Exchange.

IoCs shared on MISP can be verified from Event Actions > List Events.

Click on the Event ID that you used while configuring the sharing to view all the shared IoCs.

The IoC Labeling is added to each of the IoCs that are shared/pushed to MISP from Cloud Exchange, the format for the same will be “Netskope CE | <plugin-name>”

The IoC labeling for the Shared IoCs will be added as a part of tags on MISP; it can also be verified by the log available in Cloud Exchange Logging while IoCs are shared to MISP.

Troubleshooting the MISP Plugin

Error while upgrading the plugin repository

While trying to upgrade a plugin if the below error is received, follow the steps provided.

What to do:

Click Skip and go to Home > Threat Exchange > Plugins.
Edit the plugin as it will be disabled due to an error while upgrading the plugin.
Click Next.

Scroll down to newly added params (Pulling Mechanism and Look Back) and select the method of pulling and select Look Back as the Pulling Mechanism.

Click Save and then enable the plugin.

Unable to pull/push data to MISP Plugin

If you are not able to share IoCs from Netskope to MISP, that could be the Authentication Key of MISP could have either expired, or has been deleted from the MISP Platform.

What to do:

Make sure that all the Authentication Key of MISP is not expired. Also ensure that the Key is not deleted from the MISP. Authentication Key and its details can be found at Administration > List Auth Keys.

Pulled domains and IPs are stored as URL in Cloud Exchange

If you have data pulled from MISP and the IoCs of type domain, IPv4 and IPv6 are stored as URL in Cloud Exchange, it could be due to one of the following reasons:

Plugin is not updated to v1.4.0
Plugin version is v1.4.0 but core is not updated to v5.0.1
IoC type on MISP is URL

What to do:

The URL bifurcation for the MISP plugin is available from MISP plugin v1.4.0, if your plugin is any version below this the bifurcation update won’t be available.
The URL bifurcation is supported in Cloud Exchange from core (CE version) v5.0.1, if your Cloud Exchange is not updated to the supported version the bifurcation will not be available and all domains and IPs pulled from the platform will be stored as URL. If you want to pull data as per the URL bifurcation your plugin version must be v1.4.0 or above and core version as v5.0.1 or above.
If both your core and plugin are updated and still few of the IoCs of type domain, IPv4 and IPv6 are stored as URLs in Cloud Exchange, it must be due to the IoC type of the indicator on MISP. If the IoC on MISP has type as URL for the domains/IPs, it will be pulled and stored as URL in Cloud Exchange. If MISP has domain as IOC type, it will be pulled and stored as domain in Cloud Exchange. The IoC type for IPs on MISP is ip-src.

Limitation

Decaying Score is not in sync with MISP for Incremental Pulling Mechanism

If the user is using the Incremental pulling mechanism, after an IoC is pulled from MISP along with its decaying score, the IOoC will only be pulled again into Cloud Exchange if the timestamp for that indicator gets updated on MISP. Otherwise, the decaying score will remain as it was when initially pulled.
This is a limitation of the MISP platform: when updating the decaying score, the timestamp associated with the attribute is not updated. In Cloud Exchange, we use the timestamp fields from MISP attributes to capture only the updated indicators. Therefore, if the timestamp value is not updated, the IoC will never be pulled again into Cloud Exchange.
To address this, the user can use the Look Back pulling mechanism to pull all the indicators based on the Look Back period specified in the configuration parameters for each pull.

Known Behavior

Difference in response displayed on the UI in case Decaying Model IDs field is left empty and when all Model IDs are added

If no Model IDs are provided in the Decaying Model IDs field, the decaying scores associated with the attribute will be fetched in Cloud Exchange. For example, if there are three models in the MISP instance and only one model is attached to the attribute, only the decaying score for that specific model will be fetched in Cloud Exchange.
However, if specific Model IDs are provided, whether they are associated with the attribute or not, the scores for all available models will be returned via the API. In the MISP UI, only the associated models will be shown.

Release Notes

1.5.2 (Required minimum CE version for this is 7.0.0)

Added

Added support for IPv4 and IPv6 CIDR IOCs.

1.5.1

Fixed

Fixed an issue where excludeDecayed=1 was being sent even when the ‘Decaying Score Threshold’ parameter value was not provided in the configuration parameters.

1.5.0

Added

Added support for IoC(s) Retraction.
Added support for IoC Source Labelling.
Added support for pulling Hostname, Destination IP (ip-dst), Destination IP|Port (ip-dst|port), Domain|IP (domain|ip), Source IP|Port (ip-src|port), and Hostname|Port (hostname|port) from MISP.

1.4.1

Added

Added support for accepting multiple events in the “Exclude IOCs from Event” parameter.

1.4.0

Added

Added support for pulling indicators on the basis of Decaying Score Threshold, Decaying Model IDs, Published Events, IDS flag and Enforce Warning List IOCs flag.
Added support to divide the URL by types (Domain, IPv4, IPv6) starting from CE v5.0.1.
Added the “netskope-ce” tag when sharing IOCs to fix the issue creating the IOC cycle in Cloud Exchange.

1.3.0
Added

Added additional optimizations for a large number of indicators.

1.2.0
Added

Added batching of 2500 in push functionality.
Added new configuration parameters for defining the pulling mechanism (Incremental or Look Back) and the look back time in hours.

1.1.0
Added

Added new configuration parameters.

1.0.0
Added

Initial release.

Prerequisites

To complete this configuration, you need:

A Netskope tenant (or multiple, for example, production and development/test instances) that is already configured in Cloud Exchange.
A File Profile on your Netskope tenant.
A URL List on your Netskope tenant.
A Destination Profile on your Netskope tenant.
A Private App on your Netskope tenant.
A DNS Profile on your Netskope Tenant
A Netskope Cloud Exchange tenant with the Tenant plugin and Threat Exchange plugin already configured.
A MISP Instance.
Connectivity to the following host: the MISP platform (https://<misp-URL>)

MISP Plugin Support

Fetched Indicator Types	Shared Indicator Types
MD5, SHA256, URL (Source IP\|Port [ip-src\|port], Destination IP\|Port [ip-dst\|port], and Hostname\|Port [hostname\|port]), Domain, IPv4 and IPv6 (Source IP [ip-src] and Destination IP [ip-dst]), Domain ( Domain\|IP), Hostname, IPv4 and IPv6 CIDR	MD5, SHA256, URL, Domain (Domain, FQDN and Hostname), IP (IPv4 and IPv6)

IoC Retraction

IoC Retraction (Push): Retracted indicators present on CE will be deleted from MISP during sharing.

Retraction Type	Supported Retraction Type
IOC Retraction (Pull)	Yes
IOC Retraction (Push)	Yes

Mappings

Pull Mapping

Cloud Exchange Fields	MISP API Response Fields
value	value
type	type: md5, sha256, url, domain, ip-src, ip-dst, ip-src\|port, ip-dst\|port, hostname\|port, domain\|ip ipv4_cidr and ipv6_cidr IOC types (MISP does not have these types; while pulling data from MISP, the plugin processes the CIDR values and stores them as ipv4_cidr and ipv6_cidr in CE).
first_seen	first_seen
last_seen	last_seen
Comment	Comment \| Decaying Score: <DecayingModel.score>, Decaying Model ID: <DecayingModel.id>, Decaying Model Name: <DecayingModel.name>
tags	Tag + MISPCATEGORY-<category>
extendedInformation	<Base URL>//events/view/<event_id>

Push Mapping

Cloud Exchange Fields	MISP API Response Fields
value	value
type	type: md5, sha256, ip-src, url, domain, hostname Note that the Domain, FQDN, Hostname in CE will be shared as domain to MISP Platform.
comment	comment
first_seen	firstSeen
last_seen	lastSeen
Tag	netskope-ce Netskope CE \| <Source Plugin Name>

Permissions

Admin permissions is required to generate an Authentication Key.

API Details

List of APIs used

API Endpoint	Method	Use Case
/events/restSearch	POST	Check event existence
/attributes/restSearch	POST	Pull attributes
/attributes/deleteSelected/<Event ID>	POST	Delete/retract previously-pushed attributes from MISP (push retraction)
/tags/search/<Tag name>	POST	Check tag existence
/tags/add	POST	Create tag
/events/edit/<Event ID>	POST	Share indicators to existing event
/events/add	POST	Share indicators to new event

Check Event Existence

API Endpoint: /events/restSearch
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value
returnFormat	json
limit	1
page	1
eventinfo	<Event Name>
metadata	true

Sample API Response

{
  "response": [
    {
      "Event": {
        "id": "1517",
        "orgc_id": "1",
        "org_id": "1",
        "date": "2024-06-25",
        "threat_level_id": "4",
        "info": "new",
        "published": false,
        "uuid": "09502561-a07f-41e9-8359-59394878e47d",
        "attribute_count": "11",
        "analysis": "0",
        "timestamp": "1719493608",
        "distribution": "1",
        "proposal_email_lock": false,
        "locked": false,
        "publish_timestamp": "0",
        "sharing_group_id": "0",
        "disable_correlation": false,
        "extends_uuid": "",
        "protected": null,
        "event_creator_email": "admin@admin.test",
        "Org": {
          "id": "1",
          "name": "ORGNAME",
          "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
          "local": true
        },
        "Orgc": {
          "id": "1",
          "name": "ORGNAME",
          "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
          "local": true
        },
        "RelatedEvent": [
          {
            "Event": {
              "id": "1325",
              "date": "2024-07-05",
              "threat_level_id": "4",
              "info": "new5",
              "published": false,
              "uuid": "3508388f-b6a6-4be2-9b99-de05825cc304",
              "analysis": "0",
              "timestamp": "1720177270",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1324",
              "date": "2024-07-02",
              "threat_level_id": "4",
              "info": "new4",
              "published": false,
              "uuid": "e8d24ff9-2955-4eb6-80f1-5f276fae2e42",
              "analysis": "0",
              "timestamp": "1720424260",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1319",
              "date": "2024-06-27",
              "threat_level_id": "4",
              "info": "420test",
              "published": false,
              "uuid": "1e1f4455-8d0e-43dd-95ba-0fd4f7eb6abf",
              "analysis": "0",
              "timestamp": "1720418522",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1315",
              "date": "2024-03-12",
              "threat_level_id": "4",
              "info": "test",
              "published": false,
              "uuid": "dceaa84e-156f-4a31-b0b2-a3ae0778d72a",
              "analysis": "0",
              "timestamp": "1720175425",
              "distribution": "1",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          },
          {
            "Event": {
              "id": "1308",
              "date": "2015-01-01",
              "threat_level_id": "1",
              "info": "testevent",
              "published": false,
              "uuid": "0c3d5017-8edf-483b-a829-426bb2c56c92",
              "analysis": "0",
              "timestamp": "1699523309",
              "distribution": "0",
              "org_id": "1",
              "orgc_id": "1",
              "Org": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              },
              "Orgc": {
                "id": "1",
                "name": "ORGNAME",
                "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
              }
            }
          }
        ],
        "Galaxy": [],
        "CryptographicKey": [],
        "Tag": [
          {
            "id": "251",
            "name": "workflow:todo=\"create-missing-misp-galaxy-cluster\"",
            "colour": "#770040",
            "exportable": true,
            "user_id": "0",
            "hide_tag": false,
            "numerical_value": null,
            "is_galaxy": false,
            "is_custom_galaxy": false,
            "local_only": false,
            "local": 0,
            "relationship_type": null
          }
        ]
      }
    }
  ]
}

Pull Attributes

API Endpoint: /attributes/restSearch
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value	Comment
returnFormat	json	The format to return data in. e.g. json, csv, etc.
limit	1	Maximum records to return
page	1	Page number
eventid	< Event ID>	Event ID
timestamp	[ <Start Time>, <End Time>]	Restrict the results by the timestamp (last edit)
category	[<Categories Name>]	Categories of attributes to pull.
type	[“md5″,”sha256″,”ip-src”,”domain”,”ip-src\|port”, “ip-dst”,”ip-dst\|port”,”domain\|ip”,”hostname”,”hostname\|port”]	Type of indicators to pull
tags	[“!netskope-ce”]	Include or exclude attributes with certain tags.
includeDecayScore	1	If set to 1, decay score information will be included for attributes that are affected by decaying.
decayingModel	[1,2]	Allows you to set the decaying model(s) to use to calculate the decay score.
excludeDecayed	1	Filter out all expired IoCs.
modelOverrides.threshold	30	JSON that can be used to modify Model parameters on-the-fly.
published	1	Set whether published or unpublished events should be returned Accepted values 0 or 1.
to_ids	1	By default (0) all attributes are returned that match the other filter parameters, regardless of their to_ids setting.
enforceWarninglist	1	Remove any attributes from the result that would cause a hit on a warninglist entry.

Sample API Response

{
  "response": {
    "Attribute": [
      {
        "id": "347224",
        "event_id": "1315",
        "object_id": "0",
        "object_relation": null,
        "category": "Network activity",
        "type": "ip-src",
        "to_ids": true,
        "uuid": "567ac850-6337-4266-a646-9317661f8974",
        "timestamp": "1719987636",
        "distribution": "5",
        "sharing_group_id": "0",
        "comment": "",
        "deleted": false,
        "disable_correlation": false,
        "first_seen": null,
        "last_seen": null,
        "value": "efd:2dbe:3d3:b9ac:71e1:fd4f:5d2:2de7",
        "decay_score": [
          {
            "score": 30.478404654276954,
            "base_score": 90,
            "decayed": false,
            "DecayingModel": {
              "id": "2",
              "name": "NIDS Simple Decaying Model Updated"
            }
          }
        ],
        "Event": {
          "org_id": "1",
          "distribution": "1",
          "id": "1315",
          "info": "test",
          "orgc_id": "1",
          "uuid": "dceaa84e-156f-4a31-b0b2-a3ae0778d72a"
        }
      }
    ]
  }
}

Delete/retract previously-pushed attributes from MISP (push retraction)

API Endpoint: /attributes/deleteSelected/<Event ID>

Method: POST

Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Payload

Key	Value
id	[“attribute_id”]
event_id	“event_id”

Sample API Response

{
    "saved": true,
    "success": true,
    "name": "1 attribute deleted.",
    "message": "1 attribute deleted.",
    "url": "/attributes/deleteSelected/141",
    "id": "141"
}

Check a Tag’s Existence on MISP

API Endpoint: /tags/search/<Tag name>
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value
limit	5000
offset	“”

Sample API Response

  {
    "Tag": {
      "id": "1399",
      "name": "netskope-ce",
      "colour": "#ff0000",
      "exportable": true,
      "org_id": "0",
      "user_id": "0",
      "hide_tag": false,
      "numerical_value": null,
      "is_galaxy": false,
      "is_custom_galaxy": false,
      "local_only": false
    }
  }
]

Create a Tag on MISP

API Endpoint: /tags/add
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value
name	<Tag Name>
colour	#ff0000

Sample API Response

{
  "Tag": {
    "id": "1400",
    "name": "netskooe-ce",
    "colour": "#ff0000",
    "exportable": true,
    "org_id": "0",
    "user_id": "0",
    "hide_tag": false,
    "numerical_value": null,
    "is_galaxy": false,
    "is_custom_galaxy": false,
    "local_only": false
  }
}

Share Indicators to an Existing Event (Update Event)

API Endpoint: /events/edit/<Event ID>
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value	Comment
type	sha256	Indicator Type
value	56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a	SHA256 value
comment	Test IoC
first_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
last_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
Tag.name	netskope-ce \| <Source Plugin name>

Sample Payload

{
  "Attribute": [
    {
      "type": "sha256",
      "value": "56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a",
      "comment": "Test IoC.",
      "first_seen": "2024-07-05T09:48:51.585000",
      "last_seen": "2024-07-05T09:48:51.585000",
      "Tag": [
        {
          "name": "netskope-ce"
        }
      ]
    }
  ]
}

Sample API Response

{
  "Event": {
    "id": "1317",
    "orgc_id": "1",
    "org_id": "1",
    "date": "2024-06-25",
    "threat_level_id": "4",
    "info": "new",
    "published": false,
    "uuid": "09502561-a07f-41e9-8359-59394878e47d",
    "attribute_count": "12",
    "analysis": "0",
    "timestamp": "1720443083",
    "distribution": "1",
    "proposal_email_lock": false,
    "locked": false,
    "publish_timestamp": "0",
    "sharing_group_id": "0",
    "disable_correlation": false,
    "extends_uuid": "",
    "protected": null,
    "event_creator_email": "admin@admin.test",
    "Org": {
      "id": "1",
      "name": "ORGNAME",
      "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
      "local": true
    },
    "Orgc": {
      "id": "1",
      "name": "ORGNAME",
      "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18",
      "local": true
    },
    "Attribute": [
      {
        "id": "1488555",
        "type": "sha256",
        "category": "Payload delivery",
        "to_ids": true,
        "uuid": "343b7c36-beb7-4197-945e-4b35c8179426",
        "event_id": "1317",
        "distribution": "5",
        "timestamp": "1720443083",
        "comment": "Test IoC.",
        "sharing_group_id": "0",
        "deleted": false,
        "disable_correlation": false,
        "object_id": "0",
        "object_relation": null,
        "first_seen": "2024-07-05T09:48:51.585000+00:00",
        "last_seen": "2024-07-05T09:48:51.585000+00:00",
        "value": "56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a",
        "Galaxy": [
          
        ],
        "ShadowAttribute": [
          
        ],
        "Tag": [
          {
            "id": "1399",
            "name": "netskope-ce",
            "colour": "#ff0000",
            "exportable": true,
            "user_id": "0",
            "hide_tag": false,
            "numerical_value": null,
            "is_galaxy": false,
            "is_custom_galaxy": false,
            "local_only": false,
            "local": 0,
            "relationship_type": null
          }
        ]
      }
    ],
    "ShadowAttribute": [
      
    ],
    "RelatedEvent": [
      {
        "Event": {
          "id": "1325",
          "date": "2024-07-05",
          "threat_level_id": "4",
          "info": "new5",
          "published": false,
          "uuid": "3508388f-b6a6-4be2-9b99-de05825cc304",
          "analysis": "0",
          "timestamp": "1720177270",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1324",
          "date": "2024-07-02",
          "threat_level_id": "4",
          "info": "new4",
          "published": false,
          "uuid": "e8d24ff9-2955-4eb6-80f1-5f276fae2e42",
          "analysis": "0",
          "timestamp": "1720424260",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1319",
          "date": "2024-06-27",
          "threat_level_id": "4",
          "info": "420test",
          "published": false,
          "uuid": "1e1f4455-8d0e-43dd-95ba-0fd4f7eb6abf",
          "analysis": "0",
          "timestamp": "1720418522",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1315",
          "date": "2024-03-12",
          "threat_level_id": "4",
          "info": "test",
          "published": false,
          "uuid": "dceaa84e-156f-4a31-b0b2-a3ae0778d72a",
          "analysis": "0",
          "timestamp": "1720175425",
          "distribution": "1",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      },
      {
        "Event": {
          "id": "1308",
          "date": "2015-01-01",
          "threat_level_id": "1",
          "info": "testevent",
          "published": false,
          "uuid": "0c3d5017-8edf-483b-a829-426bb2c56c92",
          "analysis": "0",
          "timestamp": "1699523309",
          "distribution": "0",
          "org_id": "1",
          "orgc_id": "1",
          "Org": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          },
          "Orgc": {
            "id": "1",
            "name": "ORGNAME",
            "uuid": "45d545a3-859f-4645-9139-0d8fb8730f18"
          }
        }
      }
    ],
    "Galaxy": [
      
    ],
    "Object": [
      
    ],
    "EventReport": [
      
    ],
    "CryptographicKey": [
      
    ],
    "Tag": [
      {
        "id": "251",
        "name": "workflow:todo=\"create-missing-misp-galaxy-cluster\"",
        "colour": "#770040",
        "exportable": true,
        "user_id": "0",
        "hide_tag": false,
        "numerical_value": null,
        "is_galaxy": false,
        "is_custom_galaxy": false,
        "local_only": false,
        "local": 0,
        "relationship_type": null
      }
    ]
  }
}

Share Indicators to a New Event (Create Event)

API Endpoint: /events/add
Method: POST
Headers

Key	Value
User-Agent	netskope-ce-7.0.0-cte-misp-v1.5.2
Authorization	API Key
Accept	application/json
Content-Type	application/json

Parameters

Key	Value	Comment
type	sha256	Indicator Type
value	56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a	SHA256 value
comment	Test IoC.
first_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
last_seen	2024-07-05T09:48:51.585000	First Seen of the IoC
Tag.name	netskope-ce

Sample Payload

{
"info": "test"
  "Attribute": [
    {
      "type": "sha256",
      "value": "56af10adb647b1e675f3f486c7941fbf637ebd0c6632e86e6dc9879d2214441a",
      "comment": "Test IoC.",
      "first_seen": "2024-07-05T09:48:51.585000",
      "last_seen": "2024-07-05T09:48:51.585000",
      "Tag": [
        {
          "name": "netskope-ce"
        }
      ]
    }
  ]
}

Sample API Response

{
  "Tag": {
    "id": "1400",
    "name": "netskope-ce",
    "colour": "#ff0000",
    "exportable": true,
    "org_id": "0",
    "user_id": "0",
    "hide_tag": false,
    "numerical_value": null,
    "is_galaxy": false,
    "is_custom_galaxy": false,
    "local_only": false
  }
}

Performance Matrix

Here is the performance reading conducted by pulling and sharing 100K indicators from/to MISP on a Large CE Stack with these specifications.

Description	Specification
Stack details	Size: Large RAM: 32 GB CPU: 16 Cores
Indicators fetched from MISP Platform	~18K per minute
Indicators shared with MISP Platform	~8K per minute

User Agent

netskope-ce-7.0.0-cte-misp-v1.5.2

Workflow

Get your MISP URL and API key.
Configure the MISP Plugin.
Configure a Business Rule.
Configure Sharing between Netskope and MISP.
Validate the MISP Plugin.

Watch a Video

Click play to watch a video:

Get your Instance URL and Auth Key

Click Add authentication key, enter the required parameters, and click Submit.

Your MISP API key will be generated. Make a copy of this value for when you configure the Plugin.

Configure the MISP Plugin

In Cloud Exchange, go to Settings > Plugin Store. Search for and select the MISP v1.5.2 (CTE) plugin.
Enter and select the Basic Information:
- Configuration Name: Plugin configuration name
- Sync Interval: Interval to fetch data from this plugin source.
- Aging Criteria: Expire indicators after a specific time.
- Override Reputation: Set value to override reputation of indicators received from this configuration. Leave empty to keep default.
- Enable SSL Validation: Enable/Disable SSL Certificate validation based on your platform requirement.
- Use System Proxy: Use system proxy configured in Settings.
Click Next and enter the Configuration Parameters:
- MISP Base URL: Base URL of MISP instance. (https://<misp-url>)
- Authentication Key: API Key generated from the MISP platform.
- MISP Attribute Type: Select MISP attribute type(MD5, SHA256, URL (Source IP|Port (ip-src|port), Destination IP|Port (ip-dst|port), and Hostname|Port (hostname|port)), Domain, IPv4 and IPv6( Source IP(ip-src) and Destination IP (ip-dst)), Domain( Domain|IP), Hostname), Indicators from only specified attribute types will be fetched. Keep empty to fetch indicators of all types. Multiple Types are accepted.
- MISP attribute Category: Select MISP attribute Category, Indicators from only specified Attribute Categories will be fetched. Keep empty to fetch indicators of all Categories. Multiple Categories are accepted.
- MISP Attribute Tags: Enter MISP Attribute Tags, Indicators from only specified comma-separated Tags will be fetched. Keep empty to fetch indicators of all Tags. Dynamic values are accepted.
- Event Names: Leave Event Names blank to fetch indicators from all the events, or enter them separately by comma to fetch only those indicators which belong to that event. For now, we leave it blank.
- Exclude IoCs from Event: In Exclude IoCs from Event , enter the name of the event whose IoCs you want to exclude from being fetched. Indicators attached to the provided comma-separated events will be ignored while pulling data from MISP. Expected value is comma-separated event names or event IDs.
- Ioc Event Type: Indicators will be pulled based on the selected event type. Published, Unpublished or both types of events can be selected. Keep empty to fetch all types
- Decaying Score Threshold: Only indicators having Decaying Score greater than the provided value will be pulled. Value should be in the range of 0 to 100.
- Decaying Model IDs: Decaying score of only specified comma-separated decaying models will be tracked. Keep blank to fetch scores for all enabled decaying models that apply to the attribute type. Decaying model IDs can be found from Global Action > List Decaying Models.
- Filter on IDS Flag: Pull IoCs based on the Selected option for IDS. Enabled IDS flag, Disabled IDS flag or both types of indicators can be selected. Keep empty to fetch all indicators.
- Enforce Warning List IoCs: Select Yes to remove any IoC from the events that would cause a hit on a warning list entry. Warning List can be found from Input Filters > Warninglists.
- Pulling Mechanism: Select a Pulling Mechanism
  - Incremental (Default): Plugin will fetch data using stored checkpoint, like last_run_at of the plugin.
  - Look Back: Plugin will fetch data subtracting lookback value from the plugin run time on every sync. For ex: If the provided value is 72 Hrs then it will fetch 72 Hrs of IoCs from now on every sync.
- Look Back (in hours): Enter Look Back (in hours) to pull the indicators of historical time from now on every time the plugin syncs. Default value is set to empty and it will only be used when you select Look Back in Pulling Mechanism.
- Retraction Interval (in days): Retraction Interval days to run IoC(s) retraction for MISP indicators. Note: This parameter will only be considered if IoC(s) Retraction is enabled in Threat Exchange Settings. The minimum value expected for the retraction will be 1. If the retraction value is not added, and IoC(s) Retraction is enabled in the global setting, retraction will not take place for the plugin.
- Enable Tagging: Plugin will pull tags associated with the IoCs if Enable tagging is Yes. If you do not want to pull tags with the IoCs, keep Enable tagging as No.
- Initial Range (in days): Number of days to pull the data for the initial run. Note that this parameter will only be considered if Pulling Mechanism is set to Incremental.
Click Save.

Note

IoCs stored in Netskope CE will have the current date and time as Last_Seen rather than the MISP’s last seen.

Configure a Threat Exchange Business Rule for MISP

A Business Rule is used to filter out the indicators that are to be shared. In order to share IoCs with MISP, create a business rule using the following steps:

In Threat Exchange, go to Business Rules and click Create New Rule.
Enter a Rule name and select the fields for which you want to filter the IoCs as described in the next two steps.
Click Create New Rule. Enter a Rule name and select the fields for which you want to filter the IoCs. For example, IoCs by Sources is equal to CTE Netskope. Click Save.
Click Create New Rule. Enter a Rule name and select the fields for which you want to filter the IoCs. IoCs by Sources is equal to CTE MISP. Click Save.

Configure Threat Exchange Sharing for MISP

In Threat Exchange, go to Sharing and click Add Sharing Configuration.
Select the Source configuration (Source from which you want to share data to MISP), select a Business Rule, and select the Destination (MISP Plugin).
Select the Target value Add to Event.
Enter the Event Name that you want to use for pushing IoCs on the MISP Platform.
Select the type for sharing the IPv4/IPv6/CIDR IoCs to MISP.
Click Save.

Note

For URL, Domain, IPv4, IPv6 and Hostname

Add to URL List
Add to Private App
Add to Destination Profile
Add to DNS Profile

For SHA256 and MD5

Add to File Hash List

Validate the MISP Plugin

Validate the Pull

Indicators from MISP are pulled from: Event Actions > List Attributes.

Indicators stored in Cloud Exchange can be verified from the Threat Exchange > Threat IoCs page.

Search for the MISP IoCs by filtering indicators on the Threat IoCs page with source name as Configured MISP Plugin.

Example: Add a query on the Threat IoCs page like “sources.source Is equal “CTE MISP” && type IN (“<IOC_TYPE>”)”

You can also verify the indicators pulled in Cloud Exchange from the logs available at Logging.

For verifying the Retracted IoCs from MISP, check the logs for IoC Retraction. For example: message Like “CTE MISP [CTE MISP] [Retraction]:”

Note

The IoCs that are deleted on the MISP and fall under the Retraction Interval will be marked as Retracted in Cloud Exchange.

Validate the Push

Shared IoCs to MISP can be verified from logs available at Logging in Cloud Exchange.

IoCs shared on MISP can be verified from Event Actions > List Events.

Click on the Event ID that you used while configuring the sharing to view all the shared IoCs.

The IoC Labeling is added to each of the IoCs that are shared/pushed to MISP from Cloud Exchange, the format for the same will be “Netskope CE | <plugin-name>”

The IoC labeling for the Shared IoCs will be added as a part of tags on MISP; it can also be verified by the log available in Cloud Exchange Logging while IoCs are shared to MISP.

Troubleshooting the MISP Plugin

Error while upgrading the plugin repository

While trying to upgrade a plugin if the below error is received, follow the steps provided.

What to do:

Click Skip and go to Home > Threat Exchange > Plugins.
Edit the plugin as it will be disabled due to an error while upgrading the plugin.
Click Next.

Scroll down to newly added params (Pulling Mechanism and Look Back) and select the method of pulling and select Look Back as the Pulling Mechanism.

Click Save and then enable the plugin.

Unable to pull/push data to MISP Plugin

If you are not able to share IoCs from Netskope to MISP, that could be the Authentication Key of MISP could have either expired, or has been deleted from the MISP Platform.

What to do:

Pulled domains and IPs are stored as URL in Cloud Exchange

If you have data pulled from MISP and the IoCs of type domain, IPv4 and IPv6 are stored as URL in Cloud Exchange, it could be due to one of the following reasons:

Plugin is not updated to v1.4.0
Plugin version is v1.4.0 but core is not updated to v5.0.1
IoC type on MISP is URL

What to do:

The URL bifurcation for the MISP plugin is available from MISP plugin v1.4.0, if your plugin is any version below this the bifurcation update won’t be available.
The URL bifurcation is supported in Cloud Exchange from core (CE version) v5.0.1, if your Cloud Exchange is not updated to the supported version the bifurcation will not be available and all domains and IPs pulled from the platform will be stored as URL. If you want to pull data as per the URL bifurcation your plugin version must be v1.4.0 or above and core version as v5.0.1 or above.
If both your core and plugin are updated and still few of the IoCs of type domain, IPv4 and IPv6 are stored as URLs in Cloud Exchange, it must be due to the IoC type of the indicator on MISP. If the IoC on MISP has type as URL for the domains/IPs, it will be pulled and stored as URL in Cloud Exchange. If MISP has domain as IOC type, it will be pulled and stored as domain in Cloud Exchange. The IoC type for IPs on MISP is ip-src.

Limitation

Decaying Score is not in sync with MISP for Incremental Pulling Mechanism

If the user is using the Incremental pulling mechanism, after an IoC is pulled from MISP along with its decaying score, the IOoC will only be pulled again into Cloud Exchange if the timestamp for that indicator gets updated on MISP. Otherwise, the decaying score will remain as it was when initially pulled.
This is a limitation of the MISP platform: when updating the decaying score, the timestamp associated with the attribute is not updated. In Cloud Exchange, we use the timestamp fields from MISP attributes to capture only the updated indicators. Therefore, if the timestamp value is not updated, the IoC will never be pulled again into Cloud Exchange.
To address this, the user can use the Look Back pulling mechanism to pull all the indicators based on the Look Back period specified in the configuration parameters for each pull.

Known Behavior

Difference in response displayed on the UI in case Decaying Model IDs field is left empty and when all Model IDs are added

If no Model IDs are provided in the Decaying Model IDs field, the decaying scores associated with the attribute will be fetched in Cloud Exchange. For example, if there are three models in the MISP instance and only one model is attached to the attribute, only the decaying score for that specific model will be fetched in Cloud Exchange.
However, if specific Model IDs are provided, whether they are associated with the attribute or not, the scores for all available models will be returned via the API. In the MISP UI, only the associated models will be shown.