Datadog Plugin for Log Shipper

This document explains how to configure the Datadog integration with the Log Shipper module of the Netskope Cloud Exchange platform. This plugin supports ingestion of alerts, events, and web transaction logs to Datadog in CEF and JSON format.

Prerequisites

A Netskope tenant (or multiple, for example, production and development/test instances) that is already configured in Cloud Exchange.
A Netskope Cloud Exchange tenant with the Log Shipper module already configured.
A Netskope Cloud Exchange tenant with the WebTx plugin already configured.
A Netskope Cloud Exchange tenant with the Syslog for CE plugin already configured.
Datadog Platform access with the Datadog Agent installed.
Connectivity to the following hosts (any of these)
- https://app.datadoghq.com
- https://us3.datadoghq.com
- https://us5.datadoghq.com
- https://app.datadoghq.eu
- https://app.ddog-gov.com
- https://ap1.datadoghq.com

CE Version Compatibility

This plugin is compatible with Netskope CE v4.2.0 and v5.0.0.

Datadog Plugin Support

The Datadog plugin is used to ingest all the Alert, Events, Syslog CE Logs, and WebTx logs in CEF and JSON format.

Alerts Support	Yes
Event Support	Yes
WebTx Support	Yes
Syslog CE Logs	Yes

Permissions

Permission to generate an API Key in Datadog.

API Details

List of APIs used

API Endpoint	Method	Use Case
https://http-intake.logs.datadoghq.com/api/v2/logs	POST	Send Logs to Datadog Platform

Send Logs

API Endpoint:

https://http-intake.logs.datadoghq.com/api/v2/logs

Method: POST
Parameters:
dd source: dd source
ddtags: ddtags
hostname: hostname
Request Body:

[{'message': '{"cci": 43, "timestamp": 1708515322000, "ccl": "poor"}'}]

Headers:
Content-Type: application/json
DD-API-KEY: {API_KEY}
API Request Endpoint:

https://http-intake.logs.datadoghq.com/api/v2/logs

Sample API Response:

202 Accepted

Performance Matrix

This performance reading is conducted on a Large Stack CE with these VM specifications. These readings are noted with the consideration that it will ingest around 10K events in 10 seconds to the Datadog platform.

Stack details

Size: Large

RAM: 32 GB

CPU: 16 Cores

Alerts/Events ingested to SIEM

~200k EPM

WebTx ingested to SIEM

~6 MBps

User Agent

The user-agent added in this plugin is in the following format

netskope-ce-<ce_version>-<module>-<plugin_name>-v<plugin_version>

For example:

netskope-ce-5.0.0-cls-datadog-v1.0.0

Workflow

Get your Datadog API Key.
Configure the Datadog Plugin.
Configure a Business Rule.
Configure SIEM Mappings.
Validate the Datadog plugin.

Click play to watch a video.

Get your Datadog API Key

Log in to your machine where the Datadog Agent is installed.
Go to your Datadog agent directory as per your OS (https://docs.datadoghq.com/agent/?tab=Linux)
Open datadog.yaml file.
Change to logs_enabled: true.
Based on your OS flavor, you need to restart your Datadog Agent.(https://docs.datadoghq.com/agent/?tab=Linux)
Log in to Datadog Platform.
Hover on your username from the bottom Left corner.
Click Organization Settings.
Under Access click API Keys.
Click on the Key and click Copy to copy the key.

Configure the Datadog Plugin

Go to Settings > Plugins. Search for and select the CLS Datadog plugin box to configure the plugin.
Add a plugin configuration name and make sure the Datadog Default Mapping file is selected for Mapping. Click Next.
Disable the toggle button to transform the logs if you want to ingest the data in JSON format; keep it enabled if you want to ingest the data in CEF format.
Click Next and enter these parameters:
- Datadog Site: The site associated with your Datadog account. For example: datadoghq.com.
- API Key: An API Key is required by the Datadog Agent to submit metrics and events to Datadog. Provide the Datadog API Key obtained previously (from Organization Settings > Access > API Keys on the Datadog platform.
- Datadog Tags: Tags associated with your logs. For example: env:prod,region:us-east-1.
Click Save. The new plugin will be seen on the Log Shipper > Plugins page.

Configure a Log Shipper Business Rule for Datadog

Go to Log Shipper > Business Rules.
By default, there is a business rule that filters all alerts and events. If you want to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding the rule name and filter.

Configure Log Shipper SIEM Mappings for Datadog

Go to Log Shipper > SIEM Mappings and click Add SIEM Mapping.
Select the Source plugin (Netskope CLS), Destination plugin (Datadog), and business rule, and then click Save.
For WebTX, select Source plugin (Netskope WebTx) and Destination plugin (Datadog).
After the SIEM mapping is added, the data will start to be pulled from the Netskope tenant, transformed, and ingested into the Datadog platform.