Syslog Plugin with Splunk for Log Shipper

This document explains how to configure the Syslog plugin with the Cloud Log Shipper module of the Netskope Cloud Exchange platform. This plugin supports ingestion of Alerts (Anomaly, DLP, Malware, Policy, Compromised Credential, Legal Hold, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, UBA, CTEP), Events (Page, Application, Audit, Infrastructure, Network, Incident, Endpoint), WebTx and Logs (Debug, Information, Error, Warning). The data will be ingested in the SIEM platform. This plugin supports ingestion in CEF and JSON format.

Prerequisites

To complete this configuration, you need:

A Netskope tenant (or multiple, for example, production and development/test instances) that is already configured in Cloud Exchange.
A Netskope Cloud Exchange tenant with the Log Shipper module already configured.
A Netskope Cloud Exchange tenant with the WebTx plugin already configured.
A Netskope Cloud Exchange tenant with the Syslog for CE plugin already configured.
A Splunk instance.
Connectivity to a syslog server.

CE Version Compatibility

Netskope CE v5.0.1 and v5.1.0

Syslog Plugin Support

Syslog plugin is used to ingest all the Alert, Events, WebTx, and Syslog CE Logs in CEF and JSON format.

Event Support	Yes (Audit, Application, Infrastructure, Network, Incident, Page, Endpoint) The Endpoint event type requires the minimum CE version to be 5.1.0
Alert Support	Yes (DLP, Malware, Policy, Compromised Credential, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, CTEP, UBA)
WebTx Support	Yes
CE Logs Support	Yes (Info, Error, Warning, Debug)

API Details

The plugin uses a logging third-party library to push the data to the Syslog collector.

Library: logging

This module defines functions and classes which implement a flexible event-logging system for applications and libraries.

The key benefit of having the logging API provided by a standard library module is that all Python modules can participate in logging, so your application log can include your own messages integrated with messages from third-party modules.

Refer to the official documentation for more information on the logging library: https://docs.python.org/3/library/logging.html.

List of Methods Used

Method: logging.getLogger(name=None)

Return a logger with the specified name or, if the name is None, return a logger which is the root logger of the hierarchy

All calls to this function with a given name return the same logger instance. This means that logger instances never need to be passed between different parts of an application.

Method: setLevel(level)

Sets the threshold for this logger to level. Logging messages that are less severe than the level will be ignored; logging messages that have a severity level or higher will be emitted by whichever handler or handlers service this logger, unless a handler’s level has been set to a higher severity level than the level.

Method: handlers

The list of handlers is directly attached to this logger instance.

Note: This attribute should be treated as read-only; it is normally changed via the addHandler() and removeHandler() methods, which use locks to ensure thread-safe operation.

Method: addHandler(hdlr): Adds the specified handler hdlr to this logger.
Method: removeHandler(hdlr): Removes the specified handler hdlr from this logger.

Performance Matrix

This performance reading is conducted on a Large Stack CE with these VM specifications. These readings are added with the consideration that it will ingest around 10K events in 2 seconds to the Syslog platform.

Stack details

Size: Large

RAM: 32 GB

CPU: 16 Cores

Alerts/Events ingested to SIEM

~200K EPM

WebTx ingested to SIEM

~6 MBps

Workflow

Add Data Input on Splunk.
Configure the Syslog Plugin for the Splunk integration.
Configure a Log Shipper Business Rule for the Splunk integration.
Configure Log Shipper SIEM Mappings for the Splunk integration.
Validate the Syslog with Splunk plugin.

Click play to watch a video.

Create a Data Input on Splunk

If you do not have a Splunk instance, follow these steps to install Splunk.

Log in to your Splunk instance.
From the dashboard, go to Settings > Data inputs.
Click Add new for the TCP input.
Add your port and click Next.
Select the source type if you already have any, or click New to create a new source type.
Enter the source type. Select the Source Type Category based on your requirements, or keep it as is.
Scroll down to Index. If you already have any index that you want to use, select it from the Index dropdown. Otherwise, click Create a new index, add an Index Name, click Save, and then click Review.
Review the details and click Submit.
Click Start searching.

Configure the Syslog Plugin for the Splunk Integration

Go to Settings > Plugins. Search for and select the Syslog v3.2.2 (CLS) plugin box.
Add a Plugin configuration name, select the Syslog Default Mapping file, and then click Next.
Disable the first toggle button if you want to ingest your alerts and events in the JSON format. Keep it as enabled if you want to ingest your data into CEF format.
Click Next.
Enter these parameters:
- Syslog server: IP address/FQDN of Syslog server in which data will be ingested.
- Syslog Format: Data format required to ingest data.
- Syslog Protocol: Protocol to be used while ingesting data.
- Syslog Port: The port used while creating the Data input configuration on Splunk.
Click Save. Your plugin will be available on the Log Shipper > Plugins page.

Configure a Log Shipper Business Rule for the Splunk Integration

Go to Log Shipper > Business Rule.
The default business rule filters all alerts and events. If you need to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding a rule name and specific filters.
If you need to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding a rule name and specific filters.
Click Save.

Configure Log Shipper SIEM Mappings for the Splunk Integration

Go to Log Shipper > SIEM Mappings and click Add SIEM Mapping.
Select the Source plugin (Netskope CLS plugin), Destination plugin (Syslog plugin), and a business rule, and then click Save.
For WebTx, click Add SIEM Mapping, select the Source plugin (Netskope WebTx plugin), Destination plugin (Syslog plugin), and a business rule, and then click Save.
For Logs sharing, click Add SIEM Mapping, select the Source plugin (CLS Syslog for CE), the Destination plugin (CLS Syslog), and a business rule, and click Save.

After the SIEM mapping is added, the data will start to be pulled from the Netskope tenant, transformed, and ingested into the Syslog platform.

Validate the Splunk Integration

Validate the Pull

To validate the pulling of Events, Alerts, logs, and WebTx from the Netskope tenant:

In Cloud Exchange, go to the Logging and search for the pulled logs.

Validate the Push

To validate the plugin in Cloud Exchange:

Go to Logging and search for ingested Events, Alerts, WebTx & Logs with the filter “message contains ingested”. The ingested logs will be filtered.

To validate the push in Splunk:

Log in to the Splunk Platform.
Click Search & Reporting.
Enter the source and Protocol along with: and port along with Log Source Identifier (Example: source=”tcp:1111″ netskope.
This is how logs look from the plugin to Splunk.
This is how WebTx data looks from the plugin to Splunk.
This is how data looks when shared in JSON format from the plugin to Splunk (unparsed format).

Troubleshooting the Syslog Plugin

An error occurred while configuring the Syslog Plugin

Despite entering all parameters and clicking the Save, an error may occur, possibly due to the following reasons:

The server/port configuration may differ from the specified settings (Netskope CE/Splunk).
The port is not exposed on the Splunk server.

Follow these steps to fix these issues.

In the Splunk Platform, go to Settings and click Data inputs >TCP (Whichever configuration you have used). Check that both are the same.
Expose the Port on the Splunk server.

An error occurred while ingesting data from CE to Syslog

If you are unable to push alerts/events/logs/webtx data on the Syslog platform, then it could be due to one of these reasons:

Port is deleted/disabled on the Syslog platform.
Splunk server storage is full.

To resolve these issues:

Make sure the port is present and enabled. If not, then create a new port.
Make sure to clean the event data if not necessary, or increase the storage of the Splunk server.

If ingested data is not reflected on the Syslog Platform

If you are unable to view alerts/events/logs/webtx data on the Syslog platform, it could be due to one of these reasons:

The filter is not correct on the Splunk platform.
There might be any error, but UDP is selected in the Port while configuring the syslog plugin.

To resolve these issues:

Make sure Data is searched using the correct filter.
Make sure to select the TCP port to check if there is any issue.

Network Event skipped due to unexpected type for Network Session ID field

If you are not able to get value for the Network session ID field, then it could be due to using an old syslog plugin where the network session ID field is of number type.
To resolve this issue, follow these steps:

Update to latest syslog plugin or update network session id field to string type to handle non-numeric data.
To update mapping, go to Settings > Log Shipper and clone the Syslog Default Mappings. Add a name for the cloned mapping.
Click Events > Network > Extension > networkSessionId > Select Type “String”, and then click Save.
Use the updated mapping file in plugin configuration.

Syslog Plugin with Splunk for Log Shipper