Release Notes
1.1.0 (Requires minimum Cloud Exchange version 6.0.0)
Added
- Added support for the Content and Device alert types, Client Status event type and WebTx v2 type in the default mappings.
- Added the JSON format option.
- Added resolutions for error logs.
Changed
- Updated the object file name format to <epoch>_<thread_id>_<uuid>.<extension> to prevent data overwrite when data is ingested at the exact same time.
Fixed
- Fixed the error observed while sharing WebTx data using NLS.
- Fixed temporary files not being removed when an upload fails.
1.0.0
Added
- Initial release.
This document explains how to configure the Scality v1.1.0 plugin with the Log Shipper module of the Netskope Cloud Exchange platform. This plugin supports ingestion of:
- Alerts (DLP, Malware, Policy, Compromised Credential, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, UBA, CTEP, Device, and Content).
- Events (Page, Application, Audit, Infrastructure, Network, Incident, Endpoint, and Client Status).
- WebTx data.
The data will be ingested in the Scality bucket. This plugin sends logs in JSON format only.
Prerequisites
To complete this configuration, you need:
- A Netskope tenant (or multiple, for example, production and development/test instances).
- A Netskope Cloud Exchange tenant with the Tenant plugin and Log Shipper plugin already configured.
- A Netskope Cloud Exchange tenant with the AWS Netskope Log Streaming or Azure Netskope Log Streaming plugin already configured (for ingesting alerts, events, and WebTx from the Netskope Log Streaming plugins).
- Your Scality instance URL, Access Key, Secret Access Key, and Scality Bucket Name.
- A Scality instance and Bucket access.
- These Scality bucket permissions for the user:
- ListBucket
- CreateBucket
- ListAllMyBuckets
- GetBucketPolicy
- GetBucketPublicAccessBlock
- PutEncryptionConfiguration
- PutBucketPublicAccessBlock
- PutBucketPolicy
- To use a Secrets Manager, refer to this documentation. To set up Azure Key Vault, refer to this documetation.
Scality Plugin Support
This plugin supports ingestion of Alerts, Events, and WebTx data to a Scality bucket. This plugin supports ingestion in JSON format only.
| Data Type | Support |
|---|---|
| Events | Yes (Page, Application, Audit, Infrastructure, Network, Incident, Endpoint, Client Status) |
| Alerts | Yes (DLP, Malware, Policy, Compromised Credential, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, UBA, CTEP, Device, Content) |
| WebTx | Yes (via Netskope Log Streaming) |
Note
- CLS WebTX based on Google Pub Sub Lite is deprecated. Please refer to Netskope Product EOL/EOS Announcements – Netskope Knowledge Portal
- For ingesting WebTX logs to your Log delivery destinations like SIEM, SOAR, XDR, Data Lake, use the AWS Netskope Log Streaming or Azure Netskope Log Streaming plugin.
- The ingestion of Endpoint event type is supported from Cloud Exchange version 5.1.0.
Permissions
Scality bucket permissions to the user to send Events, Alerts, and Web transaction logs to buckets are required.
API Details
List of APIs Used
This plugin uses Python libraries to create file objects in Scality
Library: The AWS SDK for Python (Boto3)
Usage: The AWS SDK for Python (Boto3) to create, configure, and manage AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3). The SDK provides an object-oriented API as well as low-level access to AWS services.
Create the Client
s3_client = boto3.client(
"s3",
aws_access_key_id=access_key,
aws_secret_access_key=secret_access_key,
endpoint_url=endpoint_url,
region_name=region_name, # only for an AWS style endpoint
config=Config(
proxies=proxy,
user_agent=user_agent,
read_timeout=300,
retries={"max_attempts": 3, "mode": "standard"},
),
)
Create a Bucket
s3_client.create_bucket(Bucket=bucket_name) # us-east-1
s3_client.create_bucket(
Bucket=bucket_name,
CreateBucketConfiguration={"LocationConstraint": region_name},
)
Upload a File into the Bucket
s3_client.upload_file(
file_name,
bucket_name,
object_name,
)
Get an AWS Resource
s3_resource = boto3.resource(
"s3",
aws_access_key_id=access_key,
aws_secret_access_key=secret_access_key,
endpoint_url=endpoint_url,
region_name=region_name, # only for an AWS style endpoint
config=Config(
proxies=proxy,
user_agent=user_agent,
retries={"max_attempts": 0, "mode": "standard"},
),
)
Performance Matrix
This performance reading is for a Large Stack CE tested with these VM specifications.
| Description | Specification |
|---|---|
| Stack details | Size: Large RAM: 32 GB CPU: 16 Cores |
| Alerts/Events ingested to SIEM | ~185K EPM |
| WebTx (via Netskope Log Streaming) ingested to SIEM | ~155K EPM |
User Agent
netskope-ce-7.0.0-cls-scality-v1.1.0
Workflow
- Create a Scality Bucket.
- Configure the Scality Plugin.
- Configure a Business Rule for Scality.
- Configure Log Delivery for Scality.
- Validate the plugin.
Watch a Video
Click play to watch a video.
Create a Bucket in Scality
- Go to your Scality instance.

- Enter your credentials and login.

- Click Create Bucket and enter a bucket name in which you want to ingest your data. Click Create.

Configure the Scality Plugin
-
In Cloud Exchange, go to Settings > Plugin Store. Search for and select the Scality v1.1.0 (CLS) plugin.

-
Enter a Configuration Name and set the Mapping per your requirements.

-
Click Next and enter the Configuration Parameters:
- Scality Endpoint URL: Endpoint URL of the Scality instance. Example: https://s3-your.domain.com/
- Access Key: Access Key for the Scality instance.
- Secret Access Key: Secret Access Key for the Scality instance.
- Scality Bucket Name: Name of the Scality bucket in which the log objects will be stored. The bucket must already exist on Scality.

-
Click Save. Your new plugin will appear on the Log Shipper > Plugins page.

Configure a Log Shipper Business Rule for Scality
- In Log Shipper, go to Business Rules.
- By default, there is a business rule that filters all alerts and events. If you want to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding a rule name and desired filters.

- Click Save.

Configure Log Shipper Log Delivery for Scality
- Go to Log Shipper > Log Delivery and click Add Log Delivery Configuration.
- For alerts and events, select the Source plugin (Netskope CLS or any other source plugin), Destination plugin (CLS Scality), your business rule, and then click Save.
- For WebTx, select the Source plugin (AWS Netskope Log Streaming or Azure Netskope Log Streaming), and Destination plugin (CLS Scality), your business rule, and then click Save.

- After the Log Delivery configuration is added, the data will start to be pulled from the Netskope tenant or source platform, transformed, and ingested into Scality.
Validate the Scality Plugin
Validate the Pull
To validate the pulling of alerts/events/WebTx from the Netskope tenant.
Go to the Logging in Netskope CE. Search for the pulled logs.





Validate the Push
To validate the plugin workflow in Cloud Exchange:
Go to Logging and search for ingested Events, Alerts, and WebTx with the filter message contains ingested.
The ingested logs will be filtered.





To validate the push on the Scality:
- Log in to Scality.

- Click on your bucket.

- Click the Alerts folder.




- Click the Events folder.




- Click the WebTx folder.



Here is the sample ingested data with custom mappings:

Note
- For alerts and events, the folder structure is like alerts/feedname/year/month/day/hour/filename.txt
- For WebTx, the folder structure is like webtx/feedname/year/month/day/hour/filename.gz
- File names use the format <epoch>_<thread_id>_<uuid>.<extension>, where the extension is .txt for alerts and events and .gz for WebTx.
Example: demobucket/alerts/feedname=Malware/year=2023/month=11/day=30/hour=9/1792588265_132874188526208_37862b2b68994795874100a37cc8e2cf.txt
Troubleshooting the Scality Plugin
Facing issues while configuring the plugin
This error can occur during configuration if the Scality credentials or the bucket name is invalid.


What to do:
- Make sure your Scality Endpoint URL, Access Key, and Secret Access Key are valid.
- Make sure that the bucket entered in the plugin exists on the Scality platform. For details, refer to Create a Bucket.

