AWS Security Lake Plugin for Log Shipper

This document explains how to configure the AWS Security Lake integration with the Cloud Log Shipper module of the Netskope Cloud Exchange platform. The AWS Security Lake plugin supports:

Prerequisites

To complete this configuration, you need:

A Netskope Tenant (or multiple, for example, production and development/test instances)
A Netskope Cloud Exchange tenant with the Log Shipper module already configured.
A Security Lake enabled AWS account. You will need to get the Access Key ID and Secret Access Key for your account prior to starting this procedure.
An AWS event bridge with the configured Lambda script.

Security Lake Plugin Support

The AWS Security Lake plugin supports:

Events
Alerts
WebTx

All Netskope events, alert logs, and web transaction logs will be shared.

Workflow

Configure the AWS Security Lake plugin.
Configure the Log Shipper Business Rules for AWS Security Lake.
Configure the Log Shipper SIEM Mappings for AWS Security Lake.
Validate the AWS Security Lake plugin.

To watch a demo, click play.

Configure the AWS Security Lake Plugin

In Cloud Exchange, go to Settings > Plugins.
Search for and select the Amazon Security Lake box to open the plugin creation dialog.
For Basic Information, first enter a Configuration Name.
Select the valid Mapping. (Default Mappings for all plugins are available.)
Enable the Transform the raw logs toggle (if not already), as we cannot send data directly to S3 bucket bucket because the lambda expects those files to be in a specific form in order to convert them into parquet.
Click Next.
For Configuration Parameters, enter these values:
- AWS Access Key ID (Public Key): AWS Security Lake Access Key ID.
- AWS Secret Access Key (Private Key): AWS Security Lake Secret Access Key.
- Region Name: Region Name of the bucket.
- Bucket Name: Bucket name to store a data object.
Click Save.

Configure the Log Shipper Business Rules for AWS Security Lake

Click Log Shipper > Business Rules.
Click Create New Rule.
Enter a Rule Name and clickConfigure Filter. Enter Folder Name if any.
Click Save.

Configure the Log Shipper SIEM Mappings for AWS Security Lake

Click Log Shipper > SIEM Mappings > Add SIEM Mapping.
Select a Source Configuration , a Destination Configuration, and a Business Rule.
Click Save.

Validate the AWS Security Lake Plugin

To validate the plugin, you can check from Netskope Cloud Exchange and from AWS.

To validate from Netskope Cloud Exchange, go to Log Shipper > Logging.
To validate from the AWS, go to S3 Bucket and select the configured bucket name for the plugin.

Troubleshooting the AWS Security Lake Plugin

Check the logs from Logging (bottom left corner)
Check in the Log Shipper -> SEIM Mappings -> Total Logs Sent section if any log is sent or not.
To confirm if the logs are pulled or not, check logs from Logging (bottom left corner)
If you see that the data is sent from Netskope and still the parquet files are not visible in the destination bucket, in that case please confirm that the Bucket name in Netskope configuration and SRC_BUCKET in Lambda is the same. Also confirm that the destination bucket you are monitoring for the parquet files is the same bucket specified in DST_BUCKET in Lambda env.

Mapping Field Descriptions

The OCSF versions will be maintained with the update of the plugin.

Data Type	Sub Type	Netskope Field	OCSF Field	Default value
Webtx	v2	x-cs-app	app_name	None
Webtx	v2	sc-bytes	bytes_in	None
Webtx	v2	x-c-country	country	None
Webtx	v2	x-c-device	device.name	None
Webtx	v2	None	device.type_id	-1
Webtx	v2	time-taken	duration	None
Webtx	v2	cs-username	email_addr	None
Webtx	v2	cs-method	http_method	None
Webtx	v2	sc-status	http_status	None
Webtx	v2	x-c-os	os.name	None
Webtx	v2	None	os.type_id	-1
Webtx	v2	x-c-zipcode	postal_code	None
Webtx	v2	cs-referer	referrer	None
Webtx	v2	x-cs-session-id	session_uid	None
Webtx	v2	cs-user-agent	user_agent	None
Webtx	v2	bytes	bytes	None
Webtx	v2	c-ip	src_endpoint.ip	None
Webtx	v2	cs-bytes	bytes_out	None
Webtx	v2	cs-dns	domain	None
Webtx	v2	cs-host	hostname	None
Webtx	v2	cs-uri-port	port	None
Webtx	v2	cs-uri-query	query_string	None
Webtx	v2	cs-uri-scheme	scheme	None
Webtx	v2	s-ip	dst_endpoint.ip	None
Webtx	v2	sc-content-type	content_type	None
Webtx	v2	x-transaction-id	transaction_uid	None
Alerts	anomaly	srcip	src_endpoint.ip	None
Alerts	anomaly	dstip	dst_endpoint.ip	None
Alerts	anomaly	app	app_name	None
Alerts	anomaly	user	user.name	None
Alerts	anomaly	activity	activity	None
Alerts	anomaly	timestamp	_time	None
Alerts	anomaly	url	text	None
Alerts	anomaly	class_name	event_type	None
Alerts	Compromised Credential	timestamp	_time	None
Alerts	policy	srcip	src_endpoint.ip	None
Alerts	policy	dstip	dst_endpoint.ip	None
Alerts	policy	app	app_name	None
Alerts	policy	user	user.name	None
Alerts	policy	activity	activity	None
Alerts	policy	timestamp	_time	None
Alerts	policy	hostname	hostname	None
Alerts	policy	device	device.name	None
Alerts	policy	None	device.type_id	-1
Alerts	policy	os	os.name	None
Alerts	policy	None	os.type_id	-1
Alerts	policy	policy	policy	None
Alerts	policy	url	text	None
Alerts	policy	referer	referrer	None
Alerts	Legal Hold	srcip	src_endpoint.ip	None
Alerts	Legal Hold	dstip	dst_endpoint.ip	None
Alerts	Legal Hold	app	app_name	None
Alerts	Legal Hold	user	user.name	None
Alerts	Legal Hold	activity	activity	None
Alerts	Legal Hold	timestamp	_time	None
Alerts	Legal Hold	hostname	hostname	None
Alerts	Legal Hold	device	device.name	None
Alerts	Legal Hold	None	device.type_id	-1
Alerts	Legal Hold	os	os.name	None
Alerts	Legal Hold	None	os.type_id	-1
Alerts	Legal Hold	mime_type	mime_type	None
Alerts	Legal Hold	policy	policy	None
Alerts	Legal Hold	md5	md5	None
Alerts	Legal Hold	sha256	sha2	None
Alerts	Legal Hold	instance_id	instance_uid	None
Alerts	Legal Hold	modified	modified_time	None
Alerts	Legal Hold	lh_original_filename	file.name	None
Alerts	Legal Hold	file_path	file.path	None
Alerts	Legal Hold	None	file.type_id	-1
Alerts	Malsite	srcip	src_endpoint.ip	None
Alerts	Malsite	dstip	dst_endpoint.ip	None
Alerts	Malsite	app	app_name	None
Alerts	Malsite	user	user.name	None
Alerts	Malsite	timestamp	_time	None
Alerts	Malsite	hostname	hostname	None
Alerts	Malsite	device	device.name	None
Alerts	Malsite	None	device.type_id	-1
Alerts	Malsite	os	os.name	None
Alerts	Malsite	None	os.type_id	-1
Alerts	Malsite	policy	policy	None
Alerts	Malsite	url	text	None
Alerts	Malsite	referer	referrer	None
Alerts	Malsite	app_session_id	session_uid	None
Alerts	malware	local_md5	md5	None
Alerts	malware	local_sha256	sha2	None
Alerts	malware	local_sha1	sha1	None
Alerts	malware	srcip	src_endpoint.ip	None
Alerts	malware	dstip	dst_endpoint.ip	None
Alerts	malware	app	app_name	None
Alerts	malware	user	user.name	None
Alerts	malware	activity	activity	None
Alerts	malware	timestamp	_time	None
Alerts	malware	hostname	hostname	None
Alerts	malware	device	device.name	None
Alerts	malware	None	device.type_id	-1
Alerts	malware	os	os.name	None
Alerts	malware	None	os.type_id	-1
Alerts	malware	file_size	size	None
Alerts	malware	url	text	None
Alerts	malware	referer	referrer	None
Alerts	malware	malware_id	uid	None
Alerts	dlp	srcip	src_endpoint.ip	None
Alerts	dlp	dstip	dst_endpoint.ip	None
Alerts	dlp	app	app_name	None
Alerts	dlp	user	user.name	None
Alerts	dlp	activity	activity	None
Alerts	dlp	timestamp	_time	None
Alerts	dlp	hostname	hostname	None
Alerts	dlp	device	device.name	None
Alerts	dlp	None	device.type_id	-1
Alerts	dlp	os	os.name	None
Alerts	dlp	None	os.type_id	-1
Alerts	dlp	mime_type	mime_type	None
Alerts	dlp	policy	policy	None
Alerts	dlp	md5	md5	None
Alerts	dlp	sha256	sha2	None
Alerts	dlp	file_size	size	None
Alerts	dlp	instance_id	instance_uid	None
Alerts	dlp	url	text	None
Alerts	dlp	dlp_rule	rule.name	None
Alerts	Security Assessment	app	app_name	None
Alerts	Security Assessment	user	user.name	None
Alerts	Security Assessment	activity	activity	None
Alerts	Security Assessment	timestamp	_time	None
Alerts	Security Assessment	os	os.name	None
Alerts	Security Assessment	None	os.type_id	-1
Alerts	Security Assessment	policy	policy	None
Alerts	Security Assessment	instance_id	instance_uid	None
Alerts	Watchlist	srcip	src_endpoint.ip	None
Alerts	Watchlist	dstip	dst_endpoint.ip	None
Alerts	Watchlist	app	app_name	None
Alerts	Watchlist	user	user.name	None
Alerts	Watchlist	activity	activity	None
Alerts	Watchlist	timestamp	_time	None
Alerts	Watchlist	hostname	hostname	None
Alerts	Watchlist	device	device.name	None
Alerts	Watchlist	None	device.type_id	-1
Alerts	Watchlist	os	os.name	None
Alerts	Watchlist	None	os.type_id	-1
Alerts	Qurantine	quarantine_file_id	quarantine_uid	None
Alerts	Qurantine	srcip	src_endpoint.ip	None
Alerts	Qurantine	dstip	dst_endpoint.ip	None
Alerts	Qurantine	app	app_name	None
Alerts	Qurantine	user	user.name	None
Alerts	Qurantine	activity	activity	None
Alerts	Qurantine	timestamp	_time	None
Alerts	Qurantine	hostname	hostname	None
Alerts	Qurantine	device	device.name	None
Alerts	Qurantine	None	device.type_id	-1
Alerts	Qurantine	os	os.name	None
Alerts	Qurantine	None	os.type_id	-1
Alerts	Qurantine	mime_type	mime_type	None
Alerts	Qurantine	policy	policy	None
Alerts	Qurantine	md5	md5	None
Alerts	Qurantine	file_size	size	None
Alerts	Qurantine	transaction_id	transaction_uid	None
Alerts	Remediation	srcip	src_endpoint.ip	None
Alerts	Remediation	dstip	dst_endpoint.ip	None
Alerts	Remediation	app	app_name	None
Alerts	Remediation	user	user.name	None
Alerts	Remediation	activity	activity	None
Alerts	Remediation	timestamp	_time	None
Alerts	Remediation	hostname	hostname	None
Alerts	Remediation	device	device.name	None
Alerts	Remediation	None	device.type_id	-1
Alerts	Remediation	os	os.name	None
Alerts	Remediation	None	os.type_id	-1
Alerts	Remediation	policy	policy	None
Alerts	Remediation	md5	md5	None
Alerts	Remediation	file_size	size	None
Alerts	Remediation	url	text	None
Alerts	Remediation	app_session_id	session_uid	None
Alerts	uba	event_type	class_name	None
Alerts	uba	srcip	src_endpoint.ip	None
Alerts	uba	dstip	dst_endpoint.ip	None
Alerts	uba	app	app_name	None
Alerts	uba	user	user.name	None
Alerts	uba	activity	activity	None
Alerts	uba	timestamp	_time	None
Alerts	uba	hostname	hostname	None
Alerts	uba	device	device.name	None
Alerts	uba	None	device.type_id	-1
Alerts	uba	os	os.name	None
Alerts	uba	None	os.type_id	-1
Alerts	uba	policy	policy	None
Alerts	uba	url	text	None
Events	infrastructure	timestamp	_time	None
Events	infrastructure	device	device.name	None
Events	infrastructure	None	device.type_id	-1
Events	infrastructure	serial	serial_number	None
Events	page	app	app_name	None
Events	page	user	user.name	None
Events	page	timestamp	_time	None
Events	page	device	device.name	None
Events	page	None	device.type_id	-1
Events	page	os	os.name	None
Events	page	None	os.type_id	-1
Events	page	transaction_id	transaction_uid	None
Events	page	client_bytes	bytes_out	None
Events	page	server_bytes	bytes_in	None
Events	page	url	text	None
Events	page	site	service.name	None
Events	page	srcip	src_endpoint.ip	None
Events	page	dstip	dst_endpoint.ip	None
Events	application	app	app_name	None
Events	application	user	user.name	None
Events	application	timestamp	_time	None
Events	application	device	device.name	None
Events	application	None	device.type_id	-1
Events	application	os	os.name	None
Events	application	None	os.type_id	-1
Events	application	transaction_id	transaction_uid	None
Events	application	client_bytes	bytes_out	None
Events	application	server_bytes	bytes_in	None
Events	application	url	text	None
Events	application	srcip	src_endpoint.ip	None
Events	application	dstip	dst_endpoint.ip	None
Events	application	type	type	None
Events	application	site	service.name	None
Events	audit	user	user.name	None
Events	audit	timestamp	_time	None
Events	audit	transaction_id	transaction_uid	None
Events	audit	type	type	None
Events	network	transaction_id	transaction_uid	None
Events	network	app	app_name	None
Events	network	user	user.name	None
Events	network	domain	domain	None
Events	network	start_time	start_time	None
Events	network	end_time	end_time	None
Events	network	timestamp	_time	None
Events	network	device	device.name	None
Events	network	None	device.type_id	-1
Events	network	os	os.name	None
Events	network	None	os.type_id	-1
Events	network	policy	policy	None
Events	network	client_bytes	bytes_out	None
Events	network	server_bytes	bytes_in	None
Events	network	client_packets	packets_out	None
Events	network	server_packets	packets_in	None
Events	network	protocol	protocol_name	None
Events	network	os_version	edition	None
Events	network	srcip	src_endpoint.ip	None
Events	network	srcport	src_endpoint.port	None
Events	network	dstip	dst_endpoint.ip	None
Events	network	dstip	dst_endpoint.port	None
Events	network	site	service.name	None

AWS Security Lake Plugin for Log Shipper