Microsoft Defender for Cloud Apps Plugin for Log Shipper

This document explains how to configure the Microsoft Defender for Cloud Apps integration with the Log Shipper module of the Netskope Cloud Exchange platform. This integration enables you to ingest Netskope Events in CEF and JSON format from the Netskope Tenant to Microsoft Defender for Cloud Apps using the CLS Microsoft Defender for Cloud Apps plugin.

Prerequisites

To complete this configuration, you need:

LogScale Plugin Support

Event Support	Yes
Alert Support	No
WebTx Support	No

Compatibility

Netskope CE: v4.1.0, v4.2.0, v5.0.0

Permissions

For generating the Ingest Token, make sure your user account has the Change ingest tokens option. If not, contact your Organization Owner to generate and provide the Change ingest tokens access to your user.

Performance Matrix

This performance reading is for a Large Stack CE with the below-mentioned VM specifications. The below readings are added with the consideration that it will ingest around 10K events in 3 seconds to the MCAS platform.

Stack Size	Large CPU: 16 Cores RAM: 32 GB
Events ingested to third-party SIEM	200K EPM

API Details

List of APIs used

API Endpoint	Method	Use case
/oauth2/v2.0/token	POST	Generate Token
/api/v1/discovery/upload_url/	GET	Initiate file upload – Cloud Discovery API
<initiate_file_upload_response_url>	PUT	Perform file upload – Cloud Discovery API (obtained from “Initiate file upload”)
/api/v1/discovery/done_upload/	POST	Finalize file upload – Cloud Discovery API

Generate Token

API Endpoint

/oauth2/v2.0/token

Method

POST

Parameters

client_id: client_id

client_secret: client_secret

scope: 05a65629-4c1b-48c1-a78b-804c4abdd4af/.default

grant_type: client_credentials

Headers

Content-Type:application/x-www-form-urlencoded

API Request Endpoint

https://login.microsoftonline.com/<tenant_id>/oauth2/v2.0/token

Sample API Response

{
    "token_type": "Bearer",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IlQxU3QtZExUdnlXUmd4Ql82NzZ1OGtyWFMtSSIsImtpZCI6IlQxU3QtZExUdnlXUmd4Ql82NzZ1OGtyWFMtSSJ9.eyJhdWQiOiIwNWE2R1QVRFcTkxSzk5QUFBLiIsInJvbGVzIjpbImRpc2NvdmVyeS5tYW5hZ2UiXSwic3ViIjoiNzljMEHIywny8JmtEONTPUcOahramZDIYLL8JBGvUH5V-ebPIrAOnCZGvwcbYbVZy7joFwmjeIK22Er_4eCVDXDAzAWuF5uD-KFZp7DkZNSR06i7OD-Yo6YiGEzAP5fMW8anHREJDwh0OtkMn5GRf15ccuhBhNlGiT17uPNzAct*************************************5_DsDgVK109p1yVTrGTw"
}

Initiate File Upload

API Endpoint

/api/v1/discovery/upload_url

Method

GET

Parameters

source: GENERIC_CEF

filename: 140351985108800-ingestion_file.txt

Headers

Authorization: Bearer <token>

Accept: application/json

API Request Endpoint

https://<tenant_id>.<tenant_region>.contoso.com/api/v1/discovery/upload_url/?filename=my_discovery_file.txt&source=GENERIC_CEF

Sample API Response

{
    "url": "https://prod5usw2console1.blob.core.windows.net/discovery-logs/2023-11/115979866_1701326176744_GENERIC_CEF_140351985108800-ingestion_file.txt?se=2023-12-01T06%3A36%3A16Z&sp=w&sv=2014-02-14&sr=b&sig=7EjuI4qdniikC5ehECtNzjjiaHFQ0FMix5Y1sPDDfhs=",
    "provider": "azure"
}

Perform File Upload

API Endpoint

https://<initiate_file_upload_response_url>

Method

PUT

Headers

Accept: application/json

x-ms-blob-type: BlockBlob

API Request Endpoint

https://<initiate_file_upload_response_url>

Sample API Response

Status: 201 created

Finalize File Upload

API Endpoint

/api/v1/discovery/done_upload/

Method

GET

Request Body

{
"uploadUrl":"<initiate_file_upload_response_url>",
"inputStreamName":"<inputStreamName>"
}

Headers

Authorization: Bearer <token>

Content-Type: application/json

Accept: application/json

API Request Endpoint

https://<tenant_id>.<tenant_region>.contoso.com/api/v1/discovery/done_upload/

Sample API Response

{
    "success": true,
    "inputStreamId": "6166e2d26e047397777e71e8",
    "taskId": "656830c12339fba78191038a"
}

User Agent

The user-agent added in this plugin is in the following format netskope-ce-<ce_version>-<module>-<plugin_name>-v<plugin_version>

For example: Netskope-ce-5.0.0-cls-microsoft-defender-for-cloud-apps-v2.1.0

Workflow

Click play to watch a video.

Generate the API Token for the Legacy Method

Follow these steps to generate the API Token. You can also refer to this documentation to generate the token.

1. In the Microsoft 365 Defender portal, select Settings, and choose Cloud Apps. Under System, select API tokens.
2. Select the Add token, provide a name to identify the token in the future, and select Generate.
3. Copy the token value and save it somewhere for recovery. If you lose it you will need to regenerate the token. The token has the privileges of the user who issued it. For example, a security reader can’t issue a token that can alter data.
4. You can filter the tokens by status: Active, Inactive, or Generated.
   • Generated: Tokens that have never been used.
   • Active: Tokens that were generated and used within the past seven days.
   • Inactive: Tokens that were used but there was no activity in the last seven days.
5. After you generate a new token, you’ll be provided with a new URL to use to access the Defender for Cloud Apps portal.

Generate the Client ID, Tenant ID, and Client Secret for OAuth 2.0 (Application Context)

Follow the below steps to generate the Client ID, Tenant ID, and Client Secret for configuring the Microsoft Defender for Cloud Apps plugin using the OAuth 2.0(Application context).

1. Log in to your Microsoft Azure portal.
2. Search for App registrations.
3. Click New Registration.
4. Add your Application Name and click Register.
5. Copy the Client ID and Tenant ID to use in your plugin configuration.
6. Click Add a certificate or secret to get your Client credentials.
7. Click New client secret, enter a description and expiration time, and then click Add.
8. Copy the Value available in the Client Secret field, and make sure to store the Client value because it won’t be shown again.
   
9. Go to API permissions in the left panel and click Add permission to provide the necessary permission for ingesting the data.
10. Go to APIs my organization uses, search for Microsoft Cloud App Security, and then click on it.
11. Select Application permissions, provide the Discovery.manage and Discover.read permissions, and then click Add permission.
12. Make sure to provide the admin consent.

Create a Data Source

1. Go to Microsoft 365 Defender. Click Settings > Cloud Apps.
2. Under Cloud Discovery, click Automatic log upload.
3. Add the name of the data source, and for Source, select Generic CEF log. For Receiver type, select Syslog – TCP.
4. Click Add.
   

Configure the Microsoft Defender for Cloud Apps Plugin

1. Go to Settings > Plugins. Search for and select the Microsoft Defender for Cloud Apps (CLS) plugin box.

   

2. Enter a plugin configuration name, and make sure you have the Microsoft Defender for Cloud Apps Default Mapping selected. 
3. Disable the toggle button to transform the logs if you want to ingest the data in JSON, keep it enabled if you want to ingest the data in CEF format.

   

4. Click Next and enter these parameters:

   For Legacy Method

   • Portal URL: URL for your Microsoft Defender for Cloud Apps platform (without https://)
   • Authentication Method: Legacy Method (API Token)
   • API Token: Microsoft Defender portal’s API Token.
   • Data Source: Data Source name

   For OAuth 2.0 (Application context)

   • Portal URL: URL for your Microsoft Defender for Cloud Apps platform (without https://)
   • Authentication Method: OAuth 2.0 (Application context).
   • Client ID: Client ID of your Azure Portal.
   • Tenant ID: Tenant ID of your Azure Portal.
   • Client Secret: Client Secret of your Azure Portal.
   • Data Source: Data Source name.
5. Click Save. The new plugin configuration will be available on the Cloud Log Shipper > Plugins page.

   

Configure a Business Rule for Microsoft Defender

1. Go to Log Shipper > Business Rules.
2. By default, there’s a business rule that filters all alerts and events. If you want to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding the rule name and filter(s). When finished, click Save.

   

Configure a SIEM Mapping for Microsoft Defender

1. Go to Log Shipper > SIEM Mappings and click Add SIEM Mapping.
2. Select the Source plugin (Netskope CLS), Destination plugin (Microsoft Defender for Cloud Apps), and a business rule, and then click Save.
3. After the SIEM mapping is added, the data will start to be pulled from the Netskope tenant, transformed, and ingested into the Microsoft Defender for Cloud Apps platform.

   

Validate the Plugin

Validate the Pull

To validate the pulling of indicators from the Netskope tenant.

1. Go to Logging in Cloud Exchange and search for the pulled logs.

   

Validate the Push

To validate the plugin workflow on Cloud Exchange.

1. Go to Logging and search for ingested events with the filter message contains ingested, and the ingested logs will be filtered.

   

To validate the push on the Microsoft Defender platform, follow these steps:

1. Go to Microsoft Defender for Cloud Apps Platform.
2. On the left panel, under Cloud apps, click Governance log.
3. Check the files to verify the ingested data is listed.

   

Troubleshooting the Plugin

Receiving the certificate verification failed error while ingesting the indicators

• If you encounter any error with error code “CLS_1005” while ingesting the events after setting up SIEM mapping from Netskope to Microsoft Defender for Cloud Apps.
• It might be due to the Microsoft Defender for Cloud Apps Portal URL being blocked on your machine.

What to do: To resolve this issue, contact your IT support team for assistance to unblock your URL.

Facing issues in the existing plugin configuration after the plugin update

If you have an existing MCAS plugin configured, you might face the below issue after you update your Microsoft Defender for Cloud Apps plugin: CE goes blank while editing the existing plugin configuration.

It is possible to encounter this issue in CE 4.2.0 and CE 5.0.0 if the existing plugin configuration is edited after the user goes on the Configuration Parameters page post the plugin update.

What to do: Verify your CE version and accordingly follow the below-mentioned workarounds.

For 4.2.0: The only solution for 4.2.0 is to delete the existing plugin configuration and create a new one.

For 5.0.0: If the plugin hasn’t already been updated, make sure to follow the below steps and save the plugin configuration while updating, not skip it.

• If you are using the Legacy Method (API Token) and want to continue using the Legacy Method, make sure your Authentication Method dropdown has Legacy Method (API Token) selected before you save the configuration in the plugin update on the Plugin Repository page.
• If you want to use the OAuth 2.0 (Application context) method, make sure to select it in the Authentication Methods dropdown and provide all configuration parameters. Do not skip.

Either way one of the solutions for any of the methods during the plugin update on the Plugin Repository page is to save the plugin update and not skip it.

If you have already updated the plugin and have selected skip while updating the plugin, delete the existing plugin configuration and configure a new MCAS plugin.

Receiving the File Upload error while ingesting the events to the MCAS platform

If the plugin is configured correctly and yet the below error is received, it means that the Data Source provided in the plugin configuration is not available on the MCAS platform.

What to do: Go to your Microsoft Defender for Cloud Apps platform.

1. Go to the Automatic Log upload from Settings > Cloud Apps > Cloud Discovery.
2. Search for the Data Source name used in the plugin configuration. If the data source name is not found, create a new Data Source.
3. If the Data Source name is available, make sure the exact data source name is used.

Difficulty in tracking the ingested data

If any of the data ingestion is hard to track due to multiple files created on the Microsoft Defender platform, you can simply search the file name.

What to do: Go to Logging in Cloud Exchange and check the API Request logs from the MCAS plugin. The logs have the file name that would be created on the MCAS platform for ingested data. Copy the file name and search for the file on the MCAS platform in the Governance Log under Cloud apps.

Limitations

• There is one limitation imposed by Microsoft Defender for Cloud Apps: it can only transmit files with a maximum size of 64 MB. If this limit is exceeded, an error indicating that the file is larger than 64 MB will be encountered in the plugin while ingesting the data.
• There is a limitation regarding the data retention on the Microsoft Defender for Cloud Apps: If the ingested data is older than 90 days, it will show a failed status on the Governance Log page where the events are ingested.