Microsoft Azure Monitor Plugin for Log Shipper

This document explains how to configure the Microsoft Azure Monitor v2.0.0 plugin with the Log Shipper module of the Netskope Cloud Exchange platform. This plugin supports ingestion of Alerts (DLP, Malware, Policy, Compromised Credential, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, UBA, CTEP, Device, Content), Events (Page, Application, Audit, Infrastructure, Network, Incident, Endpoint, Client Status). The data will be ingested in the Microsoft Azure Monitor Log Analytics Workspace table. This plugin supports ingestion in CEF and JSON format.

Prerequisites

• A Netskope tenant (or multiple, for example, production and development/test instances).
• A Netskope Cloud Exchange tenant with the Tenant plugin and Log Shipper plugin already configured.
• The Microsoft Azure Application’s Tenant ID, Client ID and Client Secret.
• A Microsoft Azure Log Analytic Workspace.
• A Microsoft Azure Monitor Data Collection Endpoint.
• A Microsoft Azure Monitor Data Collection Rule.
• Connectivity to the following host: https://portal.azure.com/.

Microsoft Azure Monitor Plugin Support

This plugin supports ingestion of Alerts (DLP, Malware, Policy, Compromised Credential, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, UBA, CTEP, Device, Content), Events (Page, Application, Audit, Infrastructure, Network, Incident, Endpoint, Client Status). The data will be ingested in the Microsoft Azure Monitor Log Analytics Workspace table. This plugin supports ingestion in CEF and JSON format.

Permissions

Requires an Azure Account with Monitor access.

{
    "client_id": "<client_id>",
    "client_secret": "<application_secret>",
    "scope": "https://monitor.azure.com/.default",
    "grant_type": "client_credentials",
}

{
    "token_type": "Bearer",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6InNNMV95QXhWOEdWNHlOLUI2ajJ4em1pazVBbyIsImtpZCI6InNNMV95QXhWOEdWNHlOLUI2ajJ4em1pazVBbyJ9.eyJhdWQiOiJodHRwczovL21vbml0b3IuYXp1cmUuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvZTA0MDU2MDYtNmMw…."
}

{
    "RawData": "ID=a39cdxxxx5caexx52fd47a6a cnt=1 device=qP7NPtp5bTHzxxxxdho_AA0B8A97-xxxx-xxxx-AF0E-xxxxxx deviceExternalId=xxxxxx-9798-B907-xxxx-6029DF2F6EB2 dvchost=VMware, Inc. hostname=XXXXX-GNO7JR8 managementId=null os=0 osVersion=10.0.19045 slat=0.0 slong=0.0 smac=xx:0C:29:xx:F7:A1 suid=xxxxPtp5bTHzcR69xxxx suser=abc@test.ai timestamp=0",
    "Application": "Netskope CE",
    "DataType": "<data_type>",
    "SubType": "<sub_type>",
    "TimeGenerated": "2025-05-05 17:56:57.823408"
}

204 No Content

Performance Matrix

This performance reading is for a Large Stack CE tested on the below-mentioned VM specifications. The below readings are added with the consideration that it will ingest around 10K file size in 2 seconds for Events, Alerts.

User Agent

netskope-ce-6.0.1-cls-microsoft-azure-monitor-v2.0.0

Workflow

1. Configure a Log Analytics Workspace.
2. Configure an Application and get your Tenant ID, Application ID and Client Secret.
3. Configure a Data Collection Endpoint and get your DCE URI.
4. Configure a Basic Table in Log Analytics Workspace and get your Data Collection Rule Immutable ID.
5. Assign a Permission to DCR and DCE.
6. Configure the Microsoft Azure Monitor plugin.
7. Configure a Log Shipper Business Rule for Microsoft Azure Monitor.
8. Configure Log Shipper Log Delivery for Microsoft Azure Monitor.
9. Validate the Microsoft Azure Monitor plugin.

Watch a Video

Click play to watch a video.

Configure a Log Analytics Workspace

1. Log in to Azure and go to Log Analytics Workspace.

   

2. Click Create Tab.
3. Select Subscription, and then select an existing Resource Group (or create a new one).
4. Enter a name for your Log Analytics Workspace, select a region, and then click Next > Next > Create.

   

Configure an Application and get your Tenant ID, Application ID and Client Secret

1. Log in to Azure with an account that has a Global Administrator role.
2. Go to Azure App Registration > New Registration.

   

3. In the registration form, enter a name for your application, and then click Register.
4. Make a copy of the Tenant ID and Application (client) ID on the application page.

   

5. Click Add a Certificate or Secret, and then click New client secret to generate a Client secret. Add a description and Expire time, and then click Add.
6. Copy the value of Secret, as it will only be displayed once.

   

Configure a Data Collection Endpoint and get your DCE URI

1. Go to Azure Home and click Monitor from the Azure services.
2. Click Data Collection Endpoints on the left panel, and then click Create.

   

3. Enter a name for the Data collection Endpoint, select a Subscription and Resource Group, select a region (make sure that this region is the region of your Log Analytics Workspace), and then click Review + create.

   

4. On the Overview tab, copy the Logs Ingestion that will be your Data Collection Endpoint DCE URI.

Configure a Basic Table in Log Analytics Workspace and get your Data Collection Rule Immutable ID

1. A Custom Log Analytics Table requires sample data to be uploaded in order to create a JSON file on your system with the following content:

   [   
       {         
           "RawData": {},         
           "Application":  "",         
           "DataType": "", 
           "SubType": "", 
           "TimeGenerated": "2022-11-01 12:00:00.576165"     
       } 
   ]

2. On the Azure home tab, go to Log Analytics Workspace, select the workspace created previously, select Tables. Click Create and select New Custom log (DCR based).

   

3. Enter a name for the table.
4. For Data Collection Rule, click Create a new data collection rule, and then select a Subscription and Resource Group from the dropdown lists. Enter the region for your Log Analytics Workspace, and click Done. Make sure to keep Table plan as Basic.

   

5. The new Data Collection Rule will be selected in the Data collection rule field. Click Next.
6. On the Schema and Transformation tab, click Browse for Files and select the sample data JSON file you created previously.

   

7. Click Next and then click Create.
8. A Custom Log Table will be created with the suffix _CL.

   

   Note

   Here we are changing the table Plan from Analytics to Basic because the Basic log data plan lets you save on the cost of ingesting, and storing high-volume verbose logs in your Log Analytics workspace for debugging, troubleshooting, and auditing.If table plan is not changed and kept as Analytics, the Logs will still be ingested in the Table without any issue.The Analytics table has a configurable retention period from 30 days to 730 days. The Basic table has Retention fixed at eight days.Basic Logs tables retain data for eight days. When you change an existing table’s plan to Basic Logs, Azure archives data that’s more than eight days old, but still within the table’s original retention period.

9. To get the Data Collection Immutable ID, go to Home, select Monitor from the Azure Services > Data Collection Rules, and then select the DCR you created while creating the Custom Table.

   

10. In the Overview tab, click JSON View from the top right corner, and copy the Immutable ID.

    

Assign a Permission to DCR and DCE

1. On the Azure Home page, go to Monitor > Data Collection Endpoint and select the Endpoint created previously.
2. Select Access control (IAM) and click Add role assignment.

   

3. From the list of roles, select Monitoring Metrics Publisher and click Next.

   

4. Select a User, group, or service principal for which to assign access.
5. Click Select Members and search for the Application you created in the search box, and then select it.

   

6. Click Review + assign.
7. Repeat these same steps to assign permissions to the DCR (Data Collection Rule).

Configure the Microsoft Azure Monitor Plugin

1. In Cloud Exchange, go to Settings > Plugin Store.
2. Search for and select the Microsoft Azure Monitor v2.0.0 CLS) plugin.

   

3. Enter a Configuration Name a select a valid Mapping. (Default Mappings for all plugins are available. If you want to create a new mapping, go to Settings > Log Shipper > Mappings).
4. Transform the raw logs is enabled by default, which will transform the raw data on the basis of the Mapping file. Turn it off if you want to send Raw data directly to Azure Monitor.

   

5. Click Next and enter the Configuration Parameters:
   • Directory (tenant) ID: Directory (tenant) ID of your AzureAD Application.
   • Application (client) ID: Application (client) ID of your AzureAD Application.
   • Client Secret: Client Secret of your AzureAD Application.
   • DCE URI: URI of the Data Collector Endpoint.
   • DCR Immutable ID: Immutable ID of the Data Collection Rule.
   • Custom Log Table Name: Custom Log Table name for ingesting data. Make sure that the Table exists in your Log Analytics Workspace.
   • Log Source Identifier: This will be added in Application field of the Custom Log Table. 

   

6. Click Save.

   

Configure a Log Shipper Business Rule for Microsoft Azure Monitor

1. In Log Shipper, go to Business Rules and click Create New Rule.
2. By default, there’s a business rule that filters all alerts and events. If you want to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding the rule name and filter(s).
3. Enter a Folder Name, if any.
4. Click Save.

   

Configure Log Shipper Log Delivery (SIEM Mappings) for Azure Monitor

1. In Log Shipper, go to Log Delivery and click Add Log Delivery Configuration.
2. Select a Source Configuration, Destination Configuration, and Business Rule.

   

3. Click Save.

   

After the SIEM mapping is added, the data will start to be pulled from the Netskope tenant and ingested into the Azure Monitor platform.

Validate the Microsoft Azure Monitor Plugin

Validate the Pull

To validate the pulling of indicators from the Netskope tenant.
Go to Logging in Cloud Exchange and search for the pulled logs with the filter: message contain pulled.

Validate the Push

To validate the plugin workflow in Cloud Exchange:

Go to Logging and search for ingested events with the filter: message contains ingested. The ingested logs will be filtered.

To validate in Log Analytics Workspace:

1. In the Azure portal, go to Log Analytics Workspace, select the Log Analytics Workspace that you created, and select Logs under the General Category on the left panel.
2. Enter the Custom Log Table Name in the query editor and click Run. You can select the Time Range from the top to filter out logs.

   

   

You can verify the events/alerts by expanding:

Also, here are the ingested alerts/events in JSON format:

Troubleshooting

If receive error code 403 while configuring the plugin in toast and log message

Ensure that you have the correct permissions for your application to the DCR. Check if you have assigned permissions to the correct Data Collection endpoint as described above. It may take up to 30 minutes to reflect the assigned permissions.

Difficulties in saving the Microsoft Azure Monitor plugin

Despite entering all parameters and clicking the Save button, an error may occur, possibly due to the configuration differs from the specified settings.

What to do:

It could be because of incorrect configuration parameters, just follow the steps in the Configure a Log Analytics Workspace.

Not able to see the events on the Microsoft Azure Monitor

Even after successful ingestion of the events, not able to see the events ingested from the plugin. This could be due to one of these reasons:

• Incorrect query provided in Log searching.
• No events are ingested in the platform.
• Or the data you are looking for is outside of the searching Time Range.

What to do:

1. Check events are ingested in a longer time range.
2. Check if you have provided the correct query in Log searching.
3. Check for the logs on the Cloud Exchange for the ingested events.