Elastic Plugin for User Risk Exchange

This document explains how to configure the URE Elastic plugin for the URE module of the Netskope Cloud Exchange platform. This plugin is used to fetch the user and their risk scores from the Elastic platform. This plugin does not support performing any actions on the Elastic platform.

Prerequisites

To complete this configuration, you need:

• A Netskope tenant (or multiple, for example, production and development/test instances).
• A Netskope Cloud Exchange tenant with the Tenant plugin and Risk Exchange plugin already configured.
• Your Elastic instance credentials: Username, Password, API Base URL for Elastic Search, API Key.
• Connectivity to the following host: your Elastic instance.

CE Version Compatibility

Netskope CE v4.2.0, v5.0.0

Plugin Scope

This plugin fetches users available on the Security > Explore > Users > All users page, and their respective risk scores available on the Security > Explore > Users > User risk page from your Elastic instance.

Elastic Plugin Support

Type of data pulled	Users
Actions	No Action

Mappings

Elastic – Netskope Pull Mapping

Elastic Field	Netskope CE Field
user.name	email
risk.calculated_score_norm	score

Score Mapping

The score will be normalized in the Cloud exchange using the below formula:

URE score calculation > | 100 – Elastic Risk Score | x 10

Netskope Score	Netskope Score Range
Critical	0-250
High	251-500
Medium	501-750
Low	751-1000

Elastic Score	Elastic Score Range
Unknown	< 20
Low	20 – 40
Moderate	40 – 70
High	70 – 90
Critical	> 90 **

Permissions

Below are the permissions needed for the plugin.

• Cluster Privileges > manage_tokens
• Index privileges > logs-* and risk-score.*
• Privileges > Read
• API Key > Restrict Privileges

Performance Matrix

Below are the performance readings conducted on a Large CE Stack with below mentioned VM specifications, by pulling 500K User and their respective Risk Scores.

Stack details	Size: Large RAM: 32 GB CPU: 16 Cores
Users with User Risk Sores fetched from Elastic	500K
Time taken to ingest Users and their Risk scores	~20 mins
Actions performed on third-party product	NA

API Details

List of APIs used

API Endpoint	Method	API Client Scope	Use Case
/_security/oauth2/token	POST	manage_token	Get OAuth2 token
/logs-*/_search	POST	logs-* > read access	Fetch Users
/risk-score*/_search	POST	risk-score* > read access	Fetch User Risk Scores

Get Auth Token

API Endpoint:

<Base URL>/_security/oauth2/token

Method: POST

Headers:

Key	Value	Description
Content-Type	application/json
Accept	application/json
Authorization	ApiKey <Token>	For basic authentication, like Username-Password, Token should be utf-8 encoded
	Basic <Token>

Body:

{
  "grant_type" : "client_credentials"
}

Sample API Response:

{
    "access_token": "gLuKBBjxurcPqq7cZyTj4jL0A****j*******",
    "type": "Bearer",
    "expires_in": 1200,
    "authentication": {
        "username": "test user",
        "roles": [
            "Netskope"
        ],
        "full_name": "Test",
        "email": "",
        "metadata": {},
        "enabled": true,
        "authentication_realm": {
            "name": "native",
            "type": "native"
        },
        "lookup_realm": {
            "name": "native",
            "type": "native"
        },
        "authentication_type": "realm"
    }
}

Fetch Records

API endpoint:

<Base URL>/logs-*/_search

Method: POST

Headers:

Key	Value	Description
Content-Type	application/json
Accept	application/json
Authorization	ApiKey <Token>	For basic authentication, i.e. Username-Password, Token should be utf-8 encoded
	Basic <Token>

Body:

{
            "query": {
                "bool": {
                    "must_not": {"exists": {"field": "host.name"}},
                    "filter": [
                        {"exists": {"field": "user.name"}},
                        {"range": {"@timestamp": {"gte": "2023-01-01T18:08:11.148813Z"}}},
                    ],
                }
            },
            "sort": [{"@timestamp": "asc"}],
            "_source": True,
            "size": 10000,
        }

Sample API Response:

{
    "took": 10,
    "timed_out": false,
    "_shards": {
        "total": 75,
        "successful": 75,
        "skipped": 53,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 491,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "logs-tes",
                "_id": "wHUW2YEyx7Kaj*****",
                "_score": null,
                "_source": {
                    "event": {
                        "kind": "alert",
                        "module": "proofpoint"
                    },
                    "user": {
                        "name": "kamlesh.solanki@crestdatasys.com"
                    },
                    "@timestamp": "2024-01-03T09:00:00.000Z"
                },
                "sort": [
                    1704272400000
                ]
            }
        ]
    }
}

Fetch Scores

API Endpoint:

<Base URL>/risk-score.*/_search

Method: POST

Headers:

Key	Value	Description
Content-Type	application/json
Accept	application/json
Authorization	ApiKey <Token>	For basic authentication, i.e. Username-Password, Token should be utf-8 encoded
	Basic <Token>

Body:

{
    "query": {
        "bool": {
            "minimum_should_match": 1,
            "should": [{"match_phrase": {"user.name": "testuser@test.com"}}]
        }
    }
}

Sample API Response:

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 0.9808291,
        "hits": [
            {
                "_index": "risk-score.risk-score-latest-default",
                "_id": "X2vS7pclkEYZFSG************",
                "_score": 0.9808291,
                "_source": {
                    "@timestamp": "2024-01-23T11:16:33.549Z",
                    "user": {
                        "name": "testuser@test.com",
                        "risk": {
                            "id_field": "user.name",
                            "id_value": "testuser@test.com",
                            "calculated_level": "Low",
                            "calculated_score": 92.29454468696414,
                            "calculated_score_norm": 35.33481802716851,
                            "category_1_score": 89,
                            "category_1_count": 9,
                            "notes": [],
                            "inputs": [
                                {
                                    "id": "953f2530b0ea1ca14876583df459c8e24a21e9989876876869*****************",
                                    "index": ".internal.alerts-security.alerts-default-000003",
                                    "description": "Alert from Rule: External Alerts [Duplicate]",
                                    "category": "category_1",
                                    "risk_score": 47,
                                    "timestamp": "2024-01-10T09:36:16.888Z"
                                },
                                {
                                    "id": "2b1f87b7353ae4f1cebc95fc618a6b4cb3696b5b8383c3f***********",
                                    "index": ".internal.alerts-security.alerts-default-000003",
                                    "description": "Alert from Rule: External Alerts [Duplicate]",
                                    "category": "category_1",
                                    "risk_score": 47,
                                    "timestamp": "2024-01-10T09:36:16.890Z"
                                },
                                {
                                    "id": "3b5d3e2b1b0017567b3e2dc6835957a3118486e89fa20***********************",
                                    "index": ".internal.alerts-security.alerts-default-000003",
                                    "description": "Alert from Rule: External Alerts",
                                    "category": "category_1",
                                    "risk_score": 47,
                                    "timestamp": "2024-01-05T11:53:39.960Z"
                                },
                                {
                                    "id": "fe788d9601e817f0e7219b56686d9ba4a55d90c5ea4627***************a",
                                    "index": ".internal.alerts-security.alerts-default-000003",
                                    "description": "Alert from Rule: External Alerts",
                                    "category": "category_1",
                                    "risk_score": 47,
                                    "timestamp": "2024-01-05T11:53:39.963Z"
                                },
                                {
                                    "id": "7c7f1da36f5fbc904bb109c38d677e061fd4cbc************************************",
                                    "index": ".internal.alerts-security.alerts-default-000003",
                                    "description": "Alert from Rule: External Alerts [Duplicate]",
                                    "category": "category_1",
                                    "risk_score": 47,
                                    "timestamp": "2024-01-05T11:49:27.539Z"
                                }
                            ]
                        }
                    }
                }
            }
        ]
    }
}

User Agent

Netskope-ce-5.0.0-ure-elastic-v1.0.0

Workflow

1. Create a Role.
2. Configure a User with a Role.
3. Create an API Key.
4. Configure the Elastic plugin
5. Add a Business Rule.
6. Add Actions.
7. Validate the plugin.

Click play to watch a video.

Create a Role

1. Log in to your Elastic platform (Kibana, Elastic Search) and go to Management > Stack Management > Security > Roles.
2. Click Create Roles.
3. Provide a Role Name and select a manage_token access in the Cluster privileges.
4. Scroll down to Index privileges and add logs-* and risk-score.* indexes under Indices, and then click Create role.

Create a User with the New Role

1. Go to Stack Management > Users.
2. Click Create User and enter a Username, Email address, Password, confirm the password, and then select the previously created role in the Privileges dropdown.
3. Click Create User.

Create an API Key (for API Key Authentication Method)

1. Go to Stack Management > Security > API Keys and click Create API Key.
2. Provide a name and enable the Restrict Privileges toggle button. Add below mentioned dictionary in the role descriptor.

   {
     "role-a": {
       "cluster": [
         "all"
       ],
       "indices": [
         {
           "names": [
             "logs-*",
             "risk-score.*"
           ],
           "privileges": [
             "read"
           ],
           "allow_restricted_indices": false
         }
       ],
       "applications": [],
       "run_as": [],
       "metadata": {},
       "transient_metadata": {
         "enabled": true
       }
     }
   }
   

3. Enable Include metadata and add the below dictionary in it.

   { "application": "Netskope Cloud Exchange" }

   

4. Click Create API Key. Copy the API Key and save it somewhere safe because it will only be visible once.

Configure the Elastic Plugin

1. Log in to Cloud Exchange and go to Settings > Plugins.
2. Search for and select the Elastic plugin box to configure the plugin.
3. Enter the plugin Configuration name and Sync Interval, and then click Next.
4. Enter these values:
   • Base URL: Enter the API Base URL of your Elastic Search.
   • Authentication Method: Select the type of Auth method you want to use from Basic Authentication or API Key Authentication.
   • Username: Enter the Username of your user on Elastic, used only when Basic Authentication method is selected.
   • Password: Enter the password of your Elastic user, used only when Basic Authentication method is selected.
   • API Key: Enter the API Key generated previously, for when only the API Key Authentication method is used.
   • Initial Range: Enter an initial range from where you want to pull data from.
5. Click Save.

Add a Business Rule for Elastic

1. Go to User Risk Exchange > Business Rule.
2. Click Create New Rule.
3. Enter the Rule Name and configure the query based on your requirements. The below example fetches all the users fetched from the Elastic plugin.

Configure Actions for Elastic

The Elastic plugin does not support performing actions. But using “No Action” alerts can be generated in the Netskope CTO module.
To configure this action, follow these steps.

1. Go to User Risk Exchange > Actions.
2. Click Add Action Configuration.
3. Select a Business Rule, Configuration, and Action.
4. To generate Alerts in the CTO module, enable Generate Alert, and also enable Perform action during the maintenance window if you wish to perform an action during the Maintenance Window. Make sure to enable the CTO (Ticket Orchestrator) module from Settings > General.
5. Click Save.

Validate the Elastic Plugin

Validate the Pull

To verify the Users pulled from Elastic, go to Risk Exchange > Users.

Go to Settings > Logging and search for logs based on the plugin configuration.

To check the Users available for pulling on the Elastic platform, go to Security on the left panel, and then go to Explore > Users > All Users. Check the users available for all time on the platform.

To check that the user scores pulled, go to Security > Explore > Users > User Risk.

Validate the Actions

This plugin does not support performing actions.

Troubleshooting

Unable to save the plugin from Plugin Repo

Receiving internal Server error while fetching the repo updates from the CE plugins repository when a new plugin is added.

What to do: This issue will be addressed in the upcoming CE release, but for now download the plugin zip and add the plugin in a repo. Add the new repo in CE Plugin Repositories. The new plugin will be added in CE.

Unable to pull Users or User Risk Scores

If you are unable to pull Users or User Risk Scores in CE, check below possible scenarios.

• The users are available on the platform to pull
• The user are available in the given initial range provided in the plugin
• The User Risk Scores are not pulled in CE.

What to do:

• If the users are not fetched from the plugin check if you have users available on the Elastic platform.
• If you have users available on Elastic to pull check the last seen date on which the users are available. Check the initial range provided in the plugin configuration. The initial range should be greater than or equal to the last seen time available on the Elastic so the users can be pulled.

• If you are not able to pull User Risk Scores after users are pulled, verify if you have correct permissions added to your user and verify the scores are available for the users.