Application Risk Exchange Custom Plugin Developers Guide

This guide explains how to write a new Application Risk Exchange plugin to extract the maximum value out of the customer risk ecosystem by leveraging the functionality provided within the Application Risk Exchange module. The developer should be able to write a new plugin independently without any technical issues by following this guide.

Prerequisites

• Python 3.x programming experience (intermediate level).
• Access to the Netskope Cloud Exchange platform.
• API or Python SDK access to the product or solution for which you need to write the plugin.
• An account with minimum permission for the product.

Application Risk Exchange Module

The Cloud Exchange platform, and its Application Risk Exchange module, comes with a rich set of features and functionality that allow for a high degree of customization, so we recommend that you familiarize yourself with the different aspects of the platform as listed below. This module supports sharing of data from Netskope to a third-party tool.

Netskope Concepts and Terminology

• Core: CE core engine manages the 3rd party plugins and their life cycle methods. It has API endpoints for interacting with the platform to perform various tasks.
• Module: The functional code are as that invoke modular-specific plugins to accomplish different workflows. Application Risk Exchange is one of the modules running in Cloud Exchange.
• Plugin: Plugins are Python packages that have logic to fetch users and risk scores from 3rd party Threat Intel systems, which will then be stored in the Application Risk Exchange. It can also perform actions.
• Plugin Configurations: Plugin configurations are the plugin class objects which are configured with the required parameters and are scheduled by the Cloud Exchange core engine for fetching users and scores.
• Applications (application events): application details from the application events of a Netskope tenant, which are then shared with other Application Risk Exchange configured plugins.

Development Guidelines

• Use the Package Directory Structure for all Python code.
• Make sure all the 3rd party libraries packaged with the plugin package are checked for known vulnerabilities.
• Make sure to follow standard Python code conventions. (https://peps.python.org/pep-0008/)
• Run and verify that the flake8 lint check passes with the docstring check enabled.The maximum length of a line should be 80.
• Convert the time-stamp values to the human-readable format (from epoch to DateTime object). Make sure the time that is being displayed on the UI should be in the local timezone.
• If possible, add a default value while adding a configuration parameter in the plugin.
• For Scripts/Integrations written in Python, make sure to create unit tests. Please refer to the below section Unit Testing.
• Plugin architecture allows storing states, however, avoid storing huge objects for state management.
• Check your python code for any vulnerabilities.
• The plugin icon has to be under 10kb. Make sure to use the company logo(and not the product’s) with a transparent background. The recommended size for the logo is 300×50 or a similar aspect ratio.
• Make sure to use all the possible fields available in Application data model while pushing the Application details to the third party product.
• Follow the Plugin Directory Structure.
• If the description contains a link, it should be a hyperlink instead of plaintext.
• Use proper validation for the parameters passed to the validate method and provide the proper help text for all the parameters.
• Use a notifier object to raise the notification for failures or critical situations (like rate-limiting, or exceeding payload size) to notify the status of the plugin to the user.
• Make sure to implement a proper logging mechanism with the logger object passed by the Cloud Exchange platform. Make sure enough logging is done which helps the Ops team in troubleshooting. Make sure any sensitive data is not logged or leaked in the notification.
• The logger messages and the Toast messages should not contain the API Token and the Password type field values.
• Pagination should always be considered while developing any feature in a plugin.
• Make sure to add a retry mechanism for the 429 status code.
• Use a notifier object to raise the notification for failures or critical situations (like rate-limiting, or exceeding payload size) to notify the status of the plugin to the user.
• Make sure to implement a proper logging mechanism with the logger object passed by the Cloud Exchange platform. Make sure enough logging is done which helps the Ops team in troubleshooting. Make sure any sensitive data is not logged or leaked in the notification.
• Provide the proper help text (tooltip) for all the parameters. If feasible, make sure to explain the significance of the parameter in the tooltip.
• Make sure to provide a meaningful name and description to plugin configuration parameters.
• Make sure to provide an appropriate configuration type (text, number, password, choice, multi-choice) to the parameters.
• Make sure to use the proxy configuration dict and the SSL certificate validation flag that is passed by the Cloud Exchange platform while making any outbound request (API/SDK).
• Make sure to collect the value of a non-mandatory parameter using the .get() method and provide a default value while using .get() method.
• API Tokens and Password fields should not use strip().
• The log messages should start with “<module> <app name> Plugin [configuration_name]: “. Example: “ARE Bitsight Plugin [Bitsight Configuration Name]: <log_message>“. [This is a suggestion, we can avoid configuration name]. (logger.info(“<module> <plugin_name> Plugin: <message>”))
• While logging an error log, if possible, we should add a traceback of the exception. USE: self.logger.error(error, details=traceback.format_exc()).
• The Toast message should not contain the <app_name> <module> Plugin: in the message.
• Make sure to catch proper exceptions and status codes while and after making the API calls. If feasible, Devs can create a helper method from where the requests would be made and this method can be called with proper parameters wherever required.
• The CHANGELOG.md file should be updated with proper tags such as Added, Changed, and Fixed along with a proper user-friendly message. Make sure the file name should exactly matches CHANGELOG.md.
• Make sure the plugin directory name (sample_plugin) matches with the manifest.json’s ID field.
• User Agent should be added to the headers while making any API call. Format for the User Agent: netskope-ce-<ce_version>-<module>-<plugin_name>-<plugin_version>.Note: The plugin version should be dynamically fetched and to fetch netskope-ce-<version> string use the method defined by core.
• Pagination should always be considered while developing any feature in a plugin.
• All the logger statement should follow this format:

  logger.info(“<module> <plugin_name> Plugin: <message>”)

Writing a Plugin

This section explains the process of writing a plugin from scratch.

Download the sample plugin from the NetskopeOSS public Github repository or from the Cloud Exchange Knowledge Base found here: https://support.netskope.com/hc/en-us/articles/360052128734-Cloud-Threat-Exchange.

Development setup

Python

Our system utilizes Python3 (v3.7 and above). Make sure to set up python3 in your development environment. Pytest is used to run unit tests.

Included Python Libraries

Following Python libraries are included within the Netskope Cloud Exchange platform.

Library Name	Version
aiofiles	22.1.0
amqp	5.1.1
anyio	3.6.2
asgiref	3.6.0
attrs	22.2.0
azure-core	1.26.2
azure-storage-blob	12.14.1
bcrypt	4.0.1
boto3	1.26.51
botocore	1.29.51
billiard	3.6.4.0
celery	5.2.7
cabby	0.1.23
cachetools	5.2.1
celerybeat-mongo	0.2.0
certifi	2022.12.7
cffi	1.15.1
chardet	5.1.0
charset-normalizer	3.0.1
click	8.1.3
click-didyoumean	0.3.0
click-plugins	1.1.1
click-repl	0.2.0
colorama	0.4.6
colorlog	6.7.0
cryptography	39.0.0
cybox	2.1.0.21
defusedxml	0.7.1
dnspython	2.3.0
docker	6.0.1
fastapi	0.89.1
furl	2.1.3
google-api-core	2.11.0
google-auth	2.16.0
google-cloud-core	2.3.2
google-cloud-pubsub	2.13.12
google-cloud-pubsublite	1.6.0
google-cloud-storage	2.7.0
google-crc32c	1.5.0
google-resumable-media	2.4.0
googleapis-common-protos	1.58.0
grpc-google-iam-v1	0.12.6
grpcio	1.51.1
grpcio-status	1.51.1
gunicorn	20.1.0
h11	0.14.0
idna	3.4
importlib-metadata	6.0.0
isodate	0.6.1
jmespath	1.0.1
jsonpath	0.82
jsonschema	4.17.3
kombu	5.2.4
libcst	0.3.21
libtaxii	1.1.119
lxml	4.9.2
mongoengine	0.25.0
more-itertools	9.0.0
MarkupSafe	2.1.2
memory-profiler	0.61.0
mixbox	1.0.5
mongoquery	1.4.2
msrest	0.7.1
multidict	6.0.4
mypy-extensions	0.4.3
netskopesdk	0.0.25
numpy	1.23.5
oauthlib	3.2.2
onelogin	3.1.0
ordered-set	4.1.0
orderedmultidict	1.0.1
overrides	6.5.0
pandas	1.5.0
packaging	23.0
passlib	1.7.4
pycparser	2.21
prompt-toolkit	3.0.36
proto-plus	1.22.2
protobuf	4.21.12
psutil	5.9.4
pydantic	1.10.4
pyasn1	0.4.8
pyasn1-modules	0.2.8
PyJWT	2.6.0
pymongo	4.3.3
pyparsing	3.0.9
python-dateutil	2.8.2
pyrsistent	0.19.3
python-multipart	0.0.5
python3-saml	1.15.0
pytz	2022.7.1
PyYAML	6.0
requests	2.28.2
requests-oauthlib	1.3.1
rsa	4.9
six	1.16.0
starlette	0.22.0
sniffio	1.3.0
s3transfer	0.6.0
stix	1.2.0.11
taxii2-client	2.3.0
typing-inspect	0.8.0
typing-utils	0.1.0
typing_extensions	4.4.0
urllib3	1.26.14
uvicorn	0.20.0
vine	5.0.0
wcwidth	0.2.6
weakrefmethod	1.0.3
websocket-client	1.4.2
Werkzeug	2.2.2
xmlsec	1.3.11
zipp	3.11.0
requests-mock	1.7.0

Including Custom Plugin Libraries

Netskope advises bundling any of the third-party python libraries your plugin will need into the plugin package itself. Use the pip installer to achieve this bundling; it provides a switch which takes a directory as an input. If the directory is provided, pip will install the packages into that directory.

For example, the command shown below will install the cowsay package into the directory lib.

> pip install cowsay --target ./lib

For the official documentation on this, refer https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-t.
While importing modules from above lib folder, we should be using relative import instead of absolute import.

IDE

Recommended IDEs are PyCharm or Visual Studio Code.

Plugin Directory Structure

This section mentions the typical directory structure in an Application Risk Exchange Plugin.

/sample_plugin/
  ├── __init__.py
  ├── Changelog.md
  ├── icon.png
  ├── main.py
  └── manifest.json

• __init__.py: Every plugin package is considered a python module by the Application Risk Exchange code. Make sure every plugin package contains the empty “__init__.py” file.
• CHANGELOG.md: This file contains the details about plugin updates and should be updated with proper tags such as Added, Changed, and Fixed along with a proper user-friendly message.
• icon.png: Plugin icon logo, this will be visible in the plugin chiclet and configuration cards on the UI. The logo should have a transparent background with the recommended size of 300*50 pixels or a similar aspect ratio.
• main.py: This python file contains the Plugin class containing the concrete implementation for the push, validate, and get_target_fields method.
• manifest.json: Manifest file for the plugin package, containing information about all the configurable parameters and their data types. This file has more information about the plugin integration as well.

The listed files here are mandatory for any plugin integration, but developers can add other files based on specific integration requirements.

CHANGELOG.md

This is a file that contains details about plugin updates and should be updated with proper tags such as Added, Changed, and Fixed along with a proper user-friendly message.

• Added: Use when new features are added.
• Fixed: Use when any bug/error is fixed.
• Changed: Use when there is any change in the existing implementation of the plugin.

# 1.0.1
## Fixed
- Fixed pagination when there are more than 10k Logs.

# 1.0.0
## Added
- Initial release.

Manifest.json

This is a JSON file that stores the meta-information related to the plugin, which is then read by the Application Risk Exchange module to render the plugin in the UI, as well as enabling the Application Risk Exchange module to know more about the plugin, including required configuration parameters, the plugin-id, the plugin-name, etc.

• Every plugin must contain this file with the required information so that Application Risk Exchange can instantiate the Plugin object properly.
• Common parameters for manifest.json include
  • name: (string) Name of the plugin. (Required)
  • description: (string) Description of the plugin. Provide a detailed description that mentions the capabilities and instructions to use the plugin, (ex. Fetches users and user scores and performs actions for members in group/groups). This description would appear on the Plugin Configuration card. (Required)
  • ID: (string) ID of the plugin package. Make sure it is unique across all the plugins installed in the Cloud Exchange. The ID has to match the directory name of the plugin package. (Required
  • version: (string) Version of the plugin. Usage of a MAJOR.MINOR.PATCH (ex. 1.0.1) versioning scheme is encouraged, although there are no restrictions. (Required)
  • configuration: (array) Array of JSON objects that contains information about all the parameters required by the plugin – their name, type, id, etc. The common parameters for the nested JSON objects are explained below.
    • label: Name of the parameter. This will be displayed on the plugin configuration page. (Required)
    • key: Unique parameter key, which will be used as a key in the python dict object where the plugin configuration is used. (Required)
    • type: Value type of the parameter. Allowed values are ‘text’, ‘password’, ‘number’, ‘choice’, and ‘multichoice’. (Required) Refer to Plugin Configuration parameter types below for more details.
    • default: The default value for this parameter. This value will appear in the plugin configuration page on ARE UI. Supported data-types are “text”, “number”, and “list” (for multichoice type). (Required)
    • mandatory: Boolean which indicates whether this parameter is mandatory or not. If a parameter is mandatory ARE UI won’t let you pass an empty value for the parameter. Allowed values are `true` and `false`. (Required)
    • description: Help text level description for the parameter which can give more details about the parameter and expected value. This string will appear in the plugin configuration page as a help-text. (Required)
    • choices: A list of JSON objects containing key and value as JSON keys. This parameter is only supported by ‘type’: ‘choice and multichoice`.

Plugin Configuration Parameter Types

Make sure all the required plugin configuration parameters are listed under the configuration section of manifest.json for the plugin.

Password Parameter

Use this parameter for storing any secrets/passwords for authentication with API endpoints. Parameters with type as the password will have a password text box in the Plugin configuration page and will be obfuscated and encrypted by the platform.
Sample JSON

       "configuration": [
           {
               "label": "API Token",
               "key": "api_token",
               "type": "password"
           },
       ]

Plugin configuration view:

Text Parameter

Use this parameter for storing any string information such as base-url, username, etc. This parameter will have a normal text input on the plugin configuration page.
Sample JSON

       "configuration": [
           {
               "label": "Tenant Name",
               "key": "user_name",
               "type": "text"
           },
       ]

Plugin Configuration View

Number Parameter

Use this parameter for storing number/float values. This parameter will have a number input box on the plugin configuration page. (Note that the hash list is in the Threat Exchange module.)
Sample JSON

"configuration": [
{
      "label": "Maximum File hash list size in MB.",
      "key": "max_size",
      "type": "number"
      },
 ]

Plugin Configuration View

Choice Parameter

Use this parameter for storing any enumeration parameter values. This parameter will have a dropdown box on the plugin configuration page.
Sample JSON

	"configuration": [
        {
            "label": "Base URL",
            "key": "base_url",
            "type": "choice",
            "choices": [
                {
                    "key": "Commercial cloud (api.crowdstrike.com)",
                    "value": "https://api.crowdstrike.com"
                },
                {
                    "key": "US 2 (api.us-2.crowdstrike.com)",
                    "value": "https://api.us-2.crowdstrike.com"
                },
                {
                    "key": "Falcon on GovCloud (api.laggar.gcw.crowdstrike.com)",
                    "value": "https://api.laggar.gcw.crowdstrike.com"
                },
                {
                    "key": "EU cloud (api.eu-1.crowdstrike.com)",
                    "value": "https://api.eu-1.crowdstrike.com"
                }
            ],
            "default": "https://api.crowdstrike.com",
            "mandatory": true,
            "description": "API Base URL."
        },

Plugin Configuration View

After selecting the input:

Multichoice Parameter

Use this parameter for storing multiple choice values. This parameter will have a dropdown box on the plugin configuration page with ability to select multiple values. (Note that Severity is in the Threat Exchange module.)
Sample JSON

    "configuration": [
        {
            "label": "Severity",
            "key": "severity",
            "type": "multichoice",
            "choices": [
                {
                    "key": "Unknown",
                    "value": "unknown"
                },
                {
                    "key": "Low",
                    "value": "low"
                },
                {
                    "key": "Medium",
                    "value": "medium"
                },
                {
                    "key": "High",
                    "value": "high"
                },
                {
                    "key": "Critical",
                    "value": "critical"
                }
            ],
            "default": [
                "critical",
                "high",
                "medium",
                "low",
                "unknown"
            ],
            "mandatory": false,
            "description": "Only indicators with matching severity will be saved."
        }
    ]

Plugin Configuration View

Toggle Parameter

This parameter stores a boolean value, toggle enabled is True and toggle disabled is False.
Use System Proxy (‘proxy’):

Use system proxy configured in Settings.(Default: False)

Plugin Configuration View

main.py

This python file contains the core implementation of the plugin.
Standard Imports

from netskope.integrations.grc.plugin_base import (
    PluginBase,
    ValidationResult,
    PushResult,
)
from netskope.integrations.grc.models.configuration import (
    TargetMappingFields,
    MappingType,
)

PluginBase Variables
PluginBase provides access to variables that can be used during the plugin lifecycle Methods. Here is the list of variables.

Variable Name	Usage	Description
self.logger	self.logger.error(“Message”) self.logger.warn(“Message”) self.logger.info(“Message”)	Logger handle provided by core. Use this object to log important events. The logs would be visible in Cloud Exchange Audit logs. Refer to the Logger Object documentation.
self.configuration	self.configuration.get(<attribute-key-name>)	JSON representation of the configuration object of a Plugin instance. Use this to access the configuration attributes like authentication credentials, server details, etc. Use the key name of the attribute mentioned in manifest.json.
self.last_run_at	If self.last_run_at: self.last_run_at.timestamp() Use this format to convert the last run time in epoch format.	Provides the timestamp of the last successful run time of the Plugin’s pull method. The Cloud Exchange core maintains the checkpoint time after each successful pull() execution. For the first execution, the value would be None. The datatype of the object is datetime.
self.storage		Cloud Exchange provides the plugin a mechanism to maintain state. Use this object to persist any state that would be required during subsequent calls. The datatype of this object is python dict.
self.notifier	self.notifier.info(“message”) self.notifier.warn(“message”) self.notifier.error(“message”)	This object provides the handle of the Cloud Exchange core’s notifier. Use this object to push any notification to the platform. The notifications would be visible in the Application Risk Exchange UI. Make sure the message contains summarized information for the user to read and take necessary actions. Used notifier in Netskope plugin if the push() method exceeds the 8MB limit of the product.
self.proxy	requests.get(url=url, proxies=self.proxy)	Handle of system’s proxy settings if configured, else {}.
self.ssl_validation	requests.get(url=url, verify=self.ssl_validation)	Boolean value which mentions if ssl validation be enforced for REST API calls.

Plugin Class

• The Plugin class has to be inherited from the PluginBase class. The PluginBase class is defined in netskope.integrations.grc.plugin_base.
• Make sure the Plugin class provides an implementation for the methods validate, get_target_fields,and push.
• The plugin class will contain all the necessary parameters to establish connection and authentication with the 3rd party API.

"""Sample plugin implementation.

This is a sample implementation of base PluginBase class. Which explains the concrete implementation
of the base class.
"""
from netskope.integrations.grc.plugin_base import (
    PluginBase,
    ValidationResult,
    PushResult,
)
from netskope.common.utils import add_user_agent
from netskope.integrations.grc.models.configuration import (
    TargetMappingFields,
    MappingType,
)

from typing import List
from datetime import datetime
import requests

PLUGIN_NAME = “<module> <plugin_name> Plugin”

class SamplePlugin(PluginBase):
   """SamplePlugin class having concrete implementation for fetching information and performing actions.

   This class is responsible for implementing pull, perform actions and validate methods with proper return types, so that its lifecycle execution can be scheduled by the ARE core engine.
   """

Def get_target_fields()

This is an abstract method of PluginBase Class.

• This method should return the list of fields to be rendered in the UI when a target is selected from the dropdown.
• This method should be called after the user successfully validates the configuration parameters and selects mapping information while configuring the plugin.

    def get_target_fields(self, plugin_id, plugin_parameters):
        """Get available Target fields."""
        return [
            TargetMappingFields(
                label="Company Name",
                type=MappingType.STRING,
                value="name",
            ),
            TargetMappingFields(
                label="Company Legal Name",
                type=MappingType.STRING,
                value="company_legal_name",
            ),
   ]

Def validate()

This is an abstract method of PluginBase Class.

• This method validates the plugin configuration and authentication parameters passed while creating a plugin configuration.
• This method will be called only when a new configuration is created or updated.
• Validate against all the mandatory parameters that are passed with the proper data type.
• Validate the authentication parameters and the API endpoint to ensure the smooth execution of the plugin lifecycle.
• Return the object of ValidationResult (Refer ValidationResult Data Model) with a success flag indicating validation success or failure and the validation message containing the validation failure reason.

 def validate(self, data):      
       ""Validate the Plugin configuration parameters.

       Validation for all the parameters mentioned in the manifest.json for the existence and
       data type. Method returns the netskope.integrations.grc.plugin_base.ValidationResult object with success = True in the case of successful validation and success = False and an error message in the case of failure.
       Args:
           data (dict): Dict object having all the Plugin configuration parameters.
       Returns:
           netskope.integrations.grc.plugin_base.ValidateResult: ValidateResult object with success flag and message.
       """
       self.logger.info(f”PLUGIN_NAME}: Executing validate method for Sample plugin")
       if (
           "secret_field_id1" not in data
           or not data["secret_field_id1"]
           or type(data["secret_field_id1"]) != str
       ):
           self.logger.error(
               f"{PLUGIN_NAME}: Validation error occurred Error: Secret Field1 is required with type string."
           )
           return ValidationResult(
               success=False, message="Invalid Secret Field 1 provided.",

Def push

• This method implements the logic to push the application details to the product API endpoints or Socket connections.
• It receives all the data and then it pushes. You can use any kind of strategy to push data. Be it socket connection or REST API.
• This method will be invoked when the Cloud Exchange platform receives application data to it.
• Make sure to handle the case when the maximum payload size supported by the API endpoint is exceeded. There can be multiple ways to handle this case.
• If the API endpoint supports multiple requests with a fixed payload size, send the data in chunks.
• If the API endpoint does not support multiple requests (i.e. we can push it in one API call only) either plugin can skip the remaining logs and raise a notification to the user to adjust the sharing filters, or it can fail with the error of exceeded payload size.
• Handle all the Exceptions with connection and HTTP response code and raise exceptions with error messages along with the loggers in case errors are encountered.
• User Agent should be added to the headers while making any API call. Format for the User Agent: netskope-ce-<ce_version>-<module>-<plugin_name>-<plugin_version>.
• In this method, ‘proxy’ variable for ‘Use System Proxy’’ toggle should be used while making an API call.
• Return the PushResult object (Refer PushResult Data Model) with a success flag indicating whether the Push operation was successful or not.

Data Models

This section lists down the Data Models and their properties.

Application Model

• This model contains 15 fields: applicationId, applicationName, vendor, cci, ccl, categoryName, deepLink, users, customTags, discoveryDomains, steeringDomains, createdTime, updatedTime, firstSeen, and lastSeen.
• You will interact with the model in the push method as you push application details to a third party.

Data Model Properties

Name	Type	Description
applicationId	int	Unique Application ID.
applicationName	String	Name of the application.
vendor	String	Value of the Field.
cci	int	Can be any positive integer value.
ccl	String	Can be one of the below values [poor, low, medium, high, excellent, unknown].
categoryName	String	Name of the category.
deepLink	String	Link to third party page where shared application details can be viewed.
users	Listr[string]	List of users.
customTags	Listr[string]	List of custom tags.
discoveryDomains	Listr[string]	List of discovery domains.
steeringDomains	Listr[string]	List of steering domains.
createdTime	datetime	Date and time of creation.
updatedTime	datetime	Date and time of update.
firstSeen	datetime	Date and time of first seen.
lastSeen	datetime	Date and time of last seen.

TargetMappingFields Model

• This model contains 3 fields: label, type, and value.
• You will interact with the model in the get_target_fields method as you return a list of Target. Fields

Data Model Properties

Name	Type	Description
label	string	Label of the field which is displayed in the UI.
type	MappingType	Data Type of Field.
value	string	Value of the Field.

from netskope.integrations.grc.models import Application
Application(
    applicationId=0,
    applicationName="string",
    vendor="string",
    cci=100,
    ccl="poor",
    categoryName="string",
    deepLink="",
    source="string",
    createdTime="2023-03-20T06:02:15.893Z",
    updatedTime="2023-03-20T06:02:15.893Z",
    users=[],
    customTags=[],
    discoveryDomains=[],
    steeringDomains=[],
    firstSeen="2023-03-20T06:02:15.893Z",
    lastSeen="2023-03-20T06:02:15.893Z",
)

MappingType

This is an enum consisting of 3 values as list, string, integer.
Data Model Properties

Name	Type	Description
list	List	Can be a list of string or integer.
string	string	Can be a string.
integer	Integer	Can be a positive integer.

PushResult Class

• This class contains the result of Push operation of application details to the product API endpoint.
• The success flag indicates the push operation result, and the message flag should have a proper error message in the case of failure. (In a case of success, a simple success message with success flag True should be returned)

Data Model Properties

Name	Type	Description
success	bool	The success flag indicates the result of a push operation, whether it succeeded or failed.
message	string	Message field denotes the error in case of failure. In case of success, it can be a simple success message.

from netskope.integrations.grc.plugin_base import PushResult

PushResult(
    success=True, 
    message="Successfully pushed data to 3rd party."
)

ValidationResult Class

• This class contains the result of the validation process on the plugin configuration parameters passed to the Plugin object while creating a new configuration for the plugin.
• Make sure that all the parameters passed to the validate method are validated against the data-type and value.
• Validate method returns the object of this class with a success flag indicating the result of validation operation and message field containing the proper error message in the case of validation failure. (In a case of success a simple success message with success flag True has to be returned)
• Note that Validate Action also returns the object of this class.

Data Model Properties

Name	Type	Description
success	bool	success flag indicates the result of validation operation, whether it succeeded or failed.
message	string	Message field denotes the error in case of failure. In case of success it can be a simple success message.

from netskope.integrations.cre.plugin_base import ValidationResult

ValidationResult(
           success=True,
           message="Validation Successful for Sample plugin"
       )

Logging

Application Risk Exchange provides a handle of a logger object for logging.

• Avoid print statements in the code.
• This object logs to the central Cloud Exchange database with the timestamp field. Supported log levels are info, warn, and error.
• Make sure any API authentication secret or any sensitive information is not exposed in the log messages.
• Make sure to implement a proper logging mechanism with the logger object passed by the Cloud Exchange platform.
• Make sure enough logging is done, which helps the Ops team in troubleshooting.
• Make sure any sensitive data is not logged or leaked in the notification or logs.

 self.logger.error(
               f"{PLUGIN_NAME}: Error log-message goes here."
 )

 self.logger.warn(
               f"{PLUGIN_NAME}: Warning log-message goes here."
 )

 self.logger.info(
               f"{PLUGIN_NAME}: Info log-message goes here."
 )

Notifications

Application Risk Exchange provides a handle of notification objects, which can be used to generate notifications in the Application Risk Exchange UI.

• This object is passed from the Cloud Exchange platform to the Plugin object. Every plugin integration can use this object whenever there is a case of failure which has to be notified to the user immediately. The notification timestamp is managed by the Cloud Exchange platform.
• This object will raise the notification in the UI with a color-coding for the severity of the failure. Supported notification severities are info, warn and error.
• Make sure any API authentication secret or any sensitive information is not exposed in the notification messages.
• Use a notifier object to raise the notification for failures or critical situations (like rate-limiting, or exceeding payload size) to notify the status of the plugin to the user.

 self.notifier.info(
               f"{PLUGIN_NAME}: Info notification-message goes here."
 )

 self.notifier.error(
               f"{PLUGIN_NAME}: Error notification-message goes here."
 )

 self.notifier.warn(
               f"{PLUGIN_NAME}: Warning notification-message goes here."
 )

Testing

Linting

As part of the build process, we run a few linters to catch common programming errors, stylistic errors, and possible security issues.

Flake8

This is a basic linter. It can be run without having all the dependencies available and will catch common errors. We also use this linter to enforce the standard python pep8 formatting style. On rare occasions, you may encounter a need to disable an error/warning returned from this linter. Do this by adding an inline comment of the sort on the line you want to disable the error:
# noqa: <error-id>
For example:
example = lambda: ‘example’ # noqa: E731
When adding an inline comment always also include the error code you are disabling for. That way if there are other errors on the same line they will be reported.
For more info, go to: https://flake8.pycqa.org/en/latest/user/violations.html#in-line-ignoring-errors
PEP8 style docstring check is also enabled with the flake8 linter. So make sure every function/module has a proper docstring added.

Unit Testing

Ensure unit testing to test small units of code in an isolated and deterministic fashion. Make sure that unit tests avoid performing communication with external APIs and use mocking. Make sure unit tests ensure the code coverage is more than 70%.

Environment Setup

In order to work with unit testing the integration or automation script needs to be developed in Plugin Directory Structure,We use PIP to install all the required Python module dependencies required to run the setup. Before running the tests, make sure you install all the required dependencies mentioned in the requirements.txt of the Application Risk Exchange core repository.

Write Your Unit Tests

Make sure unit tests are written in a separate Python file named: <your_plugin_name>_test.py. Within the unit test file, each unit test function should be named: test_<your test case>. More information on writing unit tests and their format is available at the PyTest Docs.

Mocking

We use pytest-mock for mocking. pytest-mock is enabled by default and installed in the base environment mentioned above. To use a mocker object simply pass it as a parameter to your test function. The mocker can then be used to mock both the plugin class object and also external APIs.
Example:

ef test_netskope_push(mocker):
   mocker.patch("grc.plugins.sample.main.SamplePlugin")
   Push_return_result = PushResult(
            success=True,
            message="Successfully pushed data to third_party.",
        )


   samplePlugin.push.return_value = Push_return_result 
   sp = samplePlugin(None, None, None, logger)
   application = [Application(objectId="64116b18969c9579c080c5ac",
    applicationId=0,
    applicationName="Postman",
    vendor="Nexus Venture Partners",
    cci=100,
    ccl="poor",
    categoryName="Desktop",
    deepLink="",
    createdTime="2023-03-15T06:52:08.374000",
    updatedTime="2023-03-15T06:52:08.374000",
    source="netskope",
    users=[],
    customTags=[],
    discoveryDomains=[],
    steeringDomains=[],
    firstSeen="2023-03-15T06:52:08.374000",
    lastSeen="2023-03-15T06:52:08.374000"
)]
   mappings = {
    "jsonQuery": {
        "and": [
            {
                "==": [
                    {
                        "var": "name"
                    },
                    "applicationName"
                ]
            }
        ]
    },
    "query": "name == \"applicationName\""
}

   actual_push = sp.push(application,mappings)
   assert push_return_result == actual_push

To mock request module response for API calls (requests_mock Pytest plugin is installed with all the dependencies).

def test_fetch_users(requests_mock):
   endpoint_url = "https://example-api.com"
   mock_response_json = {
       {
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#directoryObjects",
    "value": [
        {
            "@odata.type": "#microsoft.graph.user",
            "id": "48a97c4b-65af-4cb7-90f9-c7835d5ecd8b",
            "userPrincipalName": "adaml@xyz.com"
        }
    ]
}


   requests_mock.get(endpoint_url, json=mock_response_json)
   config_dict = {
       "tenant_name": "ecre-ccec-jpvasq",
       "client_id": "ue4y-3id8-mnx87q",
       "client_secret": "d9oe-ple7-nfi34w",
       "api_token": "token",
   }
   sp = SamplePlugin(config_dict, None, None, logger)
   actual_users_lists = sp.fetch_users(
       endpoint_url,
       config_dict['api_token']
   )
   users_mock = [
  	Record(uid="a@def.com", type=Record.user, score=400),
       Record(uid="b@pqr.com", type=Record.user, score=300),
       Record(uid="c@xyz.com", type=Record.user, score=750),
   ]
   assert len(actual_users_lists) == len(users_mock)
   for i in range(len(actual_users_lists)):
       assert actual_ind_lists[i].value == users_mock[i].value

Running Your Unit Tests

$ PYTHONPATH=. pytest

Plugin Deployment on Cloud Exchange

Package the Plugin

Cloud Exchange expects the developed plugin in zip and tar.gz format.
Execute this command to zip the package:

zip -r sample_plugin.zip sample_plugin

Execute this command to generate tar.gz package:

tar -zcvf sample_plugin.tar.gz sample_plugin

Upload the Plugin

To deploy this zip or tar.gz on the Cloud Exchange platform, follow these steps:

1. Log in to the Cloud Exchange platform.
2. Go to Settings > Plugin.
   
3. Click Add New Plugin.
   
4. Click Browse.
5. Select your zip or tar.gz file.
   
6. Click Upload.

Add a Repo

To deploy your plugin on the Cloud Exchange platform, you can add your repo in Cloud Exchange platform by following these steps:

1. Log in to the Cloud Exchange platform.
2. Go to Settings > Plugin Repository.
   
3. Click Configure New Repository.
   
4. Provide a Repository name, Repository URL, Username, and Personal Access Token.
5. Click Save.
6. Go to Settings > Plugins.
7. Select a Repository name from the Repository dropdown.

Deliverables

Plugin Guide

• Make sure to update the plugin guide with every release.
• Plugin guide should have this content:
  • Compatibility
  • Release Notes
  • Description
  • Prerequisites
  • Permissions
  • Authorization of the product
  • Configuration of Netskope Application Risk Exchange plugin
  • Configuration of the third-party Application Risk Exchange plugin that we have developed.
  • Configuration of business rule
  • Configuration of add Sharing configuration
  • Validation

Demo Video

After the successful development of Application Risk Exchange plugin, create a demo video and show the end-to-end workflow of the plugin.