Cloud Exchange KB Articles

Expand the sections to view these KB articles.

High Availability (HA) Setup Guide for Cloud Exchange on Linux Nodes with Azure Blob Storage

This section explains how to configure a High Availability (HA) environment for Cloud Exchange on Linux nodes using Azure Blob Storage as the NFS (Network File System) for shared storage.

Cloud Exchange System Requirements are present at:

Prerequisites

System Requirements for HA Nodes

16 CPU CORES (Compute Intensive F16s series)
32 GB RAM
120 GB Available free disk space where CE will be installed; for example: /home/root/
20 GB free space at /var/lib for container storages.

NFS Setup with Azure Blob Storage

Go to: https://learn.microsoft.com/en-us/azure/storage/files/storage-files-quick-create-use-linux#create-an-nfs-azure-file-share

Network Configuration

Linux machines will be treated as HA nodes, and all nodes along with Storage accounts should be in the same Virtual network/subnet and same Azure region/zone.After configuring NFS File shares, refer to the steps below for Mount the NFS share.

Configure Cloud Exchange High Availability (HA)Validate Prerequisites

Check Required Ports: Ensure the following ports are open and available on each node:

4369	RabbitMQ Communication Port
5672	AMQP Communication PORT
15672	RABBITMQ Administration Console
25672	RabbitMQ Internal Port
35672	Used FOR CLI Communications
27017	MongoDB Port
443	Service Health Check

Network Configuration: The Virtual Network should be one across all Nodes.
Validate RAM, Disk space, and CPU.
```
sudo df -h
sudo free -h
sudo nproc
```
Git package, curl, and zip package:
```
sudo yum install -y git curl zip
```
Install nfs-utils.
```
sudo yum install -y nfs-utils
```

Python3 along with pip.

sudo yum install -y python3 python3-pip python3-devel

Podman and podman-compose (Ref: https://podman.io/docs/installation#rhel8 )

sudo yum module enable -y container-tools:rhel8
sudo yum module install -y container-tools:rhel8
sudo yum install -y podman-plugins
sudo pip3 install podman-compose
sudo chmod +x /usr/local/bin/podman-compose
sudo ln -s /usr/local/bin/podman-compose /usr/bin/podman-compose

Check Podman and podman-compose version:

podman-compose —-version
podman –-version

Required python packages:

pip3 install "pyyaml>=6.0.0"
pip3 install "python-dotenv>=0.20.0,<=1.0.0"
pip3 install "pymongo>=4.1.1,<=4.3.3

Configure NFS In Azure Blob Storage

Go to: https://learn.microsoft.com/en-us/azure/storage/files/storage-files-quick-create-use-linux#create-an-nfs-azure-file-share

Mount NFS Storage in Each Nodes

Go to NFS File Share Overview.
Create a mounting directory as mentioned in mount Path (Also obtained from NFS FIle share overview guide) using this command:
```
sudo mkdir -p <mount_path>
```
Edit the /etc/fstab file and the mounted configuration as mentioned below:
cestorageha.file.core.windows.net:/cestorageha/shared-storage-nfs /mount/cestorageha/shared-storage-nfs nfs default 0 0
- cestorageha.file.core.windows.net:/cestorageha/shared-storage-nfs will be the source NFS directory present at the Azure Blob Storage NFS File share which will be obtained from sample command presents at Home > Respective Storage account > File shares > Newly created NFS File share > Overview.
- /mount/cestorageha/shared-storage-nfs will be the mount path and created directory.
- The other remaining part is NFS Configuration.
Run this command:
```
sudo mount -a
```

Download the Installation Package

On each node, download the Cloud Exchange installation package using this command:

git clone https://github.com/netskopeoss/ta_cloud_exchange

Run the Setup Script on the Primary Node

Go to the Cloud Exchange directory. On the primary node, run the setup script with this command:

sudo ./setup

Run the Setup Script in the Secondary Nodes

On each secondary node, go to the Cloud Exchange directory. Run the setup script using the following command, replacing <mounting_dir> with the path to your NFS mounting directory:

sudo ./setup --location <mounting_dir>

Enable the CSV index

On each node, edit the Podman Compose HA configuration file to enable CSV indexing:
```
$ vi podman-compose-ha.yml
```
Locate the core section and add the following environment variable to enable network event streaming:
```
ITERATOR_EVENT_NETWORK=stream_network_events_rollsroyce
```

Update the RabbitMQ and MongoDB Tags

RabbitMQ:
```
index.docker.io/crestsystems/cloudexchange:rabbitmq-5.0.1-csv-hotfixx
```
For each node, open the podman-compose-ha.yml file.
```
$ vi podman-compose-ha.yml
```
Update the RabbitMQ and MongoDB service image tags as follows:

MongoDB:

index.docker.io/crestsystems/cloudexchange:mongo-5.0.1-csv-hotfix

Update the Core and UI Tags

Go to NFS storage.
Edit the .env file:
```
$ vi config/.env
```

Update the CORE_TAG and UI_TAG variables as follows.

CORE_TAG=crestsystems/cloudexchange:core-5.0.1-csv-hotfix
UI_TAG=crestsystems/cloudexchange:ui-5.0.1-csv-hotfix

Backup `podman-compose-ha.yml`

On each node, create a backup of the podman-compose-ha.yml file:

$ cp podman-compose-ha.yml podman-compose-ha.yml.backup

Start Cloud Exchange on the Primary Node

Run the start script on the primary node:
```
$ sudo ./start
```
Wait for the Migration completed message before proceeding to the next step.

Start Cloud Exchange on Secondary Nodes

After you receive the Migration completed message in the primary node, then execute the start script in the remaining nodes.

$ sudo ./start

Upload and Configure the Azure Sentinel Plugin

Go to Settings > Plugin Repository in CE and click on upload plugin (⬆) button.
Select the Zip file which is shared through the Support Case and click on the upload button.
After the plugin is uploaded successfully, go to Settings > plugins to configure the uploaded plugin with the version 3.0.2.

Cloud Exchange Platform Logs Error Notification Setup for Slack

This section is designed to guide you through the process of setting up notifications in Slack for Cloud Exchange errors. This will allow customers to be promptly informed about any critical issues or errors encountered during the Cloud Exchange operations. Follow the steps below to configure notifications for various Cloud Exchange modules, ensuring you get notified whenever an error occurs.

Overview

To keep customers informed of critical errors in Cloud Exchange, we have implemented a Slack notifier plugin. This allows you to receive real-time alerts directly to your Slack channel whenever an error occurs in any of the Cloud Exchange modules. By setting up business rules tailored to specific modules, you can ensure that only relevant errors trigger notifications, making it easier to monitor and address issues quickly.

Notifier for Slack Configuration

If you are freshly installing the plugin, configure the Slack Notifier Plugin by following the steps outlined in the Notifier Plugin for Ticket Orchestrator document. Once done, refer to this document to configure the business rules. If you have already configured Notifier, you can directly follow this document to edit the existing business rules. Below is how to configure the business rule for each Cloud Exchange module:

Cloud Exchange Platform Errors

This section covers general Cloud Exchange Platform errors that are not tied to any specific module. These errors may include system-wide issues.