Migrate Cloud Exchange as a VM - Netskope Technical Documentation

Use these steps to migrate Cloud Exchange as a VM, standalone or HA, to the latest version. For the supported migration paths and the other deployment type, refer to Migrate Cloud Exchange.

Prerequisites

Before you start the migration, make sure your instance meets the Cloud Exchange System Requirements.
Make sure the VM can connect to https://cloud-exchange-store.s3.us-east-1.amazonaws.com.
Disable all source plugins in all modules, and wait for the queued tasks to finish:
1. Wait 20 to 30 minutes after you disable the source plugins.
2. Go to Settings > Logging, click Filter Query, enter this filter in the Filters field, and click Load.
```
message Like "Ingested " || message Like "Stored " || message Like "task(s)" || message Like "Completed storing"
```
3. Monitor the filtered logs. When no new logs appear, continue with the migration.
4. After the migration is complete, enable the source plugins that you disabled.
Keep the maintenance password ready. You need it to complete some of the steps.

Important

If you lose the maintenance password, your existing data cannot be retained.

RabbitMQ data migration is not supported, because the queue type changed from Classic to Quorum.

If you run Cloud Exchange as a VM (OVA or Hyper-V) on a version earlier than v5.1.1, you need the root credentials of the VM.
Username: root
Password: M5#w6V+.T^8gv?%,

If you run Cloud Exchange as a VM on Azure or AWS, you can complete these steps yourself.

Standalone Deployment

To v7.0.0 from v5.1.1, v5.1.2, v6.0.0, v6.0.1, v6.1.0 Standalone Deployment

The migrate_ce script migrates your deployment to the latest Cloud Exchange standalone deployment. Run the script on the new (destination) VM. The script needs the password of the cteadmin user on the old VM.

On the new machine, go to the /opt/cloudexchange/cloudexchange directory. Copy the example configuration file, and open it for editing:
```
cd /opt/cloudexchange/cloudexchange
cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config
```
In cloudexchange.config, set the values of these fields. Also set any optional fields that you use or customized, such as the proxy configuration and UI port.
```
MAINTENANCE_PASSWORD=<old cloud exchange maintenance password>
JWT_SECRET=<old cloud exchange JWT secret>
```
JWT_SECRET is optional. If you leave it empty, the setup script generates a new JWT secret.
On the new machine, run the migration script:
```
sudo ./migrate_ce
```
When prompted, enter the username and IP address of the old machine, and its password or the path to its PEM file.
Wait for the script to finish. After it connects to the old machine, the script:
- Runs the stop script on the old machine.
- Zips the existing data on the old machine, and transfers it to the new machine.
- Unzips the data into the installation folder on the new machine.
- Runs the setup script, which asks for the inputs it needs to start the latest Cloud Exchange.
- Runs the start script.
Check the status of the containers. If you use Podman, run sudo podman ps instead.
```
sudo docker ps
```
Open the UI of the migrated Cloud Exchange in your browser. Log in with your existing Cloud Exchange credentials, and check that your configurations and data are present and working.

HA Deployment

To v7.0.0 from v5.1.1, v5.1.2, v6.0.0, v6.0.1, v6.1.0 Standalone Deployment

Before you start, complete the Prerequisites and take a backup. For details, refer to Backup Cloud Exchange.

To move from a standalone deployment to the latest Cloud Exchange as a VM HA deployment, copy the data from the old standalone VM to the node that becomes the new primary node.

For details about the HA parameters and how to set up the cluster, refer to Cloud Exchange High Availability. Make sure the /opt/shared/data directory is available, so that Cloud Exchange can store the resources that the cluster nodes share.

On the Old Standalone VM

Go to the /opt/cloudexchange/cloudexchange directory:
```
cd /opt/cloudexchange/cloudexchange
```
Stop Cloud Exchange:
```
sudo ./stop
```
Go to the data directory:
```
cd data
```

Create a zip file of the data:

sudo zip -r ce_backup.zip mongo-data/ repos/ plugins/

If you use custom plugins, add them to the zip file:
```
sudo zip -r ce_backup.zip custom_plugins
```

Copy the zip file to the new primary node:

sudo scp -i <public_key> ce_backup.zip <username>@<public_ip>:/opt/cloudexchange/cloudexchange/data

On the New Primary Node

Run these steps on the node that you want to make the primary node.

Go to the data directory:

cd /opt/cloudexchange/cloudexchange/data

Extract the data:

sudo unzip ce_backup.zip "mongo-data/*" -d .
sudo mkdir /opt/shared/data/ -p
sudo unzip ce_backup.zip "plugins/*" "repos/*" "custom_plugins/*" -d /opt/shared/data/

Go back to the original working directory:
```
cd ..
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields. Also set any optional fields that you use or customized, such as the proxy configuration and UI port.
```
MAINTENANCE_PASSWORD=<old cloud exchange maintenance password>
JWT_SECRET=<old cloud exchange JWT secret>
```
JWT_SECRET is optional. If you leave it empty, the setup script generates a new JWT secret.
Run the setup script. Run it on the primary node first.
```
sudo python3 ./setup
```
Start Cloud Exchange:
```
sudo ./start
```
Open the UI of the primary node in your browser at https://<ip>:<port>, and log in with your existing Cloud Exchange credentials.

On the Secondary Nodes

Complete steps 1 to 6 on each secondary node. Then add the secondary nodes from the UI of the primary node.

Go to the /opt/cloudexchange/cloudexchange directory:
```
cd /opt/cloudexchange/cloudexchange
```

On the primary node, copy the contents of the CA key file:

/opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key

On the secondary node, create the CA key file, paste the key that you copied, and save the file. The matching CA key lets the secondary node communicate with the HA cluster.
```
sudo vi /opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields:

Important

The JWT secret and the CA key of a secondary node must match the primary node. If they differ, adding the secondary node to the HA cluster fails with an authentication error. If you don’t know the JWT secret of the primary node, generate and apply a new one. For the steps, refer to Cloud Exchange Troubleshooting.
```
JWT_SECRET=<JWT_SECRET value of the primary node>
MAINTENANCE_PASSWORD=<old maintenance password>
```
Run the setup script:
```
sudo python3 ./setup
```
In the UI of the primary node, go to Settings > General Settings > Node Configurations and enable the Enable HA toggle. In the Confirm Action pop-up, enter the Cloud Exchange IP address or FQDN only if you need to change it, and click Enable HA.

Important

While HA is being enabled, Cloud Exchange is temporarily unavailable and may restart several times.
In the UI of the primary node, go to Settings > General Settings > Node Configurations and add the secondary nodes one at a time.

You don’t need to run the start script on the secondary nodes. It runs automatically when you add a secondary node from the UI of the primary node.

To v7.0.0 from v5.1.1, v5.1.2, v6.0.0, v6.0.1, v6.1.0 HA Deployment

Before you start, complete the Prerequisites and take a backup. For details, refer to Backup Cloud Exchange.

To move from the current Cloud Exchange as a VM HA deployment to the latest one, copy the data from the current primary node to the new primary node.

On the Current HA Deployment

Stop the secondary nodes first, and then the primary node, so that Cloud Exchange shuts down properly:
```
cd /opt/cloudexchange/cloudexchange
sudo ./stop
```
On the primary node, go to the data directory, and create a zip file of the MongoDB data:
```
cd /opt/cloudexchange/cloudexchange/data
sudo zip -r ce_backup.zip mongo-data/
```
Go to the shared drive, and add the repos and plugins folders to the zip file. For v6.0.0 and later, the shared drive path is /opt/shared/data/.
```
cd <shared_drive>
sudo zip -r <ce_backup_zip_location>/ce_backup.zip repos/ plugins/
```

If you use custom plugins, add them to the zip file:

sudo zip -r <ce_backup_zip_location>/ce_backup.zip custom_plugins/

Copy the zip file from the current primary node to the new primary node, which has the latest Cloud Exchange version:

sudo scp -i <public_key> <ce_backup_zip_location>/ce_backup.zip <username>@<public_ip>:/opt/cloudexchange/cloudexchange/data

On the New Primary Node

Go to the data directory:

cd /opt/cloudexchange/cloudexchange/data

Extract the data:

sudo unzip ce_backup.zip "mongo-data/*" -d .
sudo mkdir /opt/shared/data/ -p
sudo unzip ce_backup.zip "plugins/*" "repos/*" "custom_plugins/*" -d /opt/shared/data/

Go back to the original working directory:
```
cd ..
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields. Also set any optional fields that you use or customized, such as the proxy configuration and UI port.

HA_ENABLED=True
HA_CURRENT_NODE=<current node ip>
HA_PRIMARY_NODE_IP=<current node ip>
HA_IP_LIST=<current node ip>
MAINTENANCE_PASSWORD=<old maintenance password>

Run the setup script:
```
sudo python3 ./setup
```
Migrate the MongoDB data. Run this command only once.
```
sudo ./restore_ha_backup
```
Start Cloud Exchange:
```
sudo ./start
```

On the Secondary Nodes

Complete steps 1 to 6 on each secondary node. Then add the secondary nodes from the UI of the primary node.

Go to the /opt/cloudexchange/cloudexchange directory:
```
cd /opt/cloudexchange/cloudexchange
```

On the primary node, copy the contents of the CA key file:

/opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key

On the secondary node, create the CA key file, paste the key that you copied, and save the file. The matching CA key lets the secondary node communicate with the HA cluster.
```
sudo vi /opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields:

Important

The JWT secret and the CA key of a secondary node must match the primary node. If they differ, adding the secondary node to the HA cluster fails with an authentication error. If you don’t know the JWT secret of the primary node, generate and apply a new one. For the steps, refer to Cloud Exchange Troubleshooting.
```
JWT_SECRET=<JWT_SECRET value of the primary node>
MAINTENANCE_PASSWORD=<old maintenance password>
```
Run the setup script:
```
sudo python3 ./setup
```
In the UI of the primary node, go to Settings > General Settings > Node Configurations and add the secondary nodes one at a time.

You don’t need to run the start script on the secondary nodes. It runs automatically when you add a secondary node from the UI of the primary node.

Prerequisites

Before you start the migration, make sure your instance meets the Cloud Exchange System Requirements.

Make sure the VM can connect to https://cloud-exchange-store.s3.us-east-1.amazonaws.com.

Disable all source plugins in all modules, and wait for the queued tasks to finish:

Wait 20 to 30 minutes after you disable the source plugins.

Go to Settings > Logging, click Filter Query, enter this filter in the Filters field, and click Load.

message Like "Ingested " || message Like "Stored " || message Like "task(s)" || message Like "Completed storing"

Monitor the filtered logs. When no new logs appear, continue with the migration.
After the migration is complete, enable the source plugins that you disabled.

Keep the maintenance password ready. You need it to complete some of the steps.

Important

If you lose the maintenance password, your existing data cannot be retained.

RabbitMQ data migration is not supported, because the queue type changed from Classic to Quorum.

If you run Cloud Exchange as a VM (OVA or Hyper-V) on a version earlier than v5.1.1, you need the root credentials of the VM.
Username: root
Password: M5#w6V+.T^8gv?%,

If you run Cloud Exchange as a VM on Azure or AWS, you can complete these steps yourself.

Standalone Deployment

To v7.0.0 from v5.1.1, v5.1.2, v6.0.0, v6.0.1, v6.1.0 Standalone Deployment

On the new machine, go to the /opt/cloudexchange/cloudexchange directory. Copy the example configuration file, and open it for editing:
```
cd /opt/cloudexchange/cloudexchange
cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config
```
In cloudexchange.config, set the values of these fields. Also set any optional fields that you use or customized, such as the proxy configuration and UI port.
```
MAINTENANCE_PASSWORD=<old cloud exchange maintenance password>
JWT_SECRET=<old cloud exchange JWT secret>
```
JWT_SECRET is optional. If you leave it empty, the setup script generates a new JWT secret.
On the new machine, run the migration script:
```
sudo ./migrate_ce
```
When prompted, enter the username and IP address of the old machine, and its password or the path to its PEM file.
Wait for the script to finish. After it connects to the old machine, the script:
- Runs the stop script on the old machine.
- Zips the existing data on the old machine, and transfers it to the new machine.
- Unzips the data into the installation folder on the new machine.
- Runs the setup script, which asks for the inputs it needs to start the latest Cloud Exchange.
- Runs the start script.
Check the status of the containers. If you use Podman, run sudo podman ps instead.
```
sudo docker ps
```
Open the UI of the migrated Cloud Exchange in your browser. Log in with your existing Cloud Exchange credentials, and check that your configurations and data are present and working.

HA Deployment

To v7.0.0 from v5.1.1, v5.1.2, v6.0.0, v6.0.1, v6.1.0 Standalone Deployment

Before you start, complete the Prerequisites and take a backup. For details, refer to Backup Cloud Exchange.

To move from a standalone deployment to the latest Cloud Exchange as a VM HA deployment, copy the data from the old standalone VM to the node that becomes the new primary node.

On the Old Standalone VM

Go to the /opt/cloudexchange/cloudexchange directory:
```
cd /opt/cloudexchange/cloudexchange
```
Stop Cloud Exchange:
```
sudo ./stop
```
Go to the data directory:
```
cd data
```

Create a zip file of the data:

sudo zip -r ce_backup.zip mongo-data/ repos/ plugins/

If you use custom plugins, add them to the zip file:
```
sudo zip -r ce_backup.zip custom_plugins
```

Copy the zip file to the new primary node:

sudo scp -i <public_key> ce_backup.zip <username>@<public_ip>:/opt/cloudexchange/cloudexchange/data

On the New Primary Node

Run these steps on the node that you want to make the primary node.

Go to the data directory:

cd /opt/cloudexchange/cloudexchange/data

Extract the data:

sudo unzip ce_backup.zip "mongo-data/*" -d .
sudo mkdir /opt/shared/data/ -p
sudo unzip ce_backup.zip "plugins/*" "repos/*" "custom_plugins/*" -d /opt/shared/data/

Go back to the original working directory:
```
cd ..
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields. Also set any optional fields that you use or customized, such as the proxy configuration and UI port.
```
MAINTENANCE_PASSWORD=<old cloud exchange maintenance password>
JWT_SECRET=<old cloud exchange JWT secret>
```
JWT_SECRET is optional. If you leave it empty, the setup script generates a new JWT secret.
Run the setup script. Run it on the primary node first.
```
sudo python3 ./setup
```
Start Cloud Exchange:
```
sudo ./start
```
Open the UI of the primary node in your browser at https://<ip>:<port>, and log in with your existing Cloud Exchange credentials.

On the Secondary Nodes

Complete steps 1 to 6 on each secondary node. Then add the secondary nodes from the UI of the primary node.

Go to the /opt/cloudexchange/cloudexchange directory:
```
cd /opt/cloudexchange/cloudexchange
```

On the primary node, copy the contents of the CA key file:

/opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key

On the secondary node, create the CA key file, paste the key that you copied, and save the file. The matching CA key lets the secondary node communicate with the HA cluster.
```
sudo vi /opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields:

Important

The JWT secret and the CA key of a secondary node must match the primary node. If they differ, adding the secondary node to the HA cluster fails with an authentication error. If you don’t know the JWT secret of the primary node, generate and apply a new one. For the steps, refer to Cloud Exchange Troubleshooting.
```
JWT_SECRET=<JWT_SECRET value of the primary node>
MAINTENANCE_PASSWORD=<old maintenance password>
```
Run the setup script:
```
sudo python3 ./setup
```
In the UI of the primary node, go to Settings > General Settings > Node Configurations and enable the Enable HA toggle. In the Confirm Action pop-up, enter the Cloud Exchange IP address or FQDN only if you need to change it, and click Enable HA.

Important

While HA is being enabled, Cloud Exchange is temporarily unavailable and may restart several times.
In the UI of the primary node, go to Settings > General Settings > Node Configurations and add the secondary nodes one at a time.

You don’t need to run the start script on the secondary nodes. It runs automatically when you add a secondary node from the UI of the primary node.

To v7.0.0 from v5.1.1, v5.1.2, v6.0.0, v6.0.1, v6.1.0 HA Deployment

Before you start, complete the Prerequisites and take a backup. For details, refer to Backup Cloud Exchange.

To move from the current Cloud Exchange as a VM HA deployment to the latest one, copy the data from the current primary node to the new primary node.

On the Current HA Deployment

Stop the secondary nodes first, and then the primary node, so that Cloud Exchange shuts down properly:
```
cd /opt/cloudexchange/cloudexchange
sudo ./stop
```
On the primary node, go to the data directory, and create a zip file of the MongoDB data:
```
cd /opt/cloudexchange/cloudexchange/data
sudo zip -r ce_backup.zip mongo-data/
```
Go to the shared drive, and add the repos and plugins folders to the zip file. For v6.0.0 and later, the shared drive path is /opt/shared/data/.
```
cd <shared_drive>
sudo zip -r <ce_backup_zip_location>/ce_backup.zip repos/ plugins/
```

If you use custom plugins, add them to the zip file:

sudo zip -r <ce_backup_zip_location>/ce_backup.zip custom_plugins/

Copy the zip file from the current primary node to the new primary node, which has the latest Cloud Exchange version:

sudo scp -i <public_key> <ce_backup_zip_location>/ce_backup.zip <username>@<public_ip>:/opt/cloudexchange/cloudexchange/data

On the New Primary Node

Go to the data directory:

cd /opt/cloudexchange/cloudexchange/data

Extract the data:

sudo unzip ce_backup.zip "mongo-data/*" -d .
sudo mkdir /opt/shared/data/ -p
sudo unzip ce_backup.zip "plugins/*" "repos/*" "custom_plugins/*" -d /opt/shared/data/

Go back to the original working directory:
```
cd ..
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields. Also set any optional fields that you use or customized, such as the proxy configuration and UI port.

HA_ENABLED=True
HA_CURRENT_NODE=<current node ip>
HA_PRIMARY_NODE_IP=<current node ip>
HA_IP_LIST=<current node ip>
MAINTENANCE_PASSWORD=<old maintenance password>

Run the setup script:
```
sudo python3 ./setup
```
Migrate the MongoDB data. Run this command only once.
```
sudo ./restore_ha_backup
```
Start Cloud Exchange:
```
sudo ./start
```

On the Secondary Nodes

Complete steps 1 to 6 on each secondary node. Then add the secondary nodes from the UI of the primary node.

Go to the /opt/cloudexchange/cloudexchange directory:
```
cd /opt/cloudexchange/cloudexchange
```

On the primary node, copy the contents of the CA key file:

/opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key

On the secondary node, create the CA key file, paste the key that you copied, and save the file. The matching CA key lets the secondary node communicate with the HA cluster.
```
sudo vi /opt/cloudexchange/cloudexchange/data/ssl_certs/mongodb_rabbitmq_certs/tls_cert_ca.key
```

Copy the example configuration file, and open it for editing:

cp cloudexchange.config.example cloudexchange.config
vi cloudexchange.config

In cloudexchange.config, set the values of these fields:

Important

The JWT secret and the CA key of a secondary node must match the primary node. If they differ, adding the secondary node to the HA cluster fails with an authentication error. If you don’t know the JWT secret of the primary node, generate and apply a new one. For the steps, refer to Cloud Exchange Troubleshooting.
```
JWT_SECRET=<JWT_SECRET value of the primary node>
MAINTENANCE_PASSWORD=<old maintenance password>
```
Run the setup script:
```
sudo python3 ./setup
```
In the UI of the primary node, go to Settings > General Settings > Node Configurations and add the secondary nodes one at a time.

You don’t need to run the start script on the secondary nodes. It runs automatically when you add a secondary node from the UI of the primary node.