Manage a Publisher
Manage a Publisher
After deploying your publisher, use the following sections to make modifications to your Publishers.
Guidance for Installing 3rd-party Applications on Publishers
Be aware that any additional software (not included with the Netskope provided package) that is installed on a Publisher instance will be sharing resources with the Publisher application. In addition, Netskope and 3rd-party domains and Publisher software processes may need to be whitelisted in the 3rd-party software.
Network security software that monitors each and every action on the Publisher instance may cause performance issues. You can install security software as long as doing so does not impact Publisher performance. It is your responsibility to ensure that enough resources are allocated for all the software applications running on a Publisher instance.
To troubleshoot issues, Netskope Support may request you to test/recreate without 3rd-party software to narrow down the root cause.
Publisher Auto-updates provide a scalable means to update the Publisher software. The auto-update feature is not expected to trigger an update when there are pending underlaying operating system updates.
In high-availability Publisher deployments where two or more Publishers are assigned to an application, Netskope’s auto-update has implicit logic to stagger updates even if all Publishers are assigned to the same upgrade profile. This ensures that applications remain available during the upgrade.
You can get Publisher updates automatically, and also specify the version to be upgraded or downgraded for the Publishers. Options include the very latest Publisher version, and one of the two previous versions of the Publisher.
For example, if the latest Publisher version is 111.0.0.x, Netskope will support Auto-Updates to version 111.0.0.x, 110.0.0.x, or 109.0.0.x. Updates for software defects and security vulnerabilities will only be introduced in the latest software version. So you should consider updating your Publishers to the latest version to take advantage of enhancements and security updates.
The Publisher checks for a minimum disk space of 300 MB for System and Publisher upgrades. If the disk space verification fails, the Admin is notified through a message in the Admin UI and through email alerts, if notifications have been set up.
Auto-update use case factors include:
-
- When Auto-Update is enabled for your tenant, all Publishers are included in a Default profile. The Default profile is disabled by default. When a default profile is enabled, all Publishers associated with this profile are enabled with the Auto-Update capability. Moving forward, all Publishers will be required to create or select an Update Profile upon creation.
-
- You can perform a manual upgrade even if an Auto-Update profile is disabled.
-
- Scheduled auto-updates will not occur when an Auto-Update profile is disabled.
-
- An initiated upgrade process will continue if you disable the Auto-Update profile while the upgrade is in progress.
-
- Before upgrading all Publishers, upgrade a test Publisher first, and then proceed to the other Publishers.
Publisher Auto-Update Best Practices
In enterprise environments, Netskope recommends the following:
- Schedule Publisher updates during maintenance windows or non-peak hours for their location. You can have multiple Publisher Auto-Update profiles scoped to Publishers in different regions.
-
- Enable Auto-update alerts for successful and failed Publisher Auto-Updates including the following:
- Version update succeeded
- Version update failed
- Version update started but reconnection failed.
You can optionally also enable alerts for upgrades that start, and a 24-hour alert about when Auto-Updates will start.
- Enable Auto-update alerts for successful and failed Publisher Auto-Updates including the following:
-
- Ensure that all Publishers check for N-2 releases at least monthly to ensure you stay within the Publisher Support Policy.
-
- Ensure that at least one Publisher is available during the upgrade of other Publishers to provide administrative access should an upgrade fail. You can consider deploying dedicated Publishers for administrative functions to provide SSH access or via your virtualization solution’s interface.
Note
During auto-update of this Publisher, Netskope will update the Publisher and make an attempt to update the Ubuntu system. Kernel updates should be carried out by the admin. For more information on kernel update, go to Enable Kernel Updates.
Configure Auto-Update Profiles
You can create, edit, or delete Auto-Update profiles, including the Default profile.
- Go to Settings > Security Cloud Platform > Publishers and select Auto-Update Profile on the Auto-Update Settings dropdown.
- You can search for and sort existing profiles in the Auto-Updates Profiles dialog box, plus edit and delete profiles using the pencil and trash can icons. To create a new Auto-Update profile, click Add New.
- From the dropdown list, select Latest Release, or one of the previous versions of the latest release.
Tip
If you want to downgrade to a previous version, select the Latest-1 or Latest-2 version. The version numbers for each option are shown.
- Specify a release frequency. For a Weekly update, specify the day of the week. For a Monthly update, specify the week and the day.
- Select a time and a time zone to start the update. Publisher updates take around two hours to complete from the start time specified in the Update Profile.
- When finished, click Save.
Manage Auto-Update Profiles
After Auto-Update profiles have been created, you can search for a profile, and also sort the profiles in the table in the Auto-Update Profile dialog box.
Auto-Update profiles can be applied to a single Publisher or multiple Publishers.
There are a couple of methods to modify an existing Auto-Update Profile, depending on whether you’re modifying a single Publisher or multiple Publishers.
Single Publisher
For a single Publisher, select Edit from the Publisher side menu.
In the Edit Publisher dialog box, you can change the Update Profile for a Publisher by searching for and selecting a profile from the dropdown list. When finished, click Save.
You can also delete and update a Publisher using the options on Publisher dropdown list. Click Update to immediately upgrade the Publisher to the Publisher version specified in the Update Profile.
Multiple Publishers
For multiple Publishers, select the Publishers (in the left column), and then click Update to immediately upgrade all the Publishers to the Publisher version specified in the Update Profile.
To change Auto-Update Profiles for multiple Publishers, click Change Update Profile To and select an Update profile from the Actions dropdown, and then select an Update profile from the dropdown list. When finished, click Save.
Configure Auto-Update Alerts
To receive notifications of when updates occur, for specific users, and the type of event that occurs, you can configure Auto-Update Alerts.
- Select Auto-update Alerts on the Auto-Update Settings dropdown.
Specify who you want to receive notifications, and then the events that you want to know about. - Select the admins in the dropdown list; only users with Admin privileges for your tenant are shown in this list. To add Users, enter the user’s email address, separated by commas if there is more than one.
- Select the Alert types you want Admins and Users to receive via email:
- Version updates will start in 24 hours: Profile-based. Publishers associated with a Profile are batched.
- Version update started: Stitcher-based. Publishers associated with a stitcher are batched.
- Version update succeeded: Batch-wise, they will be divided in to 3 batches (for example, if there are three versions specified). Publisher 1 in batch 1, Publishers 2 and 3 in batch 2, and Publishers 4 and 5 in batch 3. There will be three emails.
- Version updated failed: Stitcher-based. For example, if there are seven Publishers, and batch 2 has three Publishers, out of the three Publishers in batch 2, two publishers are connected to Stitcher 1, and the remaining Publisher is connected to Stitcher 2. Upgrade failure notifications will result in two emails.
- Version update started but reconnection failed: Profile-based. Failures are due to a timeout.
- Click Next to save this configuration.
Error Codes for Publisher Auto-Update
Error Code | Reason | Suggestion for admin |
---|---|---|
0x0000 (0) | Publisher is up-to-date. | No action required. |
0x0100 (256) | Publisher auto-update failed. | Contact Netskope Support. |
0x0101 (257) | Publisher auto-update failed while attempting to open the upgrade trigger file. | Check logs/publisher_wizard.log for more detail. |
0x0102 (258) | Publisher auto-update failed due to timeout. | Contact Netskope Support. |
0x0103 (259) | Publisher reconnection failed due to timeout. | Contact Netskope Support. |
0x0200 (512) | Publisher Host OS update failed. | Contact Netskope Support. |
0x0201 (513) | Publisher auto-update failed while upgrading Docker engine. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
0x0202 (514) | Publisher Host OS update failed. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
0x0203 (515) | Publisher Host OS update failed. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
0x0204 (516) | Publisher Host OS update failed. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
0x0205 (517) | Publisher Host OS update failed. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
0x0206 (518) | Publisher Host OS update failed. | Check logs/publisher_wizard.log for more details. |
0x0207 (519) | Publisher Host OS update was stopped. Not enough disk space for the publisher Host OS update. | Check logs/publisher_wizard.log for more details and free up the disk space for the Host OS update. |
0x0300 (768) | Publisher auto-update failed. | Contact Netskope Support. |
0x0301 (769) | Publisher auto-update failed while downloading docker image. | Check network connectivity between publisher and hub.docker.com. Also check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0302 (770) | Publisher software update was skipped. The new version and the existing version are the same. | Double-check the desired upgrade version. |
0x0303 (771) | Publisher auto-update failed while attempting to stop the existing Publisher container. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0305 (773) | Publisher auto-update failed while attempting to install the Publisher UI package | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0306 (774) | Publisher auto-update failed. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0307 (775) | Publisher auto-update failed. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0308 (776) | Publisher auto-update failed. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0309 (777) | Publisher auto-update failed. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x030A (778) | Publisher auto-update failed. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x030B (779) | Publisher auto-update failed while launching the publisher UI | Collect the log bundle and contact Netskope Support. |
0x030C (780) | Publisher auto-update failed. | Check logs/publisher_wizard.log for more details. |
0x030D (781) | Publisher software update was stopped. Not enough disk space for the publisher software update. | Check logs/publisher_wizard.log for more details and free up the disk space for the publisher software update. |
0x030E (782) | Publisher failed to verify the docker image signature (China DC Only). | Check logs/publisher_wizard.log for more details. |
0x0400 (1024) | Failed to upgrade Browser Access AnyApp. | Check logs/publisher_wizard.log for more details. |
0x0401 (1025) | Failed to upgrade Browser Access AnyApp images due to insufficient disk space. | Check logs/publisher_wizard.log for more details and free up the disk space for the Browser access AnyApp update. |
0x0402 (1026) | Failed to upgrade Browser Access AnyApp. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
0x0403 (1027) | Failed to upgrade Browser Access AnyApp. | Check logs/publisher_wizard.log for more details and free up the disk space for the publisher software update. |
0x0404 (1028) | Failed to upgrade Browser Access AnyApp. | Check logs/publisher_wizard.log and the Docker log with journalctl -u docker.service command for more details. |
Publisher Auto-Update Error Guidance
If an error occurs during an auto-update, refer to these troubleshooting recommendations.
Error | Recommendation |
---|---|
Publisher auto-update failed while attempting to open the upgrade trigger file. | Check logs/publisher_wizard.log for more detail. |
Publisher auto-update failed due to timeout. | Contact Netskope Support. |
Publisher reconnection failed due to timeout. | Contact Netskope Support. |
Publisher auto-update failed while upgrading Docker engine. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
Publisher Host OS update failed. | Check logs/publisher_wizard.log and the logs in /var/log/apt for more details. |
Publisher Host OS update was stopped. Not enough disk space for the publisher Host OS update. | Check logs/publisher_wizard.log for more details and free up the disk space for the Host OS update. |
Publisher auto-update failed while downloading docker image. | Check network connectivity between the Publisher and hub.docker.com . Also check logs/publisher_wizard.log and the Docker log with the journalctl -u docker.service command for more details. |
Publisher software update was skipped. The new version and the existing version are the same. | Check the desired upgrade version. |
Publisher auto-update failed while attempting to stop the existing Publisher container. | Check logs/publisher_wizard.log and the Docker log with the journalctl -u docker.service command for more details. |
Publisher auto-update failed while attempting to install the Publisher UI package. | Please check logs/publisher_wizard.log and the Docker log with the journalctl -u docker.service command for more details. |
Publisher auto-update failed. | Check logs/publisher_wizard.log and the Docker log with the journalctl -u docker.service command for more details. |
Publisher auto-update failed while launching the Publisher U.I | Collect the log bundle and contact Netskope Support. |
Publisher software update was stopped. Not enough disk space for the publisher software update. | Check logs/publisher_wizard.log for more details and free up the disk space for the Publisher software update. |
Prevent Package Failure during a Publisher Update
What symptom will occur?
During a system upgrade (either automatic or manual), the computer encounters an error code 100. Think of this like the system saying “I can’t complete the update because something is configured incorrectly”.
2024/10/15 04:31:20 UTC - 1149212 CmdWait failed: exit status 100 (E: Sub-process /usr/bin/dpkg returned an error code (1)) 2024/10/15 04:31:20 UTC - 1149212 Failed to apt-get upgrade, exit status 100 (E: Sub-process /usr/bin/dpkg returned an error code (1)) 2024/10/15 04:31:21 UTC - 1149212 Auto upgrade failed, perform the post auto upgrade failed action
Why it happens
This is related to how the computer recognizes its hard drive. Think of it like having an old map (the config file) that points to a street name that’s been changed:
- The old configuration is looking for a traditional hard drive labeled as
xvda
. - But newer Amazon Web Services (AWS) computers use faster SSD drives labeled as
nvme
. - When the system tries to find
/dev/xvda15
(the old path), it fails because that path doesn’t exist anymore.
To make system package upgrade successfully, use the following commands to fix the issue, and note you will need to exit the publisher wizard before these commands can be executed.
sudo apt-get update sudo apt-get upgrade
During the apt-get update a pink console will pop out to re-config the grub package settings, and consequently fix the update.
Why this cannot be fixed in the Auto-Update
This requires a user-input for the Debian package config (debconf), which is not supported in an auto-update and by-design has no advanced user interaction in contrast of the normal simple Y/N input.
Configure a Publisher for Software Updates via Explicit Proxy
This section explains how to configure an Ubuntu host to enable Publisher’s software updates via an Explicit Proxy. Note that this only applies to Publisher outbound traffic for OS and Docker updates. The Publisher tunnel itself does not support traversing explicit proxy and must be allowed to connect direct to the Netskope NPA stitcher IP space.
- Configure the
http_proxy
andhttps_proxy
environment variables via/etc/environment
. Make sure to not modify the PATH= definition that should already exist, as this can impact other Publisher operations. Follow standard “vi” operating instructions to exit the file once created (Esc then :wq).
Here is an example used in a configuration to ensure*.<tenant-domain>
(replace with your tenant) is excluded.169.254.0.0/16
will also need to be excluded for AWS installations.sudo vi /etc/environment export http_proxy="http://10.1.10.1:3128/" export https_proxy="http://10.1.10.1:3128/" export no_proxy="localhost, 127.0.0.1, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, <tenant-domain>"
- Next configure
docker-ce
proxy settings, similar to the Ubuntu settings.sudo mkdir /etc/systemd/system/docker.service.d/ sudo vi /etc/systemd/system/docker.service.d/http-proxy.conf [Service] Environment="HTTP_PROXY=http://10.1.10.1:3128" Environment="HTTPS_PROXY=http://10.1.10.1:3128"
Exit vi and then restart the docker services to make the changes take effect.
sudo systemctl daemon-reload sudo systemctl restart docker
- Log out and log in so the
/etc/environment
variables are applied. - Test connectivity by attempting to Upgrade the Publisher through the Netskope UI.
Upgrade a Publisher for PRC (China)
In the Publisher release R111, a new verification feature has been added for PRC (China) Publishers to verify the authenticity of the Publisher docker image being installed. For this feature to work, certain certificates need to be present in a specific path before the verification and upgrade occurs.
Note
If your China Publishers are below release v112, you must execute one of the following.
There are two options to upgrade the Publisher:
- Option 1: Re-deploy the latest Publisher OVA VM Image
Once the verified R111 ova image is installed, it will also install all the necessary certificates in the publisher vm which shall be used for subsequent upgrades with verification (such as R111 to R112). Refer to the Netskope Private Access Publisher Release Notes to get the OVA URL hosting on AliCloud. - Option 2: Using automation on existing Publishers
If you do not wish to re-deploy the complete Publisher VM through OVA, you can run a python based automation on your existing Publishers, which upgrades the Publisher docker image to the latest (R111) version, and also installs all the necessary certificates in the Publisher VM that will be used for subsequent upgrades with verification (such as R111 to R112).
Note
This option is only applicable for China (PRC) Publishers, and not to be used for rest of the world (ROW) Publishers.
Use these steps to execute this automation.
- Retrieve the script from AliCloud. Execute this curl command to retrieve the script as
verify_and_upgrade_publisher.py
under the directory where the command is run.curl https://npa-ova.oss-cn-shenzhen.aliyuncs.com/publisher.netskope.com/latest/generic/verify_and_upgrade_publisher.py > verify_and_upgrade_publisher.py
- Verify the hash of the script. Execute this command in the same path where the script was downloaded, and verify whether the resulting SHA256 hash is same as:
904c26f4a2b5941a63edce5cfa1bdc0b4f8f9af23fd3f8919e7f535d73a00d2c
. This step is important to make sure the verified Netskope provided automation is being executed on the Publisher VM.shasum -a 256 verify_and_upgrade_publisher.py
- Execute the automation to initiate upgrade with verification. Run this command in the terminal under the same path where the script was downloaded.
sudo python3 verify_and_upgrade_publisher.py
Enable Kernel Updates
As part of Publisher software update, it’s recommended to regularly update the kernel of the host Ubuntu OS. The unattended kernel updates are used for updating the kernel on Publisher images. Publisher Release Notes contains the latest kernel versions for reference.
If the kernel version of your Publisher instance is not the latest, or the Publisher wizard shows a pending reboot, a reboot of the machine will update the kernel to the latest version. As a best practice, you could schedule regular reboots to update the kernel version to the latest version.
Enable Kernel Updates on an OVA Publisher
You can leverage automated/unattended kernel updates for OVA Publishers in one of two ways:
- Enable as part of fresh installation: Unattended Kernel updates on OVA images are enabled starting from release 110. You need to fresh install the Publisher to enable the unattended kernel updates.
In addition, you could also enable kernel updates on your existing OVA instances. Follow these steps if you wish to enable unattended kernel updates. - Enable existing OVA Publisher instances: Follow these steps to enable unattended kernel updates on existing OVA Publisher instances.
- Expand the disk of the OVA running instances from 8GB to 16GB using VM orchestrators. For ESXi, log in to ESXi, select the virtual machine, and power it off. Click Edit Settings, change the size of the Hard Disk, click OK, and power it on.
- Log in to your Publisher instance, download and run the script below. The script will expand the disk, install the required softwares, and reboot the machine.
- File:
https://s3-us-west-2.amazonaws.com/publisher.netskope.com/latest/generic/expand_drive.sh
- shasum 256:
1e858c0b600184462a661e4390169d7ce779b6f4de5d455f081f74dab8a48f7c
- File:
- Execute the script:
chmod +x ./expand_drive.sh sudo ./expand_drive.sh
Note
Run the script with sudo. If the script was interrupted by the user, or due to the SSH session down, re-run the script.
- Check the disk size on the Publishers. The total disk size should be 16GB minimum now.
- Check the package newly installed linux-image-generic.
- The Linux kernel should be up to date
Enable Kernel Updates on a non-OVA Publisher
The unattended kernel updates are turned on by default for AMI, VHD and VHDX. Updates to the kernel are installed automatically on these image instances. A standard Publisher OS reboot will bring the instance’s kernel up-to-date.
Publisher Filtering and Exporting Options
To use these features, go to Settings > Security Cloud Platform > Publishers.
Filters
These filtering options are available in the Netskope UI.
- Status
- Update Profile
- Version Update Status
- Version
- Publisher CN
Note
When you select a filter with a search icon , that value is added to the search field so you can add more specifics. When a filter has an adjacent toggle arrow
, there are expanded options to choose from.
You can also clear and remove filters.
Export
The results displayed can be exported by clicking Export.
Choose to export the displayed columns, or select the columns to export, and then click Export.
CentOS-based Publisher Support End of Life
Starting with release 105 (end of May 2023), Netskope Private Access will stop supporting CentOS as the base OS for Publishers and only support Ubuntu-based Publishers.
Ubuntu provides an improved security posture from available CIS benchmarks for Linux distros, and Ubuntu also enables the Auto-Update capability for Publishers. Netskope recommends that you replace existing CentOS Publishers with Ubuntu Publishers using one of these methods.
Method 1
You can generate a new token for an existing CentOS Publisher and use that to register a new Ubuntu Publisher. This will expire the previous registration for the existing CentOS Publisher and replace it with the Ubuntu Publisher. With this method, you do not have to update the App Definitions that reference the existing CentOS Publishers.
Method 2
You can configure new Publishers, add them to the App definitions, and then remove the existing CentOS Publishers in the App Definition.
Considerations
-
- Ubuntu Publishers have feature parity with CentOS Publishers and do not have any capability limitations.
-
- You can use a mix of CentOS and Ubuntu Publishers simultaneously for application access during this move to Ubuntu only support.
Enable SNMP on a Publisher
This section explains how to enable SNMP v3 on a Publisher and edit the firewall to allow external monitoring.
- Connect to a Publisher using SSH and log in.
- On the menu, select
6
and exit to the CLI. - Update all packages (recommended):
sudo apt-get update
- Install SNMP.
sudo apt-get -y install snmpd libsnmp-dev
- Configure the agentAddress in the
/etc/snmp/snmpd.conf
file. Add this line to the file:disk / 10000
- Stop the snmpd service so you can add a user.
sudo service snmpd stop
- Add an SNMP v3 user.
sudo net-snmp-config --create-snmpv3-user -A <AuthPassword> -X <CryptoPassword> -a <MD5|SHA> -x <AES|DES> <user>
- Set up the rouser correctly on
/etc/snmp/snmpd.conf
by addingrouser authPrivUser authpriv -V systemonly
. - Set up the TCP correctly on
/etc/snmp/snmpd.conf
by replacingagentaddress 127.0.0.1, [::1]
withagentaddress udp:161
. - Restart the SNMPD service.
sudo service snmpd restart
- Check that SNMPD is started.
sudo service snmpd status
- Verify the firewall (ufw) is running.
sudo ufw status
- Configure UFW to allow connections to SNMPD. The SNMP daemon will listen for connections on port 161.
sudo ufw allow in to any port 161 proto udp
- Verify the SNMP service has been allowed by the firewall permanently and that UDP traffic on Port 161 is allowed.
sudo ufw status Status: active To Action From -- ------ ---- 161/udp ALLOW Anywhere 161/udp (v6) ALLOW Anywhere (v6)
- To check if snmpd on the Publisher machine works correctly, check the following:
sudo service snmpd status
snmpd.service - Simple Network Management Protocol (SNMP) Daemon.
Loaded: loaded (/lib/systemd/system/snmpd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2024-05-31 07:26:11 UTC; 59min ago
Process: 14823 ExecStartPre=/bin/mkdir -p /var/run/agentx (code=exited, status=0/SUCCESS)
Main PID: 14824 (snmpd)
Tasks: 1 (limit: 1126)
Memory: 5.6M
CGroup: /system.slice/snmpd.service
└─14824 /usr/sbin/snmpd -LOw -u Debian-snmp -g Debian-snmp -I -smux mteTrigger mteTriggerConf -f -p />
Publisher Monitoring
Thresholds to Monitor
CPU Utilization > 75%
Memory Utilization > 90%
Disk Space Left < 1GB
To Validate Resolution of the NPA Cloud
curl https://dns.google/resolve?name=stitcher.npa.<tenant-domain>&type=A&edns_client_subnet=PublisherEgressIP
Nslookup stitcher.npa.<tenant-domain>
Linux OS CLI Commands to Monitor Resources
top, cat /proc/meminfo, htop, sysstat, nload, iftop, nethog, bmon
SNMP OIDs to Monitor Resources
Available space on the disk: .1.3.6.1.4.1.2021.9.1.7.1
Used space on the disk: .1.3.6.1.4.1.2021.9.1.8.1
Percentage of space used on disk: .1.3.6.1.4.1.2021.9.1.9.1
Percentage of inodes used on disk: .1.3.6.1.4.1.2021.9.1.10.1
Path where the disk is mounted: .1.3.6.1.4.1.2021.9.1.2.1
Path of the device for the partition: .1.3.6.1.4.1.2021.9.1.3.1
Total size of the disk/partion (kBytes): .1.3.6.1.4.1.2021.9.1.6.1
Percentage of user CPU time: .1.3.6.1.4.1.2021.11.9.0
Raw user CPU time: .1.3.6.1.4.1.2021.11.50.0
Percentage of system CPU time: .1.3.6.1.4.1.2021.11.10.0
Raw system CPU time: .1.3.6.1.4.1.2021.11.52.0
Percentage of idle CPU time: .1.3.6.1.4.1.2021.11.11.0
Raw idle CPU time: .1.3.6.1.4.1.2021.11.53.0
Total real memory: .1.3.6.1.4.1.2021.4.5.0
Available real memory: .1.3.6.1.4.1.2021.4.6.0
Total free memory (includes SWAP): .1.3.6.1.4.1.2021.4.11.0
Total bytes received on the interface: .1.3.6.1.2.1.2.2.1.10
Total bytes transmitted on the interface: .1.3.6.1.2.1.2.2.1.16
Enable an SNMP Trap for a Publisher
Prerequisites
- Publisher: Must have snmpd installed to send trap information.
- Server (Trap Receiver): Must have snmptrapd installed to receive trap information.
Configure SNMP for the Publisher
- Install SNMP on your Publisher using these instructions.
- Install the required
snmpd
andsnmpget
packages using this command:sudo apt-get update && sudo apt-get install snmp snmpd
. - Configure the
snmpd
file located at:/etc/snmp/snmpd.conf
.- Set the community string with the following line:
rocommunity COMMUNITY_STRING
. - Define the trap receiver. Replace
$TRAP_RECEIVER_IP
with the IP address of your SNMP manager and$COMMUNITY_STRING
) with your community string:- Ensure that
$TRAP_RECEIVER_IP
is the IP address of your server. - Add the line:
trapsink $TRAP_RECEIVER_IP $COMMUNITY_STRING
.
- Ensure that
- Configure the agent to send traps by adding:
authtrapenable 1
.
- Set the community string with the following line:
- Restart the
snmpd
service with the command:sudo service snmpd restart
. - Configure the
snmptrapd
file located at:/etc/snmp/snmptrapd.conf
. Add the line:disableAuthorization yes
. - Here is an example configuration for
/etc/snmp/snmpd.conf
. In this example, assume your$COMMUNITY_STRING
isNETSKOPE
, and your$TRAP_RECEIVER_IP
is13.231.185.233
. The remaining settings are left unchanged.
ubuntu@publisherIP:~$ sudo cat /etc/snmp/snmpd.conf ########################################################################### # # snmpd.conf # An example configuration file for configuring the Net-SNMP agent ('snmpd') # See snmpd.conf(5) man page for details # ############################################################## # SECTION: System Information Setup # # syslocation: The [typically physical] location of the system. # Note that setting this value here means that when trying to # perform an snmp SET operation to the sysLocation.0 variable will make # the agent return the "notWritable" error code. IE, including # this token in the snmpd.conf file will disable write access to # the variable. # arguments: location_string sysLocation Sitting on the Dock of the Bay sysContact Me <me@example.org> # sysservices: The proper value for the sysServices object. # arguments: sysservices_number sysServices 72 ############################################################## # SECTION: Agent Operating Mode # # This section defines how the agent will operate when it # is running. # master: Should the agent operate as a master agent or not. # Currently, the only supported master agent type for this token # is "agentx". # # arguments: (on|yes|agentx|all|off|no) master agentx # agentaddress: The IP address and port number that the agent will listen on. # By default the agent listens to any and all traffic from any # interface on the default SNMP port (161). This allows you to # specify which address, interface, transport type and port(s) that you # want the agent to listen on. Multiple definitions of this token # are concatenated together (using ':'s). # arguments: [transport:]port[@interface/address],... agentaddress udp:161 ########################################################################### # SECTION: Access Control Setup # # This section defines who is allowed to talk to your running # snmp agent. # Views # arguments viewname included [oid] # system + hrSystem groups only view systemonly included .1.3.6.1.2.1.1 view systemonly included .1.3.6.1.2.1.25.1 # rocommunity: a SNMPv1/SNMPv2c read-only access community name # arguments: community [default|hostname|network/bits] [oid | -V view] # Read-only access to everyone to the systemonly view rocommunity NETSKOPE <<<<< Modify this rocommunity6 public default -V systemonly trapsink 13.231.185.233 NETSKOPE <<<<< Modify this authtrapenable 1 <<<<< Modify this # SNMPv3 doesn't use communities, but users with (optionally) an # authentication and encryption string. This user needs to be created # with what they can view with rouser/rwuser lines in this file. # # createUser username (MD5|SHA|SHA-512|SHA-384|SHA-256|SHA-224) authpassphrase [DES|AES] [privpassphrase] # e.g. # createuser authPrivUser SHA-512 myauthphrase AES myprivphrase # # This should be put into /var/lib/snmp/snmpd.conf # # rouser: a SNMPv3 read-only access username # arguments: username [noauth|auth|priv [OID | -V VIEW [CONTEXT]]] rouser authPrivUser authpriv -V systemonly disk / 10000
Configure the SNMP Trap on a Linux Server
- Install
snmptrapd
with this command:sudo apt-get update && sudo apt-get install snmptrapd
. - Configure
snmptrapd
by editing the configuration file located at:/etc/snmp/snmptrapd.conf
.
To allow incoming traps, add this line to the configuration file:disableAuthorization yes
. - Configure the
snmptrapd systemd
configuration located at:/lib/systemd/system/snmptrapd.service
.
For example, you can modify the log output file by changing theExecStart
line to:ExecStart=/usr/sbin/snmptrapd -f -Lf $OUTPUT_PATH
. - Configure the firewall to allow incoming traffic from port
162/udp
by running:sudo ufw allow 162/udp
.
Note that if your UFW is disabled, you should skip this step; otherwise, you risk blocking your SSH access. - Reload the
snmptrapd
configuration, and restart the service.
sudo systemctl daemon-reload sudo systemctl enable snmptrapd sudo systemctl start snmptrapd
Example output for /lib/systemd/system/snmptrapd.service
:
cat /lib/systemd/system/snmptrapd.service [Unit] Description=Simple Network Management Protocol (SNMP) Trap Daemon. After=network.target ConditionPathExists=/etc/snmp/snmptrapd.conf [Service] Type=simple ExecStart=/usr/sbin/snmptrapd -f -Lf /var/log/snmptrap.log ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target
Example output for /lib/systemd/system/snmptrapd.service
:
cat /etc/snmp/snmptrapd.conf # # EXAMPLE-trap.conf: # An example configuration file for configuring the Net-SNMP snmptrapd agent. # ############################################################################### # # This file is intended to only be an example. # When the snmptrapd agent starts up, this is where it will look for it. # # All lines beginning with a '#' are comments and are intended for you # to read. All other lines are configuration commands for the agent. # # PLEASE: read the snmptrapd.conf(5) manual page as well! # #authCommunity log,execute,net private #authCommunity log,execute,net public # ## send mail when get any events #traphandle default /usr/bin/traptoemail -s smtp.example.org foobar@example.org # ## send mail when get linkDown #traphandle .1.3.6.1.6.3.1.1.5.3 /usr/bin/traptoemail -s smtp.example.org foobar@example.org disableAuthorization yes
E2E Testing and Expected Result
- On the Publisher, send the trap message with the public IP of the server:
- You may need to install MIB files related module by running:
sudo apt-get install snmp-mibs-downloader
. - Send the
snmptrapd
message.- For example, send available disk space:
#!/bin/bash # SNMP community string and target IP COMMUNITY="NETSKOPE" TARGET_IP="52.199.37.185" # Retrieve the value for available space on the disk available_space=$(snmpget -v 2c -c $COMMUNITY localhost .1.3.6.1.4.1.2021.9.1.7.1 -Ov | awk '{print $2}') # Send the trap snmptrap -v 2c -c $COMMUNITY $TARGET_IP '' .1.3.6.1.4.1.2021.9.1.7.1 i integer $available_space
- Other metrics can be found here.
- The generalized syntax / rule might be:
- Use
snmpget
to get the result, names as$METHOD_RESULT
of desired metric based on OID on localhostsnmpget -v 2c -c $COMMUNITY_NAME localhost $OID -Ov | $PROCESSING_METHOD?
. - Use
snmptrap
to send the result to the target machine:sudo snmptrap -v 2c -c $COMMUNITY_NAME $TRAP_RECEIVER_IP "" $OID [i|s] [integer|string] $METHOD_RESULT
.
- Use
- For example, send available disk space:
- You may need to install MIB files related module by running:
- On the server (trap receiver), receive the
snmptrap
message withsudo tail -f /var/log/snmptrap.log
.
You should see that 15G is the disk remaining space:sudo tail -f /var/log/snmptrap.log unknown snmp version 193 NET-SNMP version 5.8 AgentX subagent connected NET-SNMP version 5.8 2024-01-11 07:16:20 ec2-13-114-11-108.ap-northeast-1.compute.amazonaws.com [UDP: [13.114.11.108]:34966->[172.31.20.184]:162]: iso.3.6.1.2.1.1.3.0 = Timeticks: (17718887) 2 days, 1:13:08.87 iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.2021.9.1.7.1 iso.3.6.1.6 = STRING: "15G"
- You can use
crontab -e
along with bash scripts to make it periodically process.
SNMP Trap Troubleshooting
- The
snmpd
failed to run withexit-code
.root@ip-172-31-29-201:/home/ubuntu# service snmpd status snmpd.service - Simple Network Management Protocol (SNMP) Daemon. Loaded: loaded (/lib/systemd/system/snmpd.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2024-01-10 06:07:06 UTC; 6min ago Process: 98894 ExecStartPre=/bin/mkdir -p /var/run/agentx (code=exited, status=0/SUCCESS) Process: 98895 ExecStart=/usr/sbin/snmpd -LOw -u Debian-snmp -g Debian-snmp -I -smux,mteTrigger,mteTriggerConf -f > Main PID: 98895 (code=exited, status=1/FAILURE) Jan 10 06:07:06 ip-172-31-29-201 systemd[1]: Starting Simple Network Management Protocol (SNMP) Daemon.... Jan 10 06:07:06 ip-172-31-29-201 systemd[1]: Started Simple Network Management Protocol (SNMP) Daemon.. Jan 10 06:07:06 ip-172-31-29-201 snmpd[98895]: Error opening specified endpoint "[::1]" Jan 10 06:07:06 ip-172-31-29-201 snmpd[98895]: Server Exiting with code 1 Jan 10 06:07:06 ip-172-31-29-201 systemd[1]: snmpd.service: Main process exited, code=exited, status=1/FAILURE Jan 10 06:07:06 ip-172-31-29-201 systemd[1]: snmpd.service: Failed with result 'exit-code'
Solution: Modify the agentaddress in
/etc/snmp/snmpd.conf
fromagentaddress 127.0.0.1, [::1]
toagentaddress udp:161
. After making this change, restart the SNMP service. Once restarted, verify that thesnmpd
service is running correctly.snmpd.service - Simple Network Management Protocol (SNMP) Daemon. Loaded: loaded (/lib/systemd/system/snmpd.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2024-01-10 06:28:09 UTC; 5min ago Process: 102590 ExecStartPre=/bin/mkdir -p /var/run/agentx (code=exited, status=0/SUCCESS) Main PID: 102591 (snmpd) Tasks: 1 (limit: 4603) Memory: 6.5M CGroup: /system.slice/snmpd.service └─102591 /usr/sbin/snmpd -LOw -u Debian-snmp -g Debian-snmp -I -smux mteTrigger mteTriggerConf -f -p /run> Jan 10 06:28:09 ip-172-31-29-201 systemd[1]: Starting Simple Network Management Protocol (SNMP) Daemon.... Jan 10 06:28:09 ip-172-31-29-201 systemd[1]: Started Simple Network Management Protocol (SNMP) Daemon..
- If the
snmptrap
is complaining about the syntax error:sudo snmptrap -v 2c -c NETSKOPE 52.199.37.185 "" .1.3.6.1.4.1.2021.9.1.7.1 s $(df -h / | awk '/\/$/ {print $4}')
.
s: Missing type/value for variable
Solution: Add a string suffix after thes:
:sudo snmptrap -v 2c -c NETSKOPE 52.199.37.185 "" .1.3.6.1.4.1.2021.9.1.7.1 s string $(df -h / | awk '/\/$/ {print $4
.
}') - For integer
i
, add integer suffix after thei
.
Publisher Logs for Troubleshooting
Connection Segment | Description | Example |
---|---|---|
Registration Logs – Publisher | Logs to verify successful registration, or failed registration. | Logs to check:
~/logs/publisher_wizard.log Successful Registration: 2021/07/27 20:00:41 UTC Registering with your Netskope address: ns-6413.us-sv5.npa.<tenant-domain> 2021/07/27 20:00:41 UTC Publisher certificate CN: 130dbd9d40e4ad35 2021/07/27 20:00:41 UTC Attempt 1 to register publisher. 2021/07/27 20:00:43 UTC Publisher registered successfully. Failed Registration: 2021/08/19 13:21:06 UTC Attempt 1 to register publisher. 2021/08/19 13:21:08 UTC Get https://ns-6413.us-sv5.npa.<tenant-domain>/api/discovery: x509: certificate signed by unknown authority 2021/08/19 13:21:08 UTC Registration failed because a discovery call didn’t succeed. Please generate a new token and try again. |
Publisher ⇔ Netskope connectivity logs | Logs to check:
~/logs/agent.txt Succesful tunnel connection: eventlog.cpp:115:logPublisherTunnelEvent():0x0 {“eventId”: “NPACONNECTED”, “publisherId”: “130dbd9d40e4ad35”, “stitcherIp”: “163.116.135.6”, “tenant”: “ns-6413.us-sv5.npa.<tenant-domain>“} Successful connection and certificate verification: sslhelper.cpp:80:verify_callback():0x0 Verified: /DC=io/DC=newedge/CN=New Edge Root CA Failed connection due to SSL error sslhelper.cpp:302:logSslError():0x0 SSL Error 5 error:00000005:lib(0):func(0):DH lib |
|
Publisher⇔ Netskope HTTPS logs | Management Plane: openssl s_client -connect ns-{TENANTID}.{POPNAME}.npa.<tenant-domain>:443 -servername ns-{TENANTID}.{POPNAME}.npa.<tenant-domain>
Data Plane: openssl s_client -connect stitcher.npa.<tenant-domain>:443 -servername ns-{TENANTID}.{POPNAME}.npa.<tenant-domain> |
|
Publisher⇔ Application Connection Logs | Logs to check:
~/logs/agent.txt Application definition and reachability: reachability.cpp:109:parse():0x2484790 Added protocols login.microsoftonline.com:tcp:443-443; tcp:80-80; udp:443-443; udp:80-80;Application connection: tcpproxyhandler.cpp:35:TcpProxyHandler():0x2504cf0 Creating tcp connection to login.microsoftonline.com:443 |
|
Client connects and disconnects | May follow Publisher disconnects and can be used to correlate issues: neconfig.cpp:121:setClientId():0x0 Set clientId l0ThzLYeZnqA
Indicates a graceful shut down and will not always be present if there’s an issue: L3ClientChannel.cpp:48:destroy():0x1292810 Cleaning up l3clientChannel |
Disable Password Expiry for a Publisher
Password policy for the Publisher are enabled for versions 101 or lower. The Publisher host user password may expire if not changed regularly. This article explains how to disable the password expiry. Versions 102 and above will have the password policy disabled, and you are now required to apply your corporate password policy to your Publishers.
AWS AMI Publisher
If your AWS publisher was deployed from the Netskope prebuilt images (from AWS marketplace), the following instructions can help you to remove the password expiry. This approach is applicable for version 94+ Ubuntu Publishers.
Note
Publisher images on Amazon marketplace (AMI) now use 16GB HDD space by default.
The Ubuntu publisher built in with the AWS System Manager (SSM) agent. You can use SSM to log in to the Publisher EC2 instance and remove the password expiry.
- Create an IAM role with the SSM permissions.
- Create an IAM role.
- Add permission policy AmazonSSMManagedInstanceCore into the IAM role.
- Attach the IAM role to the Publisher EC2 instance.
- Connect with the Publisher EC2 instance via SSM.
- After you log in into the Publisher, use this following command to disable the password expiry.
sudo chage -m 0 -M 99999 ubuntu
- Use the following to confirm the password expiry was disabled or not.
sudo chage -l ubuntu
- You should able to log in to the Publisher via SSH after disabling the password expiry.
Azure VHD Publisher
If your Azure publisher was deployed from the Netskope prebuilt images (from Azure marketplace), the following instructions can help you to remove the password expiry. This approach is applicable for version 96+ Ubuntu Publishers.
- You can use the built-in Reset password function in the Azure portal.
- For Mode, select Reset Password, your username, and new password (twice) to reset your ubuntu password.
- After resetting the password, you should be able to log in to the Publisher via SSH.
- Disable the password expiry using this command.
sudo chage -m 0 -M 99999 ubuntu
- Use this command to confirm if the password was disabled successfully or not.
sudo chage -l ubuntu
OVA/VHDX Publisher
If your Azure publisher was deployed from the Netskope prebuilt OVA/VHDX images, use these steps to remove the password expiry. You should be able to boot into Single User Mode from Linux GRUB to remove the password expiry.
- Reboot the VM.
- Enter the GRUB menu by keeping pressing the shift key. If you are using Windows, you may need to disable the sticky key.
- From the GRUB boot prompt, press the E button to edit the first boot option.
- In the GRUB menu, find the kernel line starting with
linux /vmlinuz
and addinit=/bin/bash
at the end of the line.
- Press CTRL+X to save the changes and boot the server into single-user mode. Once booted. the server will boot into the root prompt.
- Type in the command
mount -o remount,rw /
to mount the file system.
- Use
chage -m 0 -M 99999 ubuntu
to disable the password expiry, and usechage -l ubuntu
to confirm if the password was disabled successfully or not.
- Reboot the system. Use
reboot -f
to reboot the VM. - You will see the GRUB menu again. Press enter on the first item or wait 30 seconds, the boot process will continue. And you should be able to log in into your VM again with your password.
Re-enroll a Publisher
You can re-enroll a new instance of Publisher into an existing entry in the Admin Console. Follow these steps to re-enroll a Publisher instance.
- In the Netskope UI, go to Settings > Security Cloud Platform > Publishers.
- Click on the Publisher that needs to be re-enrolled. Make sure the Publisher is in the Disconnected State.
- Click Save and Continue.
- Click Generate Token.
- Click Copy to get the registration token.
You can now install the new Publisher instance on a new VM, or on the existing VM. Use the token to register the new Ubuntu Publisher instance. All the existing App Definitions that reference this Publisher will continue to work.