Customers currently using Transaction Event Streaming are requested to migrate to Netskope Log Streaming, which is Netskope’s new, unified log delivery solution that offers enhanced flexibility, broader event coverage, and a simpler path to integrate with their downstream systems.
The HTTP Transaction events are critical for enterprises for deeper visibility as companies move to adopt cloud products. Netskope already provides page, app events, etc. that are rolled up and summarized to avoid “noisy” web traffic. The rolled up view is great for admins, in addition, transaction events provide granular information about the web sites that users have accessed.
Configure Transaction Events
You will need a subscription endpoint URL and subscription key to access the streamed events.
To retrieve the subscription key and path:
- Navigate to Settings > Tools > REST API v2 > New Token. The Create REST API Token window displays.
- Click the Add Endpoint dropdown and select the /api/v2/events/token/transaction_events endpoint to create the API token.
- Select the Read radio button in the Privilege column.
- Click Save. A confirmation box displays stating the token creation is successful.
- Click Copy Token to save it for later use in your API requests and add it to the Netskope API token header.
- Navigate to REST API v2 > API Documentation. A new Swagger UI tab will open. You can access the API Documentation from the Create REST API Token window as well. Click
, the Available Authorizations pop-up opens, paste the saved token in the Value field and click Authorize.
- From the Swagger UI tab, under to the Events section, navigate to the /api/v2/events/token/transaction_events endpoint and click the line. The details display, click Try it out. Select decode section > true in the dropdown > click Execute.
Upon successful execution, the response will carry the subscription key and subscription path.
Query Transaction Events Metrics
Transaction events are stored in Google PubSubLite by Netskope ingestion services. Once the transaction events feature is enabled in your account, you can consume the data using the subscription endpoint URL and subscription key retrieved using the token/transaction_events API. This enables you to query metrics once your account is configured to receive the transaction events.
API returns data for the following PubSubLite metrics:
subscription/backlog_message_count: Number of messages that have been sent to a PubSubLite subscription but have not yet been acknowledged by any consumer configured by customers.
subscription/oldest_unacked_message_age: The age of the oldest unacknowledged message in a PubSubLite subscription e.g., How long the oldest unacknowledged message has remained unacknowledged.
The API returns hourly buckets equivalent to hours parameter value. The default value is 24 Hours and the maximum value is one week: 168 Hours. Time Series buckets maintain the ascending order e.g., the latest hour data is the last data point.
Follow the steps below to retrieve transaction events metrics for your account.
-
Navigate to Settings > Tools > REST API v2 > New Token. The Create REST API Token window displays.
-
Click the Add Endpoint dropdown and select the /api/v2/events/metrics/transactionevents endpoint to create the API token.
-
Select the Read radio button in the Privilege column.
-
Click Save. A confirmation box displays stating the token creation is successful.
-
Click Copy Token to save it for later use in your API requests and add it to the Netskope API token header.
-
Go to REST API v2 > API Documentation. A new Swagger UI tab will open. You can access the API Documentation from the Create REST API Token window as well. Click Authorize, the Available Authorizations pop-up opens, paste the saved token in the Value field and click Authorize.
-
From the Swagger UI tab, under to the Events section, navigate to the /api/v2/events/metrics/transactionevents endpoint and click the line. The details display, click Try it out. Select Parameters section > type a number in the query field > click Execute.
-
Upon successful execution, the response will carry the transaction events metrics details.
Alternate Method to Generate a Subscription Endpoint
Log in to your Netskope UI and go to Settings > Tools > Event Streaming. Copy your subscription endpoint and generate your download key from the Event Streaming page.
The old subscription path and key expires. With a new subscription path, event streaming will start fresh from the beginning of the retention period, i.e., 7 days ago by default.
Consume Transaction Events
Once the transaction events feature is enabled in your account, you’ll be able to consume the data from the subscription endpoint.
To receive the events from the subscription, refer to the Receiving messages from Lite subscriptions link .
The Netskope SDK also provides an example of using the REST API v2 token as authentication which provides the subscription endpoint URL and subscription key internally to use the Google SDK.
In addition, you can receive messages with various Client libraries. Netskope retains transaction events for seven days by default if not consumed.
There are different access methods for transaction events:
- Google SDK
- Netskope Splunk Integration
- Netskope Cloud Exchange
Google SDK
The enhanced transaction events streaming is delivered through a streaming mechanism.
The transaction event near real-time subscription messages have the following format:
Attributes
- Content-Encoding. gzip – currently the only available value
- Log-Count – number of events enclosed in the message data
- Fields – transaction event fields for each transaction event
Data
- The message data contains gzip compressed transaction events
Refer to the sample code to receive and decode the transaction events.
Netskope Splunk Integration
The Netskope App (Add-on) for Splunk has dashboards for visualization of Events, Alerts, and Web Transaction details. This information is populated on the dashboard.
Users can get information related to data collected in addition to transforming and parsing data with the Add-on app available from splunkbase.
Click the following links to set up the Netskope Splunk Integration:
- Netskope Splunk App Installation and Configuration Guide
- Netskope Add-on for Splunk:
Important
UPGRADING IS SUPPORTED WITHIN THE 2.X CHAIN. IF COMING FROM 1.X, PLEASE REMOVE 1.X BEFORE INSTALLING 2.X.
- Splunk Netskope Dashboards available from splunkbase (optional): download from here
Netskope Cloud Exchange
Netskope Cloud Exchange is available for download on GitHub and accompanying installation documentation is available:
Transaction Events Streaming Service Data Retention Policy
Transaction Events are retained in the streaming service for up to seven days for admins to pull, for error recovery on the customer client side. The data is kept in a series of files by the streaming service. The garbage collection service will remove all files older than seven days, with the exception of the most recent data file.
For most customers during normal operational load, there will be numerous data files in a given hour. However, for some customers with a small data volume, (e.g. who have not implemented Netskope widely or in an active proof of concept phase), the most recent data file might accumulate events for more than seven days resulting in a longer data retention period.