Note
You must have the Netskope Secure Web Gateway license to create URL Lists.
URL lists are a component of custom categories, which offer the flexibility to override the predefined Netskope URL category mapping for a given URL. To create a URL list, select the format and enter the desired URLs. Once you save the URL list, add it to a custom category as an inclusion or exclusion. To learn more, see Custom Category.
Before you begin, compile a list of URLs to include in a policy scan, and if needed, create another list of URLs to exclude from a policy scan. You can enter the URLs individually in the UI, create a CSV file with all the URLs, or use the REST API V2. Once you complete this step, you must create custom categories for the URL lists to apply in a policy.
Important
URL List enhancements are applicable for Real-time Protection policies, SSL Decryption policies, steering exceptions, and REST API V2. Contact Support to enable these enhancements in your account.
Configuring URL Lists
To create a URL list:
- Go to Policies > Profiles > URL Lists.
- Click New URL List.
- In the New URL List window, enter a name for your URL list.
- For URL Type, select the format for the URL list (Exact or Regex). The default behavior is exact match.
- For URL & IP Address, you can manually enter the URLs or IP addresses. If you chose Regex as the URL Type, you can enter regex for URL matching.
- Alternatively, you can also upload a CSV file (maximum upload is 8 MB).
Note
For more information for creating URL lists, see URL List Best Practices.
When adding URLs, you can enter URLs for exact matching or enter wildcard URLs. You can also define port numbers for URLs. When adding URLs, ensure you are following the formatting rules.
| URL Format | Matched URLs |
|---|---|
| www.example.com/path/to/resource | www.example.com/path/to/resource/a www.example.com/path/to/resource/a/b |
| example.com | example.com/ example.com/path/a |
| www.example.com | www.example.com www.example.com/path/a www.example.com/some/path www.example.com/query?a=b http://www.example.com https://www.example.com http://www.example.com:80 https://www.example.com:443 |
| *.example.com | example.com www.example.com www.us.example.com www.uk.example.com |
| www.example.com:8080 | http://www.example.com:8080 https://www.example.com:8080 |
| www.example.com:80 | http://www.example.com https://www.example.com:80 http://www.example.com:80 |
| www.example.com:443 | https://www.example.com https://www.example.com:443 http://www.example.com:443 |
| www.example.com/path/*/ | Not Supported |
| www.example.com/* www.example.com/path/* | Not Supported |
Wildcard domains (e.g., *.example.com) include the root domain and all subdomains. Keep in mind that if two or more wildcard entries, subdomains, and paths are present in your URL list, then the Netskope service uses the longest entry for categorization.
When adding IP addresses, you can add IP addresses ranges and IP addresses with CIDR notation. When adding non-standard ports, you must define the port number for a match.
Tip
When creating a URL list for destination IP addresses, consider the following:
- When you create a Real-time Protection policy for destination IP addresses, you can also create an SSL Do Not Decrypt policy. However, if the Real-time Protection policy’s action is Block, then the activity will be blocked without SSL interception. In other words, the block page will not appear for users when they access the destination IP addresses.
- To block the activity with SSL interception, you must create a Real-time Protection policy only without an SSL policy.
- When you create a URL list for destination IP addresses, the No SNI error setting option is unavailable. This is because when a user accesses a destination IP address, there’s no SNI.
Blocking traffic to a single destination IP address without SNI
When a request goes straight to an IP address (for example, https://1.2.3.4) instead of a domain name, the TLS handshake doesn’t include a Server Name Indication (SNI). Without an SNI, Netskope has no hostname to categorize, so by default this traffic bypasses decryption and policy enforcement entirely — even if that IP address appears in one of your URL lists.
Netskope offers a tenant-level setting that changes this default. The table below shows how enabling it changes what happens to this traffic.
| Setting disabled (default) | Setting enabled | |
|---|---|---|
| Traffic to a destination IP with no SNI | Bypasses decryption and policy enforcement. URL list entries for that IP are not applied. | Evaluated against your URL lists. Matching Real-time Protection and SSL Decryption policies apply — block, allow, decrypt, or do not decrypt — the same as traffic with an SNI. |
| “No SNI” option in Manage Error Settings | Available, since this traffic is bypassed by default. | Removed. Traffic is no longer automatically bypassed, so the option no longer applies. |
Before you enable this setting, review your existing URL lists for any destination IP addresses. Once enabled, those entries are enforced on traffic that was previously bypassed, so confirm your Real-time Protection and SSL Decryption policies produce the outcome you expect for that traffic.
API Endpoints
The following API endpoints are available with REST API V2 for URL lists:
Note
REST API V2 is part of the URL List enhancements, contact Support to enable this feature in your account.
| Endpoint | Description |
|---|---|
| GET /api/v2/policy/urllist | Gets all applied and pending URL lists |
| POST /api/v2/policy/urllist | Creates a new URL list |
| POST /api/v2/policy/urllist/file | Uploads multiple configurations via a JSON file |
| GET /api/v2/policy/urllist/{id} | Gets URL list by ID |
| PUT /api/v2/policy/urllist/{id} | Replaces a URL list configuration |
| DELETE /api/v2/policy/urllist/{id} | Deletes a URL list |
| PATCH /api/v2/policy/urllist/{id}/{action} | Patches a URL list |
| POST /api/v2/policy/urllist/deploy | Applies a pending URL list of changes |
Example:
You can query the URL list followed by a field from a specific URL list. To query specific fields use the following query:
GET http://<tenant-URL>/api/v2/policy/urllist?field=id,name
Available field / query parameters include:
- Id
- name
- data
- pending
- modify_time
- modify_type
- modify_by
Note
The REST API can be used to add to an existing URL list, but not create a new URL list. For more information, refer to Update a URL List.