De-Identification for Semi-Structured / Unstructured Data

Overview

Netskope Data Security Posture Management (DSPM) provides data de-identification (DID) capabilities to remediate sensitive data across your organization’s repositories. De-identification transforms sensitive fields, such as Personally Identifiable Information (PII) and Protected Health Information (PHI), using defined masking, hashing, or redaction rules while preserving structural data validity.

In addition to relational databases, Netskope DSPM supports de-identification for semi-structured tabular files stored across network file shares. When you enable de-identification for semi-structured files, the DSPM sidecar parses the file schema, maps detected fields to column definitions, applies configured de-identification rules associated with Entity Data Types (EDTs), and atomically writes the remediated file back.

De-identification operates in one of two ways:

• Overwriting the source at rest: Directly modifies and replaces the original files in the source file share.
• Outputting to a separate destination: Writes the remediated files to an isolated, designated destination file share while keeping original files intact.

To learn more about entity data types, sidecar deployment, and structured de-identification:

• To learn more: Entity Data Types.
• To learn more: DSPM Sidecar Administration Overview.
• To learn more: Apply DSPM Data De-Identification

Supported File Formats and Repositories

Supported File Formats

Netskope DSPM supports de-identification for the following semi-structured and tabular file types:

Supported Repositories

You can configure de-identification for semi-structured content on the following data store type:

• Server Message Block (SMB): Network-attached storage shares and local file directory shares.
  • Note: De-identification for unstructured and semi-structured files is currently available only for SMB file shares.

Prerequisites and Scanning Requirements

Before you enable de-identification on a semi-structured data store, verify the following operational requirements:

• DSPM Sidecar: A connected DSPM sidecar is recommended for optimal processing and localized scanning.
• Service Account Permissions: Standalone data stores do not require cloud IAM policies. However, the account credentials configured in Netskope DSPM to connect the SMB share require write privileges beyond the standard read-only role defined in Connect SMB File Share to DSPM.
  • Overwriting source at rest: The service account must have Change and Read share permissions, along with NTFS Modify, Write, and Read & execute permissions on the source share folder.
  • Outputting to a separate destination: The service account must retain read-only permissions on the source share, and have Change share permissions along with NTFS Write (Create Files / Write Data) permissions on the designated destination share or folder.
• Scan Type Coercion: Enabling de-identification automatically sets the scan mode to Advanced Scan and the sampling rate to 100%. Smart scans and sampling rates below 100% are not permitted on de-identification stores to ensure complete data integrity.

Enable De-Identification on a Data Store

To allow de-identification workflows on a semi-structured data store:

1. In the Netskope tenant UI, go to Settings > DSPM > Data Stores > Data Store Inventory.

2. Select your data store from the list and click Edit.

3. Under the Classification section, ensure the classification toggle is enabled.

4. Select the Remediate De-Identification Violations checkbox. This option appears only on supported data store services.

5. In Scan Settings, the system automatically sets the scan mode to Advanced Scan and the sample size to 100%.

6. Configure your output destination:

   • To overwrite original files at rest, leave the output directory empty.

     Overwriting existing files at rest is irreversible. Overwritten files cannot be recovered.

   • (Optional) To write remediated files to an isolated folder instead of replacing existing files, specify a designated destination SMB path.

7. Click Save.

Enroll Columns for De-Identification

De-identification operates on an explicit opt-in basis per column. Because de-identification overwrites data, this design ensures you make explicit choices and allows you to validate your de-identification rules before they execute. Even when de-identification is active on a data store, files remain unchanged until you explicitly enroll one or more columns.

To enroll columns from the Semi-Structured inventory:

1. In the Netskope tenant UI, go to DSPM > Classification > Classification Management.
2. Click the Semi-Structured tab.
3. Use the filters to locate your data store, file name, or specific entity data type (EDT).
   • Tip: You can also navigate directly to this view from the Files tab by clicking the Review File for De-Identification action icon on any detected file.
4. Review the columns associated with your semi-structured files:
   • File Name: The relative path and name of the file.
   • Column: The detected tabular field or header.
   • Data Type: The identified entity data type (such as US SSN, Email, or Credit Card Number). Click the de-identification rule name (distinct from DLP rules) to review the assigned transformation logic.
   • De-Id Violations: The number of records matching the entity data type. Click the eye icon in the table toolbar to show or hide columns with zero violations.
5. Select the fields you want to remediate:
   • To enroll individual fields, select the checkbox in the Enable De-Identification column next to each target field.
   • To enroll all visible fields at once, select the bulk-selection checkbox in the table header.
6. Click Save.

Remediation and File Integrity

When a scheduled or manual scan runs on an enrolled data store, the DSPM sidecar executes the remediation process.

Atomic Write-Back

File write-back operations are atomic:

• The sidecar processes the source file in memory and validates all field transformations.
• The remediated file is written back to the source SMB share (overwriting in place) or output to the designated separate destination file share, depending on your configuration.
• If a parsing or write error occurs during processing, the operation halts immediately and the source file remains untouched. The system never produces partial, truncated, or half-masked files.

Structural Fidelity

The de-identification engine preserves structural fidelity across all supported semi-structured formats:

• Row and record counts remain identical to the original file.
• Headers, column ordering, delimiters, quote qualifiers, line endings (CRLF or LF), and file encodings are preserved.
• Line widths may change depending on the rule definition and the length of the replacement value used during the overwrite.
• Columns not enrolled for de-identification remain unchanged byte-for-byte.
• Masking algorithms maintain referential consistency across multiple files and datasets when using deterministic rules.

Monitor De-Identification Results

To review remediation results and inspect audit logs:

1. In the Netskope tenant UI, go to DSPM > Classification > Classification Management.
2. Click the Semi-Structured tab and verify that the violation counters reflect the updated state following the scan.
3. In the table, expand any column row to review the execution logs for that specific column.
   • Note: De-identification logs appear only if de-identification has executed for that column.
4. Go to DSPM > Policies > DLP Profiles & Rules to generate and review dedicated scan logs tracking policy matches and operational events.
5. Go to Settings > DSPM > Activity Log to view administrative records, including when de-identification was enabled, columns were enrolled, and scan jobs finished.