AWS S3 Events and Alerts Plugin for Log Shipper

Prerequisites

To complete this configuration, you need:

AWS S3 Events, Alerts Plugin Support

This plugin is used to fetch Alerts (DLP, Malware, Policy, Compromised Credential, Malsite, Quarantine, Remediation, Security Assessment, Watchlist, UBA, CTEP, Device, and Content) and Events (Page, Application, Audit, Infrastructure, Network, Incident, Endpoint and Client Status) from the Netskope Tenant.

Permissions

Amazon S3 bucket permissions to the IAM user to send Events and Alerts data to buckets.

• ListBucket
• CreateBucket
• ListAllMyBuckets
• GetBucketPolicy
• GetBucketPublicAccessBlock
• PutEncryptionConfiguration
• PutBucketPublicAccessBlock
• PutBucketPolicy
• GetBucketLocation 
• PutObject

s3_client = boto3.client(
                "s3",
                aws_access_key_id=self.aws_public_key,
                aws_secret_access_key=self.aws_private_key,
                aws_session_token=self.aws_session_token,
                region_name=self.region_name,
                config=Config(proxies=self.proxy, user_agent=self.useragent),
            )

bucket = s3_client.create_bucket(
                        Bucket=bucket_name,
                        CreateBucketConfiguration=location,
                    )

s3_client.upload_file(
                file_name,
                bucket_name,
                object_name,
            )

s3_resource = boto3.resource(
                "s3",
                aws_access_key_id=self.aws_public_key,
                aws_secret_access_key=self.aws_private_key,
                region_name=self.region_name,
                config=Config(proxies=self.proxy, user_agent=self. user-agent),
            )

Performance Matrix

This performance reading is for a Large Stack CE tested on these VM specifications. These readings are added with the consideration that it will ingest around 10K file size in 30 seconds.

User Agent

User-Agent: APN/1.1 (ahq9d89xj9gspapczzdb59goq)

Workflow

Watch a Video

Click play to watch a video.

Create AWS S3 Source Bucket

This bucket will be used in the AWS S3 Source Bucket configuration parameter while configuring the plugin.

1. To Create a Bucket and Set the required policies, log in to the AWS console.

   

2. From All Services, select S3 or search for S3 in the search bar.

   

3. Click Create Bucket.

   

4. Enter Bucket Name, scroll down, and click Create Bucket. Example: netskope-ce-source-bucket.

5. Search for your bucket and click on the bucket name to open it.

   

6. Click Permissions to open the permission tab and to set the policy.

   

7. Click Edit in the Block public access (bucket settings) section. Uncheck all checkboxes and click Save Changes. You’ll be asked for confirmation; confirm it and click Confirm.

   

8. Under the Permissions tab in the bucket, click Edit in the Bucket Policy section.

   

9. Click Policy generator.

   

10. Select S3 Bucket policy as the policy type, add statement details, and generate the policy: 
    Replace the <user-arn> with the user ARN used for accessing the source bucket and <bucket-name> with the source bucket created above. Sample <user-arn>: arn:aws:iam::7111xxxxxxxx:user/xxxxxxxx, and <bucket-name>: netskope-ce-source-bucket
    • Select Type of Policy: S3 Bucket Policy
    • Effect: Allow
    • Principal: <user-arn>
    • Actions:
      1. GetBucketAcl
      2. GetBucketPolicy
    • ARN: arn:aws:s3:::<bucket-name>
    • Click Add Statement.

    

• Scroll back up and add another statement.
  • Select Type of Policy: S3 Bucket Policy
  • Effect: Allow
  • Principal: <user-arn>
  • Actions:
    1. PutObject
    2. PutObjectAcl
  • ARN: arn:aws:s3:::<bucket-name>/*
  • Click Add Statement.

  

  

11. Click Generate Policy. Make sure to copy the policy generated in this step because it will be used in the next step to attach in the source bucket.

    

12. Add this policy to the Textbox. Scroll to the bottom and click Save Changes.

    

Create a Bucket Policy

1. Search for IAM in the search box, and in the left panel, click Policies.
2. Click Create Policy.

   

2. On the JSON tab, enter this policy. Click Next: Tags, and click Next: Review.

   

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": [
                   "s3:GetBucketPublicAccessBlock",
                   "s3:PutEncryptionConfiguration",
                   "s3:PutBucketPublicAccessBlock",
                   "s3:ListAllMyBuckets",
                   "s3:PutBucketPolicy",
                   "s3:CreateBucket",
                   "s3:ListBucket",
                   "s3:GetBucketPolicy",
                   "s3:GetBucketLocation"
                   "s3:PutObject"
               ],
               "Resource": "*"
           }
       ]
   }

3. Enter a Name and click Create Policy.

   

4. Attach this policy to the user. Go to IAM > Users and select the user for which you want to attach a policy. Click Add permissions, and click Add permissions again.

   

5. Select Attach policies directly under Permissions, and then search for and select the policy created in the previous step for the source queue.

   

6. Click Next and then click Add permissions. A policy will be attached to the user.

   

Plugin Authentication Methods

IAM Role Anywhere Configuration

Prerequisites

The AWS Certificate Manager service is required to be enabled to authenticate the plugin using the AWS IAM Roles Anywhere Authentication Method.

Create an IAM Policy

This Policy contains the required permissions for creating a Private CA Certificate (including Permissions for creating a Trust Anchor and Profile) and using IAM Roles Anywhere.

1. Go to Policy Generator and click Add Statement to generate a policy.
2. Select Type of Policy: IAM Policy
   • Effect: Allow
   • AWS Service: AWS Private Certificate Authority
   • Actions:
     • CreateCertificateAuthority
     • DescribeCertificateAuthority
     • GetCertificate
     • GetCertificateAuthorityCertificate
     • GetCertificateAuthorityCsr
     • ImportCertificateAuthorityCertificate
     • IssueCertificate
     • ListCertificateAuthorities
   • ARN: *
   • Click Add Statement.

   

3. Select Type of Policy: IAM Policy
   • Effect: Allow
   • AWS Service: AWS Identity and Access Management (IAM)
   • Actions:
     • AttachRolePolicy
     • CreateAccessKey
     • CreateRole
     • DeleteRole
     • PassRole
   • ARN: *
   • Click Add Statement.

   

4. Select Type of Policy: IAM Policy
   • Effect: Allow
   • AWS Service: AWS Certificate Manager
   • Actions:
     • DescribeCertificate
     • ExportCertificate
     • GetCertificate
     • ListCertificates
     • ListTagsForCertificate
     • RequestCertificate
   • ARN: *
   • Click Add Statement.

   

5. Select Type of Policy: IAM Policy
   • Effect: Allow
   • AWS Service: AWS Identity and Access Management Roles Anywhere
   • Actions:
     • CreateProfile
     • CreateTrustAnchor
     • GetProfile
     • GetTrustAnchor
     • ListProfiles
     • ListTrustAnchors
   • ARN: *
   • Click Add Statement.

   

6. Click Generate Policy.

   

7. Copy the Policy as it is used in the next step for creating the policy required for creating the Private CA certificates.
8. Go to the AWS Console and select IAM from All Services. Click Policies from the left panel, and click Create Policy.

   

8. Copy the policy to the JSON tab, and then click on Next: Tags and Next: Review..

   

9. Enter a name, like netskope-ce-rolesAnywhere-policy, and click Save Changes.

   

Create a Private Certificate Authority

1. Log in to AWS Console.
2. Search for Certificate Manager.

   

3. Click AWS Private CA.
4. Click Create a private CA.

   

5. Select General-purpose for Mode Options.
6. Select Root for CA Type Options.

   

7. Enter the Organization (O).

   

8. Select RSA 2048 for Key Algorithm Options.

   

   

9. Add tags if any (optional).
10. Enable the checkbox in the CA permissions options section.
11. Enable the checkbox in the Pricing section.
12. Click Create to create the CA certificate.

    

    

13. From Actions, select Install.

    

14. Click Confirm and Install.

    

    

Create a Trust Anchor

1. Search for the IAM service, and go to Roles under Access management. Scroll down to Roles Anywhere and select Manage.

   

2. Click Create a Trust anchor.

   

3. Enter the Trust anchor name, like netskope-ce-trust-anchor.

   

4. Select AWS Certificate Manager Private CA (created in the previous steps) as a Certificate authority (CA) source.
5. Add tags if required.
6. Click Create a trust anchor.

   

   

7. Click on the created Trust Anchor and copy the Trust Anchor ARN.

   

Create an IAM Role

1. Go to IAM services in the AWS Console.
2. Click Role under Access management.
3. Click Create Role.

   

4. For the Trusted entity type, select Custom Trust Policy.

   

4. Replace the Custom trust Policy with the below given Trust Policy – this policy contains the permissions for using the Roles Anywhere service:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Principal": {
                   "Service": [
                       "rolesanywhere.amazonaws.com"
                   ]
               },
               "Action": [
                   "sts:AssumeRole",
                   "sts:TagSession",
                   "sts:SetSourceIdentity"
               ]
           }
       ]
   }

   

5. Click Next.
6. In the Permissions policies, select the previously created Bucket Policy.

   

7. Click Next.
8. Provide a Role name, like netskope-ce-roleAnywhere, and add a Description for the role.
9. Click Create role.

   

10. Make a note of the Role ARN as this will be required in the Plugin configuration parameter: Role ARN for the authentication method AWS IAM Roles Anywhere.

    

Create a Profile

1. Select Roles under Access management.
2. Scroll down to Roles Anywhere and click Manage.

   

3. Expand the Setup steps.
4. Click Step 2: Configure roles.
5. Click Configure a profile.

   

6. Enter a Profile name, like netskope-ce-profile.
7. Select the role created in Create IAM Role section (netskope-ce-roleAnywhere).

   

8. Remove the Inline Policy.
9. Click Create profile.

   

   

   

10. Click on the created Profile and copy the Profile ARN.

    

Request a Private Certificate

1. Go to AWS Certificate Manager > Request certificate.
2. Select Request a private certificate.

   

3. Click Next.
4. Select the Certificate authority created in the previous steps.

   

5. Provide a domain name in the Fully Qualified Domain Name field, like netskope-ce.com.
6. Select RSA 2048 for the Key algorithm.

   

7. Add tags if required.
8. Acknowledge the Certificate renewal permissions.
9. Click Request.

   

10. Go to List certificates from the navigation pane of AWS Certificate Manager.
11. Select the certificate created previously.

    

12. Click Export.

    

13. Enter the passphrase. Make a note of the passphrase as it will be required for the Configuration of the AWS S3 Plugin using the AWS IAM Roles Anywhere Authentication method.
14. Click Generate PEM Encoding.

    

15. Download all the Certificates because they won’t be visible again. For new certificates, you will need to Export it again. For More Info visit AWS IAM Role Anywhere

    

Deployed on AWS Configuration

Create a Role

1. Go to IAM services in the AWS Console.
2. Click Create role.

   

3. Select the AWS Service.
4. For Use Case, select EC2.
5. Click Next.

   

6. Select the permission policy created in your Bucket Policy.
7. Click Next.

   

6. Enter a Role Name, like netskope-ce-instance-role, and add a Description. 
7. Click Create Role.

   

   

Assign a Role to an EC2 Instance

1. Open your EC2 instance console.
2. Click Instances under Instances.

   

   

3. Go to Action > Security > Modify IAM Role.

   

4. Select the Role that you created previously (like netskope-ce-instance-role).
5. Click Update IAM Role.

   

Assign a Role to a Fargate Instance

1. Open your CFN script.
2. Get the ExistingECSTaskRole Parameter value if present.

   

3. Go to IAM > Roles.
4. Search for the ExistingECSTaskRole parameter value. Otherwise, search for NetskopeCloudExchangeTaskRole-<CFN name> and select the role.

   

5. Attach the previously created Create Bucket Policy to this role.
6. Click Add permission > Attach policies.

   

7. Search and select the bucket policy previously created (netskope-ce-s3-policy).
8. Click Add permissions.

   

Assign the Role to a K8s Instance

1. Open your Role created for ServiceAccount while creating K8s instance.
2. Attach the Bucket policy created previously.

Configure the AWS S3 Events, Alerts Plugin

1. In Cloud Exchange, go to Settings > Plugin Store.
2. Search for and select the AWS S3 Events, and Alerts v1.2.0 (CLS) plugin box.

   

3. Enter a Configuration Name.

   

5. Click Next and enter the Configuration Parameters:
   • Authentication Method: Select the method to be used for authentication (Deployed on AWS/AWS IAM Roles Anywhere)
   • Private Key: Private Key for decrypting the AWS Private CA Certificate. Required for AWS IAM Roles Anywhere authentication type.
   • Certificate Body: Certificate Body for AWS Public/Private CA Certificate. Required for AWS IAM Roles Anywhere authentication type.
   • Password Phrase: Password Phrase for decrypting the CA Certificate. Required for AWS IAM Roles Anywhere authentication type.
   • Profile ARN: AWS Profile ARN for AWS client authentication. Required for AWS IAM Roles Anywhere authentication type.
   • Role ARN: AWS Role ARN for AWS client authentication. Required for AWS IAM Roles Anywhere authentication type.
   • Trust Anchor ARN: AWS Trust Anchor ARN for AWS client authentication. Required for AWS IAM Roles Anywhere authentication type.
   • AWS S3 Bucket Region Name: AWS S3 Bucket Region Name from where to get the AWS S3 Bucket. Make sure that the region name matches the region in the Profile ARN and Trust Anchor ARN.
   • AWS S3 Bucket Name: AWS S3 Bucket Name in which the data object will be stored.

   

   

   

6. Click Save. Your plugin configuration will be available on the Cloud Log Shipper > Plugins page.

   

Configure a Log Shipper Business Rule for AWS S3 Events, Alerts

1. In Log Shipper, go to the Business Rules. 
2. By default there is a business rule that filters all alerts and events. If you want to filter out any specific type of alert or event, click Create New Rule and configure a new business rule by adding the rule name and filter(s).
3. Click Save.

   

Configure a SIEM Mapping for AWS S3 Events, Alerts

1. Go to SIEM Mappings and click Add SIEM Mapping. 
2. Select the Source plugin (Netskope CLS), the Destination plugin (AWS S3 Events, Alerts), your business rule, and click Save.
3. After the SIEM mapping is added, the data will start to be pulled from the Netskope tenant, transformed, and ingested into the AWS platform.

   

Validate the AWS S3 Events, Alerts Plugin

Validate the Pull

To validate the pulling of indicators from the Netskope tenant.

Go to Logging in Cloud Exchange and search for the pulled logs.

Validate the Push

To validate the plugin workflow, on Netskope Cloud Exchange.

1. Go to Logging and search for ingested events with the filter “message contains ingested”. 
2. The ingested logs will be filtered.

   

   

To validate the push from the AWS S3 platform:

1. Go to AWS S3. On the Search bar, search for and select Buckets.
2. Search the bucket you used while creating the plugin.
3. Click on the bucket name.

Example: awsdemobucket/alerts/feedname=Malware/year=2023/month=11/day=30/hour=9/1701336881_139977276685128.txt

After downloading the file, the alert/event will look like this:

Troubleshooting the AWS S3 Events, Alerts Plugin

Facing issues in the existing plugin configuration after the plugin update

If you’ve recently updated your AWS S3 plugin, there might be an issue where editing the existing plugin configuration leads to a blank Cloud Exchange interface. This problem may occur in Cloud Exchange versions 4.2.0 and 5.0.0 if users try to modify the plugin configuration after the update and navigate to the Configuration Parameters page.

Here’s what you can do on your Cloud Exchange version:

For version 4.2.0, 5.0.0: The only solution available is to delete the current plugin configuration and set up a new one from scratch.

Facing issues while configuring the new plugin 

If you’re creating a new plugin with AWS IAM Roles Anywhere and face an error of 400, the root cause for not being able to save the plugin can be found in the logging section when expanding the logs

What to do:
While expanding the log, you can see At least one of the Trust Anchor ARN, Role ARN, and Profile ARN has a different account ID, so you would have to check that the provided parameters are generated from the same account and have the same region.