[Digital Guardian] Initial Release of Digital Guardian (elastic#10617)

Initial Release of Digital Guardian.

New integration to pull events and alerts from the 
Digital Guardian Analytics & Reporting Cloud (ARC) API.

.github/CODEOWNERS

@@ -158,6 +158,7 @@
 /packages/darktrace @elastic/security-service-integrations
 /packages/ded @elastic/ml-ui @elastic/sec-applied-ml
 /packages/dga @elastic/ml-ui @elastic/sec-applied-ml
+/packages/digital_guardian @elastic/security-service-integrations
 /packages/docker @elastic/obs-cloudnative-monitoring
 /packages/elastic_agent @elastic/elastic-agent
 /packages/elastic_package_registry @elastic/ecosystem

packages/digital_guardian/_dev/build/build.yml

@@ -0,0 +1,3 @@
+dependencies:
+  ecs:
+    reference: "[email protected]"

packages/digital_guardian/_dev/build/docs/README.md

@@ -0,0 +1,74 @@
+# Digital Guardian
+
+This integration is for ingesting events and alerts from [Fortra's Digital Guardian](https://www.digitalguardian.com/). Fortra’s Digital Guardian helps organizations protect data, performing across the corporate network, traditional endpoints, and cloud applications. Digital Guardian's data loss prevention, available as a software-as-a-service or managed service, helps to see that data, support compliance initiatives, and protect against serious risk. 
+
+The integration allows collection of events and alerts from [Digital Guardian Analytics & Reporting Cloud (ARC)](https://www.digitalguardian.com/blog/new-dawn-dlp-digital-guardian-releases-its-analytics-reporting-cloud-arc) via the REST API.
+
+## Data streams
+
+The Digital Guardian integration collects events to populate following data-streams:
+
+- `digital_guardian.arc`: Collects all events and alerts from `Digital Guardian Analytics & Reporting Cloud (ARC)` via the REST API.
+
+## Requirements
+
+Elastic Agent must be installed. For more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
+
+### Installing and managing an Elastic Agent:
+
+You have a few options for installing and managing an Elastic Agent:
+
+### Install a Fleet-managed Elastic Agent (recommended):
+
+With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.
+
+### Install Elastic Agent in standalone mode (advanced users):
+
+With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.
+
+### Install Elastic Agent in a containerized environment:
+
+You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.
+
+There are some minimum requirements for running Elastic Agent and for more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
+
+## Setup
+
+### Digital Guardian ARC
+
+#### Copy Digital Guardian ARC required configuration properties:
+
+1. Copy `Client ID`: From ARC Tenant Settings, copy the Tenant ID.
+2. Copy `Client Secret`: From ARC Tenant Settings, copy the Authentication Token.
+3. Copy `ARC Server URL`: From Digital Guardian Management Console (DGMC), copy the Access Gateway Base URL.
+4. Copy `Authorization Server URL`: From Digital Guardian Management Console (DGMC), copy the Authorization server URL.
+5. Copy `ARC Export Profile ID`: 
+    - Navigate to `Admin > reports > export profiles`
+    - Copy only the GUID part from the export profile.
+
+#### Enabling the Digital Guardian integration in Elastic with ARC dataset:
+
+1. In Kibana go to Management > Integrations.
+2. In "Search for integrations" search bar, type Digital Guardian.
+3. Click on the "Digital Guardian" integration from the search results.
+4. Click on the "Add Digital Guardian" button to add the integration.
+5. Configure all required integration parameters. 
+    - ARC data requires following parameters:
+        - `Client ID`
+        - `Client Secret`
+        - `ARC Server URL`
+        - `Authorization Server URL`
+        - `ARC Export Profile ID`
+6. Save the integration.
+
+## Logs reference
+
+### arc
+
+This is the `arc` dataset.
+
+#### Example
+
+{{event "arc"}}
+
+{{fields "arc"}}

packages/digital_guardian/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,14 @@
+version: "2.3"
+services:
+  dg-arc:
+    image: docker.elastic.co/observability/stream:v0.15.0
+    ports:
+      - 8080
+    volumes:
+      - ./files:/files:ro
+    environment:
+      PORT: 8080
+    command:
+      - http-server
+      - --addr=:8080
+      - --config=/files/config.yml

packages/digital_guardian/_dev/deploy/docker/files/config.yml

@@ -0,0 +1,263 @@
+rules:
+  - path: /as/token.oauth2
+    methods: ['POST']
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - 'application/json'
+        body: |
+          {"access_token":"xxxx","expires_in":3600,"token_type":"Bearer"}
+  - path: /rest/1.0/export_profiles/abc123/export_and_ack
+    methods: ['POST']
+    request_headers:
+      Accept:
+        - "application/json"
+    responses:
+      - status_code: 200
+        body: |-
+          {{ minify_json `
+          {
+            "fields": [
+              {
+                "array": false,
+                "name": "dg_name",
+                "type": "string",
+                "display_name": "Name",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "inc_creator",
+                "type": "string",
+                "display_name": "Creator",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "inc_assign",
+                "type": "string",
+                "display_name": "Assignee",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "inc_state",
+                "type": "string",
+                "display_name": "Incident State",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "dg_description",
+                "type": "string",
+                "display_name": "Description",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "inc_id",
+                "type": "string",
+                "display_name": "Incident ID",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "dg_comment",
+                "type": "string",
+                "display_name": "Comment",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "inc_mtime",
+                "type": "timestamp",
+                "display_name": "Modified Time",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "dg_guid",
+                "type": "string",
+                "display_name": "Unique ID",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "inc_sev",
+                "type": "int",
+                "display_name": "Severity",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "dg_utype",
+                "type": "string",
+                "display_name": "Operation Type",
+                "tags": [],
+                "encrypt": false
+              },
+              {
+                "array": false,
+                "name": "dg_tenant",
+                "type": "guid",
+                "display_name": "Tenant ID",
+                "tags": [],
+                "encrypt": false
+              }
+            ],
+            "data": [
+              [
+                "test has attached a Salesforce data to an email",
+                "dg",
+                "test@dgdemo",
+                "Created",
+                "This file outlook.exe was going to [[email protected]]",
+                "230523-WIQHA",
+                "-",
+                "2023-05-23 06:56:39",
+                "1dc3c1fa-5474-4fc0-a7c3-74ff42d28e5e",
+                "Critical",
+                "Incident",
+                "279b59f3-02f3-44ea-a7c3-9bac2eb0224d"
+              ],
+              [
+                "Demo 10",
+                "demo@dgdemo",
+                "demo@dgdemo",
+                "Escalated",
+                "-",
+                "230523-RG0AB",
+                "-",
+                "2023-05-23 11:53:11",
+                "c742c377-b429-428a-b0c9-515cbbf143be",
+                "Critical",
+                "Incident",
+                "279b59f3-02f3-44ea-a7c3-9bac2eb0224d"
+              ],
+              [
+                "Demo 11",
+                "demo11@dgdemo",
+                "demo11@dgdemo",
+                "Escalated",
+                "-",
+                "230624-RG1AB",
+                "-",
+                "2024-06-23 11:53:11",
+                "a842c377-b429-428a-b0c9-515cbbf133be",
+                "Critical",
+                "Incident",
+                "129b59f3-02f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 12",
+                "demo12@dgdemo",
+                "demo12@dgdemo",
+                "Escalated",
+                "-",
+                "210624-RG1AB",
+                "-",
+                "2024-07-23 12:53:11",
+                "a942b377-b429-428a-b0c9-515cbbf133be",
+                "Critical",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 13",
+                "demo13@dgdemo",
+                "demo13@dgdemo",
+                "Created",
+                "-",
+                "200624-RG1AB",
+                "-",
+                "2024-07-21 10:53:11",
+                "aa42b379-b429-428a-b0c9-515cbbf133be",
+                "Medium",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 14",
+                "demo14@dgdemo",
+                "demo14@dgdemo",
+                "Created",
+                "-",
+                "200524-RG1AB",
+                "-",
+                "2024-07-22 05:43:11",
+                "ba42f379-b329-428a-b0c9-515cbbf133be",
+                "High",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 15",
+                "demo15@dgdemo",
+                "demo15@dgdemo",
+                "Created",
+                "-",
+                "201524-RG3AB",
+                "-",
+                "2024-07-25 05:43:11",
+                "da42fg79-b129-428a-b0c9-515cbbf133be",
+                "Minor",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 16",
+                "demo16@dgdemo",
+                "demo16@dgdemo",
+                "Created",
+                "-",
+                "201424-RG3AB",
+                "-",
+                "2024-07-26 05:43:11",
+                "db42ff79-b129-428a-b0c9-515cbbf133be",
+                "Informational",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 17",
+                "demo17@dgdemo",
+                "demo17@dgdemo",
+                "Created",
+                "-",
+                "201224-RG3AB",
+                "-",
+                "2024-07-28 05:45:11",
+                "db31ff78-b129-428a-b0c9-515cbbf133be",
+                "Informational",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ],
+              [
+                "Demo 18",
+                "demo18@dgdemo",
+                "demo18@dgdemo",
+                "Created",
+                "-",
+                "201214-RG2AB",
+                "-",
+                "2024-07-29 05:41:11",
+                "db21fg78-b129-428a-b0c9-515cbbf133be",
+                "Informational",
+                "Incident",
+                "128b59f3-01f3-44ea-a7c3-9bac2eb0124d"
+              ]
+            ],
+            "total_hits": 10
+          }
+          `}}

packages/digital_guardian/changelog.yml

@@ -0,0 +1,6 @@
+# newer versions go on top
+- version: "0.1.0"
+  changes:
+    - description: Initial release of the package.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/10617

packages/digital_guardian/data_stream/arc/_dev/test/pipeline/test-common-config.yml

@@ -0,0 +1,4 @@
+fields:
+  tags:
+    - preserve_original_event
+    - preserve_duplicate_custom_fields

packages/digital_guardian/data_stream/arc/_dev/test/pipeline/test-dg-arc.log

@@ -0,0 +1,2 @@
+{"dg_comment":"-","dg_description":"This file outlook.exe was going to [[email protected]]","dg_guid":"1dc3c1fa-5474-4fc0-a7c3-74ff42d28e5e","dg_name":"test has attached a Salesforce data to an email","dg_tenant":"279b59f3-02f3-44ea-a7c3-9bac2eb0224d","dg_utype":"Incident","inc_assign":"test@dgdemo","inc_creator":"dg","inc_id":"230523-WIQHA","inc_mtime":"2023-05-23 06:56:39","inc_sev":"Critical","inc_state":"Created"}
+{"dg_comment":"-","dg_description":"-","dg_guid":"c742c377-b429-428a-b0c9-515cbbf143be","dg_name":"Demo 10","dg_tenant":"279b59f3-02f3-44ea-a7c3-9bac2eb0224d","dg_utype":"Incident","inc_assign":"demo@dgdemo","inc_creator":"demo@dgdemo","inc_id":"230523-RG0AB","inc_mtime":"2023-05-23 11:53:11","inc_sev":"Critical","inc_state":"Escalated"}