Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate cloud-bench #155

Merged
merged 3 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .terraform-registry

This file was deleted.

3 changes: 0 additions & 3 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,4 +1 @@
* @sysdiglabs/cloud-native

# compliance
/modules/services/cloud-bench/ @haresh-suresh @nkraemer-sysdig @sysdiglabs/cloud-native
1 change: 0 additions & 1 deletion CONTRIBUTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
- Use **conventional commits** | https://www.conventionalcommits.org/en/v1.0.0
- Current suggested **scopes** to be used within feat(scope), fix(scope), ...
- threat
- bench
- scan
- docs
- tests
Expand Down
136 changes: 9 additions & 127 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,19 @@
# Sysdig Secure for Cloud in GCP
# Sunset Notice

Terraform module that deploys the [**Sysdig Secure for Cloud** stack in **Google Cloud**](https://docs.sysdig.com/en/docs/installation/sysdig-secure-for-cloud/deploy-sysdig-secure-for-cloud-on-gcp/).
<br/>

Provides unified threat-detection, compliance, forensics and analysis through these major components:


* **[Threat Detection](https://docs.sysdig.com/en/docs/sysdig-secure/insights/)**: Tracks abnormal and suspicious activities in your cloud environment based on Falco language. Managed through `cloud-connector` module. <br/>

* **[Compliance](https://docs.sysdig.com/en/docs/sysdig-secure/posture/compliance/compliance-unified-/)**: Enables the evaluation of standard compliance frameworks. Requires both modules `cloud-connector` and `cloud-bench`. <br/>

* **[Image Scanning](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/)**: Automatically scans all container images pushed to the registry (GCR) and the images that run on the GCP workload (currently CloudRun). Managed through `cloud-connector`. <br/>Disabled by Default, can be enabled through `deploy_scanning` input variable parameters.<br/>

For other Cloud providers check: [AWS](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud), [Azure](https://github.com/sysdiglabs/terraform-azurerm-secure-for-cloud)

<br/>
> [!CAUTION]
> Sysdig released a new onboarding experience for GCP in November 2024. We recommend connecting your cloud accounts by [following these instructions](https://docs.sysdig.com/en/docs/sysdig-secure/connect-cloud-accounts/).
>
> This repository should be used solely in cases where Agentless Threat Detection cannot be used.

## Usage

There are several ways to deploy Secure for Cloud in you GCP infrastructure,

- **[`/examples`](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples)** for the most common scenarios
- [Single Project](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples/single-project/)
- [Single Project with a pre-existing Kubernetes Cluster](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples/single-project-k8s/README.md)
- [Organizational](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples/organization/README.md)
- Many module,examples and use-cases, we provide ways to **re-use existing resources (as optionals)** in your
infrastructure. Check input summary on each example/module.
- **[`/use-cases`](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/use-cases)** with self-baked customer-specific alternative scenarios.

Find specific overall service arquitecture diagrams attached to each example/use-case.
- [Single Project](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples/single-project/)
- [Single Project with a pre-existing Kubernetes Cluster](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples/single-project-k8s/README.md)
- [Organizational](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/tree/master/examples/organization/README.md)

In the long-term our purpose is to evaluate those use-cases and if they're common enough, convert them into examples to make their usage easier.

If you're unsure about what/how to use this module, please fill the [questionnaire](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/blob/master/use-cases/_questionnaire.md) report as an issue and let us know your context, we will be happy to help.


### Notice
* [GCP regions](https://cloud.google.com/compute/docs/regions-zones/#available)
* Do not confuse required `region` with GCP location or zone. [Identifying a region or zone](https://cloud.google.com/compute/docs/regions-zones/#identifying_a_region_or_zone)
* All Sysdig Secure for Cloud features but [Image Scanning](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/) are enabled by default. You can enable it through `deploy_scanning` input variable parameter of each example.<br/>
* For **free subscription** users, beware that organizational examples may not deploy properly due to the [1 cloud-account limitation](https://docs.sysdig.com/en/docs/administration/administration-settings/subscription/#cloud-billing-free-tier). Open an Issue so we can help you here!
* This example will create resources that **cost money**. Run `terraform destroy` when you don't need them anymore.
* For a normal load, it should be <150$/month aprox.
* [Cloud Logging API](https://cloud.google.com/service-usage/docs/enabled-service#default) is activated by default so no extra cost here
* Cloud Run instance comes as the most expensive service. Default [cpu/memory specs](https://github.com/sysdiglabs/terraform-google-secure-for-cloud/blob/master/modules/services/cloud-connector/variables.tf#L73-L83), for an ingestion of 35KK events/hour, for 2 instances 24x7 usage
* Cloud Run ingests events from a pub/sub topic, with no retention. It's cost is quite descpreciable, but you can check with the calculator based on the events of the Log Explorer console and 4KB of size per event aprox.<br/>Beware that the logs we consume are scoped to the projects, and we exclude kubernetes events `logName=~"^projects/SCOPED_PROJECT_OR_ORG/logs/cloudaudit.googleapis.com"`
<br/>
If you're unsure about how to use this module, please contact your Sysdig representative. Our experts will guide you through the process and assist you in setting up your account securely and correctly.

## Prerequisites

Expand Down Expand Up @@ -84,16 +52,6 @@ Besides, the following GCP **APIs must be enabled** ([how do I check it?](#q-how
* [Cloud Build API](https://console.cloud.google.com/marketplace/product/google/cloudbuild.googleapis.com)
* [Identity and access management API](https://console.cloud.google.com/marketplace/product/google/iam.googleapis.com)

##### Cloud Benchmarks
* [Identity and access management API](https://console.cloud.google.com/marketplace/product/google/iam.googleapis.com)
* [IAM Service Account Credentials API](https://console.cloud.google.com/marketplace/product/google/iamcredentials.googleapis.com)
* [Cloud Resource Manager API](https://console.cloud.google.com/marketplace/product/google/cloudresourcemanager.googleapis.com)
* [Security Token Service API](https://console.cloud.google.com/marketplace/product/google/sts.googleapis.com)
* [Cloud Asset API](https://console.cloud.google.com/marketplace/product/google/cloudasset.googleapis.com)


<br/>

## Confirm the Services are Working

Check official documentation on [Secure for cloud - GCP, Confirm the Services are working](https://docs.sysdig.com/en/docs/installation/sysdig-secure-for-cloud/deploy-sysdig-secure-for-cloud-on-gcp/#confirm-the-services-are-working)
Expand Down Expand Up @@ -128,8 +86,6 @@ It may take some time, but you should see logs detecting the new image in the `c

And a CloudBuild being launched successfully.

<br/>

## Troubleshooting

### Q: Module does not find project ID
Expand All @@ -155,29 +111,6 @@ output "me" {
}
```

### Q: In organizaitonal setup, Compliance trust-relationship is not being deployed on our projects

As for 2023 April, organizations with projects under organizational unit folders, is supported with the
[organizational compliance example](./examples/organization-org_compliance)

<br/>S: If you want to target specific projects, you can still use the `benchmark_project_ids` parameter so you can define
the projects where compliance role is to be deployed explicitly.
<br/>You can use the [fetch-gcp-rojects.sh](./resources/fetch-gcp-projects.sh) utility to list organization member projects
<br/>Let us know if this workaround won't be enough, and we will work on implementing a solution.

### Q: Compliance is not working. How can I check everything is properly setup

A: On your GCP infrastructure, per-project where Comliance has been setup, check following points<br/>
1. there is a Workload Identity Pool and associated Workload Identity Pool Provider configured, which must have an ID of `sysdigcloud` (display name doesn't matter)
2. the pool should have a connected service account with the name `sfcsysdigcloudbench`, with the email `[email protected]`
3. this serviceaccount should allow access to the following format `principalset: principalSet://iam.googleapis.com/projects/<PROJECTID>/locations/global/workloadIdentityPools/sysdigcloud/attribute.aws_role/arn:aws:sts::***:assumed-role/***`
4. the serviceaccount should have the `viewer role` on the target project, as well as a custom role containing the "storage.buckets.getIamPolicy", "bigquery.tables.list", "cloudasset.assets.listIamPolicy" and "cloudasset.assets.listResource" permissions
5. the pool provider should allow access to Sysdig's trusted identity, retrieved through
```
$ curl https://<SYSDIG_SECURE_URL>/api/cloud/v2/gcp/trustedIdentity \
--header 'Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>'
```

### Q: Getting "Error creating Service: googleapi: got HTTP response code 404" "The requested URL /serving.knative.dev/v1/namespaces/***/services was not found on this server"

```
Expand All @@ -193,57 +126,6 @@ A: This error is given by the Terraform GCP provider when an invalid region is u
### Q: Error because it cannot resolve the address below, "https://-run.googleapis.com/apis/serving.knative.dev"
A: GCP region was not provided in the provider block

### Q: Why do we need `google-beta` provider?

A: Some resources we use, such as the [`google_iam_workload_identity_pool_provider`](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/iam_workload_identity_pool_provider) are only available in the beta version.<br/>

### Q: Getting "Error creating WorkloadIdentityPool: googleapi: Error 409: Requested entity already exists"<br/>
A: Currently Sysdig Backend does not support dynamic WorkloadPool and it's name is fixed to `sysdigcloud`.
<br/>Moreover, Google, only performs a soft-deletion of this resource.
https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers#delete-pool
> You can undelete a pool for up to 30 days after deletion. After 30 days, deletion is permanent. Until a pool is permanently deleted, you cannot reuse its name when creating a new workload identity pool.<br/>

<br/>S: For the moment, federation workload identity pool+provider have fixed name.
Therea are several options here

- For single-account, in case you want to reuse it, you can make use of the `reuse_workload_identity_pool` attribute available in some
examples.
- For organizational setups, you can make use of a single workload-identity for all the organization, with the [/organization-org_compliance](./examples/organization-org_compliance)
- Alternatively, you can reactivate and import it, into your terraform state manually.
```bash
# re-activate pool and provider
$ gcloud iam workload-identity-pools undelete sysdigcloud --location=global
$ gcloud iam workload-identity-pools providers undelete sysdigcloud --workload-identity-pool="sysdigcloud" --location=global

# import to terraform state
# for this you have to adapt the import resource to your specific usage
# ex.: for single-project, input your project-id
$ terraform import 'module.secure-for-cloud_example_single-project.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool.pool' <PROJECT_ID>/sysdigcloud
$ terraform import 'module.secure-for-cloud_example_single-project.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool_provider.pool_provider' <PROJECT_ID>/sysdigcloud/sysdigcloud

# ex.: for organization example you should change its reference too, per project
$ terraform import 'module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool.pool' <PROJECT_ID>/sysdigcloud
$ terraform import 'module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["<PROJECT_ID>"].google_iam_workload_identity_pool_provider.pool_provider' <PROJECT_ID>/sysdigcloud/sysdigcloud
```

The import resource to use, is the one pointed out in your terraform plan/apply error messsage
```
-- for
Error: Error creating WorkloadIdentityPool: googleapi: Error 409: Requested entity already exists
with module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["org-child-project-1"].google_iam_workload_identity_pool.pool,
on .... in resource "google_iam_workload_identity_pool" "pool":
resource "google_iam_workload_identity_pool" "pool" {

-- use
' module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["org-child-project-1"].google_iam_workload_identity_pool.pool' as your import resource

-- such as
$ terraform import 'module.secure-for-cloud_example_organization.module.cloud_bench[0].module.trust_relationship["org-child-project-1"].google_iam_workload_identity_pool.pool' 'org-child-project-1/sysdigcloud'

```

Note: if you're using terragrunt, run `terragrunt import`

### Q: Getting "Error creating Topic: googleapi: Error 409: Resource already exists in the project (resource=gcr)"
```text
│ Error: Error creating Topic: googleapi: Error 409: Resource already exists in the project (resource=gcr).
Expand Down
Loading
Loading