Constraints not being validated correctly

[stuartgunter]

Forseti Version: 2.25.1
Module: Config Validator
Deployment mechanism: Terraform (based on setup instructions)

I have constraints that work locally with Terraform Validator but not with Forseti Config Validator. This is an example of one of the constraints (there's another based on GCPComputeZoneConstraintV1 that has similar issues):

apiVersion: constraints.gatekeeper.sh/v1alpha1
kind: GCPStorageLocationConstraintV1
metadata:
  name: only_allow_european_storage_locations
  annotations:
    description: Checks Cloud Storage bucket locations against allowed or disallowed
      locations.
spec:
  severity: high
  match:
    target: # {"$ref":"#/definitions/io.k8s.cli.setters.target"}
    - "organization/*"
  parameters:
    mode: "allowlist"
    locations:
    - EU
    exemptions: []

I'm running Forseti on GCE and using git-sync to pull my policy library directly from a git repo. I've confirmed this is working correctly and the right policy files are in the configured location on the server instance. There are no violations after running the scanner, but I can see resources in the CAI model that violate the policy. The logs show that it's finding resources to scan, but it seems to contain a lot of null-related values. This is what a typical log message looks like when CV runs (I've redacted sensitive data e.g. <PROJECT>):

2020-06-17 06:44:52,363 INFO [forseti-security][2.25.1] google.cloud.forseti.scanner.scanners.config_validator_util.validator_client(review): Reviewing 66 assets, content: [name: "//storage.googleapis.com/eu.artifacts.<PROJECT>.appspot.com"
asset_type: "storage.googleapis.com/Bucket"
ancestry_path: "project/<PROJECT>/"
resource {
  version: "v1"
  data {
    fields {
      key: "acl"
      value {
        null_value: NULL_VALUE
      }
    }
    fields {
      key: "billing"
      value {
        null_value: NULL_VALUE
      }
    }
    fields {
      key: "cors"
      value {
        null_value: NULL_VALUE
      }
    }
    fields {
      key: "defaultObjectAcl"
      value {
        null_value: NULL_VALUE
      }
    }
    fields {
      key: "encryption"
      value {
        null_value: NULL_VALUE
      }
    }
    fields {
      key: "etag"
      value {
        string_value: "CAE="
      }
    }
    fields {
      key: "iamConfiguration"
      value {
        struct_value {
          fields {
            key: "bucketPolicyOnly"
            value {
              struct_value {
                fields {

Due to log truncation I'm unable to know exactly which bucket this is referring to, or even whether the null_value permeates throughout the content or just applies to empty maps/lists. I've included an example bucket from the CAI export below:

{"name":"//storage.googleapis.com/forseti-cai-export-<RANDOM_VALUE>","asset_type":"storage.googleapis.com/Bucket","resource":{"version":"v1","discovery_document_uri":"https://www.googleapis.com/discovery/v1/apis/storage/v1/rest","discovery_name":"Bucket","parent":"//cloudresourcemanager.googleapis.com/projects/<PROJECT>","data":{"acl":[],"billing":{},"cors":[],"defaultObjectAcl":[],"encryption":{},"etag":"CAE=","iamConfiguration":{"bucketPolicyOnly":{"enabled":true,"lockedTime":"2020-09-13T09:56:20.402Z"},"uniformBucketLevelAccess":{"enabled":true,"lockedTime":"2020-09-13T09:56:20.402Z"}},"id":"forseti-cai-export-<RANDOM_VALUE>","kind":"storage#bucket","labels":{},"lifecycle":{"rule":[{"action":{"type":"Delete"},"condition":{"age":14,"matchesStorageClass":[]}}]},"location":"US-CENTRAL1","locationType":"region","logging":{},"metageneration":1,"name":"forseti-cai-export-<RANDOM_VALUE>","owner":{},"projectNumber":<PROJECT>,"retentionPolicy":{},"selfLink":"https://www.googleapis.com/storage/v1/b/forseti-cai-export-<RANDOM_VALUE>","storageClass":"STANDARD","timeCreated":"2020-06-15T09:56:20.402Z","updated":"2020-06-15T09:56:20.402Z","versioning":{},"website":{},"zoneAffinity":[]},"location":"us-central1"},"ancestors":["projects/<PROJECT>","folders/<FOLDER>","organizations/<ORG>"],"update_time":"2020-06-15T09:56:20.402Z"}

The bucket clearly violates the policy, so I was expecting to see this reported by the notifier but instead I get:

2020-06-17 07:44:53.559 BST
[forseti-security][2.25.1] google.cloud.forseti.notifier.notifier(run): Resource 'config_validator_violations' has no violations

I've looked for CV logs (following the troubleshooting docs) but nothing appears. I was expecting some information there to help diagnose the problem - or even evidence that it's working correctly and the issue may be elsewhere. Unfortunately no logs mean I'm now unsure how to diagnose the problem - or even whether CV is running/being called at all.

Please could someone help me figure out what might be going wrong here?

[auto-comment]

Thank you for opening an issue. Our team's interrupts engineer will review your issue shortly.

Issue Resolution:

• [Interrupts Engineer] Triage / apply categorization labels
• [Interrupts Engineer] Verify / Reproduce the reported issue
• [Forseti Engineer] Perform root cause analysis
• Forseti Engineer] Add tasks and next steps to resolve this issue.

[gkowalski-google]

Hi @stuartgunter, I did a quick test to see if the location was being mangled by Forseti and it appears that the location is correctly being sent to CV. I actually submitted a PR earlier this year to handle some of these issues, but this is not the case here. Can you tell me what the root resource is for your Forseti deployment (or composite resources)? Just curious if you are targeting an org, or a mix of folders/projects. If you are targeting a project (which it kind of looks like you are based on the log output for the bucket data), then the target for the policy constraint needs to be updated. The default for the policies is organizations/**, but you will need to change this to projects/* if you are targeting a single project. Let me know if that helps at all.

[stuartgunter]

Hi @gkowalski-google

Thanks for looking into this for me. I was targeting a single project using the composite_root_resources property. I deliberately used composite_root_resources (rather than the singular root_resource_id) as I intend on covering a set of projects / folders eventually.

I will try your suggestion and amend the target for the policy constraint and see if that helps. I had assumed that a target of organizations/* would apply to all folders and all projects within all organisations (i.e. effectively a match-all constraint). I can target projects, but that feels like the constraint target is being influenced too much by the deployment configuration of Forseti. I expected that a constraint would work the same regardless of how Forseti has been configured. Ultimately resources must live under a project, so I assumed matching all orgs would cascade down to matching all projects.

It's interesting that the constraint (even when targeting organizations/*) works with Terraform Validator.

I'll test the change you suggested and reply here with more info.

Thanks again for your help

[stuartgunter]

Right, here's a quick update on what I've found based on your advice. Firstly, thanks for the pointers - you're right that the target was the issue, although I discovered that the correct way to specify a target is quite hard to find. There don't appear to be any docs on this (or maybe I just couldn't find them).

I first tried setting the target to projects/* as you described, but this didn't work. There were no errors, but the output reported No violations found.

I then tried setting the target to project/* (note the singular form) and this appeared to work - although it also seems to be an incorrect value for target. There were no errors reported though.

I looked for an authoritative guide on how to specify a target value, but none seem to exist. I did find some tests within the config-validator project that I expected to help me, but many of these variations do not work. Whenever I specify a target using a double asterisk, I get an error. Here's an example drawn from the test cases in that file:

With target set to **/projects/* (based on this test case), I get the following error when running Terraform Validator (version 2020-03-05):

Error: validating: FCV: auditing: core.dependencies.audit:25: eval_builtin_error: re_match: error parsing regexp: missing argument to repetition operator: `*`

For now, I've found that the easiest way to avoid these issues is to not specify a target at all and have the constraint apply to everything (at least, that's what I'm assuming it does). This seems to work with TV and CV.

It would be great to have more detailed guidance on how to work with target values. I'm curious to see how difficult this gets when I introduce other projects that may have different policy requirements. Without a clear direction on target usage, this could become difficult to manage.

[gkowalski-google]

@stuartgunter Thanks for trying that, I believe the glob is intended to work, but it might not work with the version of latest release of Forseti. In the next release of Forseti, the version of CV is going to be updated and I believe this will work with that version. For now, it looks like using the project target or removing it altogether is the best approach.

Config Validator is using the target to match the ancestry path of the resource being scanned. When Forseti is targeting a project instead of an org, then the ancestry path for that resource doesn't have the Organization as part of the path. Sorry for the confusion on this, I thought there was some documentation on this in the Policy Library but I don't see any. I created an issue for this and will try to get to soon, let me know if you need any additional help.

[stuartgunter]

Great, thanks @gkowalski-google. I'll close this issue and keep an eye on the other one you created.

Appreciate all your help.