Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Check other share exists before clean up #769

Merged
merged 10 commits into from
Oct 18, 2023
Merged

Conversation

noah-paige
Copy link
Contributor

@noah-paige noah-paige commented Sep 15, 2023

Feature or Bugfix

  • Bugfix

Detail

  • Fix method to detect if other share objects exist on the environment before cleaning up environment-level shared resources (i.e. RAM invitation and PivotRole permissions)

    • Originally, if TeamA in EnvA had 2 shares approved and succeeded on DatasetB and TeamA rejects 1 of the pre-existing shares, the method other_approved_share_object_exists was returning Falseand deleting necessary permissions for the other existing Share
    • Also disables the other existing shares ability to Revoke the still existing share since pivotRole no longer has permissions
  • Also fixes the removal of dataall QS Group permissions if there are still existing shares to EnvA

Security

NA

Please answer the questions below briefly where applicable, or write `N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
  - Is the input sanitized?
  - What precautions are you taking before deserializing the data you consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
  - How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@noah-paige noah-paige requested a review from dlpzx September 15, 2023 19:29
@noah-paige noah-paige self-assigned this Sep 15, 2023
@noah-paige
Copy link
Contributor Author

noah-paige commented Sep 15, 2023

Tested in AWS Deployment:

Pre-requisites

  • Dataset X owned by Team X in Env X
  • Team A and Team B in Env Y (w/ dashboards enabled)

Testing Scenarios:

  • Team A Requests Access to Dataset X Table and Folder in Env Y

    • Share Succeeds
  • Team B Requests Access to Dataset X Table and Folder in Env Y

    • Share Succeeds
  • Team B Revokes Share in Env Y

    • Revoke Succeeds
    • Team B loses access to data
    • Team A still has access to table and folder
  • Team A Revokes Share in Env Y

    • Revoke Succeeds
    • Access Revoked to External Account (Env Y) since no other shares exist
    • Works only when Requestor Enviornment has Dashboards Disabled

Upon testing, another issue seems to come up when trying to revoke the QS dataall Group for the second time (for the second revoke in the above testing scenario). It fails with the following error message:

An error occurred (InvalidInputException) when calling the RevokePermissions operation: No permissions revoked. Grantee has no permissions and no grantable permissions on resource.

With the current implementation of QuickSight using 1 QS Group for all EnvGroups in the environment - I believe we will have to avoid revoking any permissions from the source database/table on target QS Group unless there are no other existing shares in the environment for that dataset - @dlpzx let me know your thoughts on this

@noah-paige
Copy link
Contributor Author

noah-paige commented Sep 20, 2023

To test this fix again for the following:

  • 1 Table Shared 1 Time Cross Account to EnvA (QS enabled)

    • Share Succeeds with Proper LF Permissions
    • Share is Revoked Successfully (no more permissions for Team Role, QS Group, + Pivot Role in EnvA)
  • 1 Table Shared 2 Times Cross Account to EnvA (QS enabled)

    • Both Shares Succeed
    • When 1 Share Revoke - no permissions for Team1 but QS Group and Team2 still can access Table in EnvA
    • Second Share Revoked Successfully
  • 2 Tables in Same Dataset Shared 1 Time Each to Cross Account to EnvA (QS enabled)

    • Both Shares Succeed
    • When 1 Share Revoke - no more permissions for Team Role, QS Group, + Pivot Role in EnvA for Table1 (still permissions for other table)
    • Second Share Revoked Successfully

@noah-paige
Copy link
Contributor Author

There is one remaining thing to fix - we currently only revoke external account access for cross account shares when there are no other share objects with the same target env and source dataset URIs

If that is the case we revoke external account access to the target env for all of the self.revoked_tables in the revoke request

The above does not work when there is 2 table shared from the same dataset to the same env (either to the same or different teams). Ultimately this leaves remaining permissions in LakeFormation when revoking a table

Instead, we can revoke external access on a tabular level and check for other existing share items with the same tableUri that are shared to the same envUri. Then if there are no other existing shares on that table we can easily revoke external access to the env for that table

@noah-paige
Copy link
Contributor Author

Final Testing - all works below, PR ready for review

  • 1 Table Shared 1 Time Cross Account to EnvA (QS enabled)

    • Share Succeeds with Proper LF Permissions
    • Share is Revoked Successfully (no more permissions for Team Role, QS Group, + Pivot Role in EnvA)
  • 1 Table Shared 2 Times Cross Account to EnvA (QS enabled)

    • Both Shares Succeed
    • When 1 Share Revoke - no permissions for Team1 but QS Group and Team2 still can access Table in EnvA
    • Second Share Revoked Successfully
  • 2 Tables in Same Dataset Shared 1 Time Each to Cross Account to EnvA (QS enabled)

    • Both Shares Succeed
    • When 1 Share Revoke - no more permissions for Team Role, QS Group, + Pivot Role in EnvA for Table1 (still permissions for other table)
    • Second Share Revoked Successfully

Copy link
Contributor

@dlpzx dlpzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a couple of minor comments

@dlpzx
Copy link
Contributor

dlpzx commented Oct 11, 2023

I deployed the feature in AWS and I confirm that:

  • CICD completes successfully

With Quicksight enabled in envA (dataset A with 2 tables) and envB (groupB1 and groupB2)

  • 1 table, 1 group
    • share table1 with groupB1 succeeds, proper LF permissions
    • revoke table1 for groupB1 cleans up shared-db and permissions source to target account
  • 1 table, 2 groups
    • share table1 with groupB1 and B2 succeeds, proper LF permissions (permissions source to target account, shared-db1, shared-db2, permissions in shared-dbs)
    • revoke table1 for groupB1 keeps shared resources for B2 (permissions source to target account, shared-db2, permissions in shared-db2)
    • revoke table1 for groupB2 cleans up everything
  • 2 tables, 1 group
    • share table1 and table2 with groupB1 succeeds, proper LF permissions (permissions source to target account for table1 and table2, shared-db1, permissions in shared-db1 for table1 and 2)
    • revoke table1 for groupB1 keeps shared resources for TABLE2 (permissions source to target account to table2 only, shared-db1, permissions in shared-db1 for table2 only)
    • revoke table2 for groupB1 cleans up everything

@noah-paige after the small cosmetic changes I think this PR is ready to be merged! Nice job

@dlpzx dlpzx added the type: bug Something isn't working label Oct 11, 2023
@dlpzx
Copy link
Contributor

dlpzx commented Oct 18, 2023

@noah-paige I think it is good to merge :) Should we go for v2m1m0 or directly to main? If it is merged to main it needs to include #816 to avoid npm-audit issues

@dlpzx dlpzx changed the base branch from main to v2m1m0 October 18, 2023 06:50
### Feature or Bugfix
- Bugfix

### Detail
- Upgrade package.json, package-lock.json and yarn.lock and pin
`@babel/traverse` to version `7.23.2` to solve found vulnerability
reported here: GHSA-67hx-6x53-jw92

### Relates
- This PR is needed for other PRs to pass the github workflows tests

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@noah-paige noah-paige merged commit 6cc564e into v2m1m0 Oct 18, 2023
@dlpzx dlpzx added this to the v2.1.0 milestone Oct 30, 2023
@dlpzx dlpzx removed this from the v2.1.0 milestone Oct 30, 2023
@dlpzx dlpzx mentioned this pull request Oct 30, 2023
dlpzx added a commit that referenced this pull request Nov 8, 2023
### Feature or Bugfix
- Feature
- Bugfix
- Refactoring

### Detail

#### Features
* Limit pivot role S3 permissions by @dlpzx in
#780
* Limit pivot role KMS permissions by @dlpzx in
#830
* Add configurable session timeout to IDP by @manjulaK in
#786
* Allow to submit a share when you are both an approver and a requester
by @zsaltys in #793
* Redirect upon creating a share request by @zsaltys in
#799
* Handle Pre-filtering of tables by @anushka-singh in
#811
* Email Notification on Share Workflow - Issue - 734 by @TejasRGitHub in
#818
* Refactor notifications from core to modules by @dlpzx in
#822
* Add frontend and backend feature flags by @zsaltys in
#817
* Make hosted_zone_id optional by @lorchda in
#812

#### Fixes
* Add Additional Error Messages for KMS Key lookup on imported dataset
by @noah-paige in #748
* Handle Environment Import of IAM service roles by @noah-paige in
#749
* Build Compliant Names for Opensearch Resources by @noah-paige in
#750
* Update Lambda runtime by @nikpodsh in
#782
* Ensure valid environments for share request and other objects creation
by @dlpzx in #781
* Fix shell true semgrep by @dlpzx in
#760
* Add condition when there are no public subnets by @lorchda in
#794
* Remove unused variable by @zsaltys in
#815
* Check other share exists before clean up by @noah-paige in
#769


### Relates
- v2.1.0 minor release

## New Contributors
* @manjulaK made their first contribution in
#786
* @zsaltys made their first contribution in
#793
* @anushka-singh made their first contribution in
#811
* @TejasRGitHub made their first contribution in
#818

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Noah Paige <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: jaidisido <[email protected]>
Co-authored-by: mourya-33 <[email protected]>
Co-authored-by: nikpodsh <[email protected]>
Co-authored-by: MK <[email protected]>
Co-authored-by: Zilvinas Saltys <[email protected]>
Co-authored-by: Daniel Lorch <[email protected]>
Co-authored-by: Anushka Singh <[email protected]>
Co-authored-by: trajopadhye <[email protected]>
@dlpzx dlpzx deleted the fix-other-share-exist branch November 8, 2023 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants