-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Check other share exists before clean up #769
Conversation
Tested in AWS Deployment: Pre-requisites
Testing Scenarios:
Upon testing, another issue seems to come up when trying to revoke the QS dataall Group for the second time (for the second revoke in the above testing scenario). It fails with the following error message:
With the current implementation of QuickSight using 1 QS Group for all EnvGroups in the environment - I believe we will have to avoid revoking any permissions from the source database/table on target QS Group unless there are no other existing shares in the environment for that dataset - @dlpzx let me know your thoughts on this |
backend/dataall/modules/dataset_sharing/aws/lakeformation_client.py
Outdated
Show resolved
Hide resolved
To test this fix again for the following:
|
There is one remaining thing to fix - we currently only revoke external account access for cross account shares when there are no other share objects with the same target env and source dataset URIs If that is the case we revoke external account access to the target env for all of the The above does not work when there is 2 table shared from the same dataset to the same env (either to the same or different teams). Ultimately this leaves remaining permissions in LakeFormation when revoking a table Instead, we can revoke external access on a tabular level and check for other existing share items with the same tableUri that are shared to the same envUri. Then if there are no other existing shares on that table we can easily revoke external access to the env for that table |
Final Testing - all works below, PR ready for review
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just a couple of minor comments
backend/dataall/modules/dataset_sharing/db/share_object_repositories.py
Outdated
Show resolved
Hide resolved
backend/dataall/modules/dataset_sharing/services/share_managers/lf_share_manager.py
Outdated
Show resolved
Hide resolved
.../dataall/modules/dataset_sharing/services/share_processors/lf_process_cross_account_share.py
Show resolved
Hide resolved
backend/dataall/modules/dataset_sharing/services/share_managers/lf_share_manager.py
Show resolved
Hide resolved
I deployed the feature in AWS and I confirm that:
With Quicksight enabled in envA (dataset A with 2 tables) and envB (groupB1 and groupB2)
@noah-paige after the small cosmetic changes I think this PR is ready to be merged! Nice job |
@noah-paige I think it is good to merge :) Should we go for v2m1m0 or directly to main? If it is merged to main it needs to include #816 to avoid npm-audit issues |
### Feature or Bugfix - Bugfix ### Detail - Upgrade package.json, package-lock.json and yarn.lock and pin `@babel/traverse` to version `7.23.2` to solve found vulnerability reported here: GHSA-67hx-6x53-jw92 ### Relates - This PR is needed for other PRs to pass the github workflows tests ### Security Please answer the questions below briefly where applicable, or write `N/A`. Based on [OWASP 10](https://owasp.org/Top10/en/). - Does this PR introduce or modify any input fields or queries - this includes fetching data from storage outside the application (e.g. a database, an S3 bucket)? - Is the input sanitized? - What precautions are you taking before deserializing the data you consume? - Is injection prevented by parametrizing queries? - Have you ensured no `eval` or similar functions are used? - Does this PR introduce any functionality or component that requires authorization? - How have you ensured it respects the existing AuthN/AuthZ mechanisms? - Are you logging failed auth attempts? - Are you using or adding any cryptographic features? - Do you use a standard proven implementations? - Are the used keys controlled by the customer? Where are they stored? - Are you introducing any new policies/roles/users? - Have you used the least-privilege principle? How? By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Feature - Bugfix - Refactoring ### Detail #### Features * Limit pivot role S3 permissions by @dlpzx in #780 * Limit pivot role KMS permissions by @dlpzx in #830 * Add configurable session timeout to IDP by @manjulaK in #786 * Allow to submit a share when you are both an approver and a requester by @zsaltys in #793 * Redirect upon creating a share request by @zsaltys in #799 * Handle Pre-filtering of tables by @anushka-singh in #811 * Email Notification on Share Workflow - Issue - 734 by @TejasRGitHub in #818 * Refactor notifications from core to modules by @dlpzx in #822 * Add frontend and backend feature flags by @zsaltys in #817 * Make hosted_zone_id optional by @lorchda in #812 #### Fixes * Add Additional Error Messages for KMS Key lookup on imported dataset by @noah-paige in #748 * Handle Environment Import of IAM service roles by @noah-paige in #749 * Build Compliant Names for Opensearch Resources by @noah-paige in #750 * Update Lambda runtime by @nikpodsh in #782 * Ensure valid environments for share request and other objects creation by @dlpzx in #781 * Fix shell true semgrep by @dlpzx in #760 * Add condition when there are no public subnets by @lorchda in #794 * Remove unused variable by @zsaltys in #815 * Check other share exists before clean up by @noah-paige in #769 ### Relates - v2.1.0 minor release ## New Contributors * @manjulaK made their first contribution in #786 * @zsaltys made their first contribution in #793 * @anushka-singh made their first contribution in #811 * @TejasRGitHub made their first contribution in #818 ### Security Please answer the questions below briefly where applicable, or write `N/A`. Based on [OWASP 10](https://owasp.org/Top10/en/). - Does this PR introduce or modify any input fields or queries - this includes fetching data from storage outside the application (e.g. a database, an S3 bucket)? - Is the input sanitized? - What precautions are you taking before deserializing the data you consume? - Is injection prevented by parametrizing queries? - Have you ensured no `eval` or similar functions are used? - Does this PR introduce any functionality or component that requires authorization? - How have you ensured it respects the existing AuthN/AuthZ mechanisms? - Are you logging failed auth attempts? - Are you using or adding any cryptographic features? - Do you use a standard proven implementations? - Are the used keys controlled by the customer? Where are they stored? - Are you introducing any new policies/roles/users? - Have you used the least-privilege principle? How? By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Noah Paige <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: jaidisido <[email protected]> Co-authored-by: mourya-33 <[email protected]> Co-authored-by: nikpodsh <[email protected]> Co-authored-by: MK <[email protected]> Co-authored-by: Zilvinas Saltys <[email protected]> Co-authored-by: Daniel Lorch <[email protected]> Co-authored-by: Anushka Singh <[email protected]> Co-authored-by: trajopadhye <[email protected]>
Feature or Bugfix
Detail
Fix method to detect if other share objects exist on the environment before cleaning up environment-level shared resources (i.e. RAM invitation and PivotRole permissions)
other_approved_share_object_exists
was returningFalse
and deleting necessary permissions for the other existing ShareAlso fixes the removal of dataall QS Group permissions if there are still existing shares to EnvA
Security
NA
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.