merge-cachi2-sboms: re-generate test data, fix uncovered bugs #204
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Reviewer guide:
|
chmeliik
force-pushed
the
merge-cachi2-sboms-tests
branch
2 times, most recently
from
6ef813a to
1214a67
Compare
The previous input SBOMs were created in a mysterious way which is no longer replicable. Re-generate them in a more repeatable way and add a script to do that. Signed-off-by: Adam Cmiel <[email protected]>
chmeliik
force-pushed
the
merge-cachi2-sboms-tests
branch
from
1214a67 to
504d0c0
Compare
|
I noticed the generated SBOMs were changing every time I reran the |
chmeliik
commented
Dec 18, 2024
| purl = component.purl() | ||
| if not purl: | ||
| raise ValueError(f"cachi2 component with no purl? name={component.name()}, version={component.version()}") | ||
| return purl._replace(qualifiers=None, subpath=None).to_string() |
There was a problem hiding this comment.
Just FYI: NamedTuple methods don't follow the same naming scheme as most other python classes. Despite starting with an underscore, the _replace method is public API https://docs.python.org/3/library/collections.html#collections.somenamedtuple._replace
(PackageURL is a NamedTuple)
tnevrlka
reviewed
Dec 18, 2024
sbom-utility-scripts/scripts/merge-cachi2-sboms-script/test_merge_cachi2_sboms.py
Outdated
Show resolved
Hide resolved
sbom-utility-scripts/scripts/merge-cachi2-sboms-script/merge_cachi2_sboms.py
Show resolved
Hide resolved
The SBOM files are too large to inspect by hand. Check the property that we care about most - the components present in the merged SBOM that were not present in the cachi2 SBOM. Note: this uncovers some bugs (the gomod and npm packages shouldn't be there, they're duplicates). To be fixed later. Also disable the flake8 line-length check; it is useless since we already use 'black' for formatting. Signed-off-by: Adam Cmiel <[email protected]>
An SBOM item is anything that has a name, version and purl. Such as a CycloneDX component or SPDX package. Will be used to implement SPDX support more seamlessly. Also set flake8 to run on python 3.12. In Github CI, the check was failing with a syntax error otherwise (generics syntax). Signed-off-by: Adam Cmiel <[email protected]>
Much easier to use than parsing raw purls manually Signed-off-by: Adam Cmiel <[email protected]>
This removes 4 of the 5 false-positive Go modules in the merged SBOM. One remains - Syft reports it *completely* wrong, not much we can do about that. Signed-off-by: Adam Cmiel <[email protected]>
Cachi2 puts the path in the purl subpath, syft puts it in the namespace and name. Signed-off-by: Adam Cmiel <[email protected]>
chmeliik
force-pushed
the
merge-cachi2-sboms-tests
branch
from
504d0c0 to
4e2ef25
Compare
|
@brunoapimentel you're probably the only one with more experience regarding this script if you want to take a look 😄 |
tnevrlka
approved these changes
Dec 19, 2024
|
Gonna merge this, there are more big changes coming 😅 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://issues.redhat.com/browse/STONEBLD-3045
Primarily, the need to do this came from working on SPDX. I needed to get equivalent SPDX SBOMs for the CycloneDX SBOMs in our test data. But nobody knows how those were generated, so I re-generated them in a more repeatable way. And that uncovered some bugs.
See the individual commits for more details