Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can you help improve isa-l's security score of ossf scorecard? #218

Open
giantcroc opened this issue Jul 11, 2022 · 8 comments
Open

Can you help improve isa-l's security score of ossf scorecard? #218

giantcroc opened this issue Jul 11, 2022 · 8 comments

Comments

@giantcroc
Copy link

Hi! I am very interested in using isa-l in open-source service mesh community like Envoy.
And I have done benchmark of isa-l and zlib, the result looks great. So I want to integrate isa-l into envoy as a kind of gzip lib.
I have ran OSSF scorecard against isa-l to see how it would behave as an envoy dependency. The result is as followed:

RESULTS
-------
Aggregate score: 3.7 / 10

Check scores:
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
|  SCORE  |          NAME          |             REASON             |                 DETAILS                  |                                               DOCUMENTATION/REMEDIATION                                               |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Binary-Artifacts       | no binaries found in the repo  |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#binary-artifacts       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | Branch-Protection      | branch protection not enabled  | Warn: branch protection not              | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#branch-protection      |
|         |                        | on development/release         | enabled for branch 'master'              |                                                                                                                       |
|         |                        | branches                       |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | CI-Tests               | 0 out of 1 merged PRs          |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#ci-tests               |
|         |                        | checked by a CI test -- score  |                                          |                                                                                                                       |
|         |                        | normalized to 0                |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | CII-Best-Practices     | no badge detected              |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#cii-best-practices     |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | Code-Review            | no reviews found               | Warn: no reviews found for commit:       | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#code-review            |
|         |                        |                                | 8b7c1b80b22833e0629ccf00e62fda6b3884c7ae |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 5b1a519ffcdde3707710720286e648e37973c024 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | e297ecae7aa82e0790b294134d97d8515bd47548 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | ad8dce15c6d3f0c7f3d1b486d9c649ed39223b45 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 57846f414f3b28aa9e6b1c115959175b58ad23a0 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | e3783f28f890c9b745b29a83a4cd9faa2ebbb328 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | d3cfb2fb772e375cf2007e484e0a6ec0c6a7c993 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 3b3d7cc47b9c21be4d004d2a53455f0ddee968a2 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 642ef36874c59903f3bdef5964a2f8b9b50f3df0 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 0de83dbff76cae050d415735ec1305aca96d9057 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 78f5c31e66fedab78328e592c49eefe4c2a733df |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | fd83ed19240af037f602f6797c24cbeaccae11de |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 6d17992b6d8e35f9e4dfa15cdc66332faac7e1b0 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 87908c9060a90a2b59914281f0b44b119e55dc09 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 0e6511713823881aa90611627f90294c1a79fdc7 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | f980b366556d785ea7701a529c6d1c3b33d05502 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 998e03bf95d0b84f92659e79a133c79f9afb192f |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 066940a9a78dd208081d15e8ec33ccf536cd0259 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 908726e255a3c2980d6964af085fff4ab6f8fd88 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 94ec6026ce5ec9d163b3552190cdc3d26ffb09ab |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 1db0363c49679494fc9a22aab08255f8017e00a8 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 112dd72c01a25b57748dba74fc4cbbd31e7fc63c |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | cfdd3497d1fe13528a8915f130735cc7955c2d6b |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | d5928e3760cc5150b9734d8ae2b7bdded905e6e7 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 628f4e91eaa8dc8bb4be3659fba596c9f47020ec |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 0f7bf1c04d91d846b8f1dff13e0fc2ea5a1ae8e5 |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 393f69fcaccac732b28845e570279b2ec00fb71e |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | 240ca46ffbb8aa2a0d45f3f50d991dbb3fd23d8d |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | d7bac36be47851239295d8ac884b822cc8b321fd |                                                                                                                       |
|         |                        |                                | Warn: no reviews found for commit:       |                                                                                                                       |
|         |                        |                                | fe4b7f9acc6850f15a7385e97bedb9f8cd03442c |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Contributors           | 5 different organizations      | Info: contributors work for              | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#contributors           |
|         |                        | found -- score normalized to   | LUMC,arm,intel,lumc,spdk                 |                                                                                                                       |
|         |                        | 10                             |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Dangerous-Workflow     | no dangerous workflow patterns |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#dangerous-workflow     |
|         |                        | detected                       |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | Dependency-Update-Tool | no update tool detected        | Warn: dependabot config                  | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#dependency-update-tool |
|         |                        |                                | file not detected in source              |                                                                                                                       |
|         |                        |                                | location. 			We recommend setting           |                                                                                                                       |
|         |                        |                                | this configuration in code               |                                                                                                                       |
|         |                        |                                | so it can be easily verified             |                                                                                                                       |
|         |                        |                                | by others. Warn: renovatebot             |                                                                                                                       |
|         |                        |                                | config file not detected in              |                                                                                                                       |
|         |                        |                                | source location. 			We recommend            |                                                                                                                       |
|         |                        |                                | setting this configuration               |                                                                                                                       |
|         |                        |                                | in code so it can be easily              |                                                                                                                       |
|         |                        |                                | verified by others.                      |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | Fuzzing                | project is not fuzzed          |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#fuzzing                |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | License                | license file detected          | Info: : LICENSE:1                        | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#license                |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 1 / 10  | Maintained             | 2 commit(s) out of 30 and 0    |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#maintained             |
|         |                        | issue activity out of 30 found |                                          |                                                                                                                       |
|         |                        | in the last 90 days -- score   |                                          |                                                                                                                       |
|         |                        | normalized to 1                |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| ?       | Packaging              | no published package detected  | Warn: no GitHub publishing               | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#packaging              |
|         |                        |                                | workflow detected                        |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 7 / 10  | Pinned-Dependencies    | dependency not pinned by hash  | Warn: GitHub-owned                       | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#pinned-dependencies    |
|         |                        | detected -- score normalized   | GitHubAction not                         |                                                                                                                       |
|         |                        | to 7                           | pinned by hash:                          |                                                                                                                       |
|         |                        |                                | .github/workflows/ci.yml:15              |                                                                                                                       |
|         |                        |                                | Warn: GitHub-owned                       |                                                                                                                       |
|         |                        |                                | GitHubAction not                         |                                                                                                                       |
|         |                        |                                | pinned by hash:                          |                                                                                                                       |
|         |                        |                                | .github/workflows/ci.yml:34              |                                                                                                                       |
|         |                        |                                | Warn: GitHub-owned                       |                                                                                                                       |
|         |                        |                                | GitHubAction not                         |                                                                                                                       |
|         |                        |                                | pinned by hash:                          |                                                                                                                       |
|         |                        |                                | .github/workflows/ci.yml:56              |                                                                                                                       |
|         |                        |                                | Warn: third-party GitHubAction           |                                                                                                                       |
|         |                        |                                | not pinned by hash:                      |                                                                                                                       |
|         |                        |                                | .github/workflows/ci.yml:58              |                                                                                                                       |
|         |                        |                                | Warn: third-party GitHubAction           |                                                                                                                       |
|         |                        |                                | not pinned by hash:                      |                                                                                                                       |
|         |                        |                                | .github/workflows/ci.yml:60              |                                                                                                                       |
|         |                        |                                | Info: Dockerfile dependencies            |                                                                                                                       |
|         |                        |                                | are pinned Info: no insecure             |                                                                                                                       |
|         |                        |                                | (not pinned by hash)                     |                                                                                                                       |
|         |                        |                                | dependency downloads found               |                                                                                                                       |
|         |                        |                                | in Dockerfiles Info: no                  |                                                                                                                       |
|         |                        |                                | insecure (not pinned by hash)            |                                                                                                                       |
|         |                        |                                | dependency downloads found in            |                                                                                                                       |
|         |                        |                                | shell scripts                            |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | SAST                   | SAST tool is not run on all    | Warn: 0 commits out of 1 are             | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#sast                   |
|         |                        | commits -- score normalized to | checked with a SAST tool Warn:           |                                                                                                                       |
|         |                        | 0                              | CodeQL tool not detected                 |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | Security-Policy        | security policy file not       |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#security-policy        |
|         |                        | detected                       |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| ?       | Signed-Releases        | no releases found              | Warn: no GitHub releases found           | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#signed-releases        |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10  | Token-Permissions      | non read-only tokens detected  | Warn: no top level                       | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#token-permissions      |
|         |                        | in GitHub workflows            | permission defined:                      |                                                                                                                       |
|         |                        |                                | .github/workflows/ci.yml:1               |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Vulnerabilities        | no vulnerabilities detected    |                                          | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#vulnerabilities        |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| ?       | Webhooks               | check is not supported for     | Warn: SCORECARD_V6 is not set,           | https://github.com/ossf/scorecard/blob/3957460c2b2070d30e6c62cfed1ff754a8d388a9/docs/checks.md#webhooks               |
|         |                        | this request: SCORECARD_V6     | not running the Webhook check            |                                                                                                                       |
|         |                        | is not set, not running the    |                                          |                                                                                                                       |
|         |                        | Webhook check                  |                                          |                                                                                                                       |
|---------|------------------------|--------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|

The score is a bit low. However, many of the issues could be easily improved in isa-l. For example, adding a security policy is easy and important. Also setting branch protection and token permissions should just be a github-level thing.

Can you help fix some very simple issues to improve isa-l's security score of ossf scorecard? Thanks!

@gbtucker
Copy link
Contributor

Hi @giantcroc thanks for the post and thanks for integrating isa-l. Glad to know you found a fit in Envoy.

We take security seriously so we are always interested on how to improve. Let's look at each of these. Please let us know if this spurs any ideas.

Add security policy: This one we should be able to put into a file as we already have a policy of course as part of our security lifecycle.

Branch-Protection: I'll have to look into this one further. I think the org only has the permissions to change branch permissions on the main branch.

CI-Tests: We have two levels of CI testing and it looks like the ossf tool is not recognizing either of them. The first is internal and perhaps difficult to register even though every commit and PR goes through this stage. The second is with github actions but not getting detected I suspect because of necessary rebasing to get the gerrit ID added for the first stage. This also seems to be an issue with the next item

Code-Review: We have two levels of code review also and these also are not getting picked up. Looking at the first three commit ids listed, they were all PRs with multiple reviews in github but not detected by the ossf tool. Again I think this is because of the necessary rebase to get the gerrit ID added for the first review step. Perhaps we can promote PRs to add the gerrit ID and fewer will need to be rebased.

CII-Best-Practices: Looking at the list of criteria we should already comply so perhaps we can enroll.

Dependency-Update-Tool: isa-l is a library without any other runtime library dependencies so dependabot or renovatebot are probably not beneficial.

Fuzzing: We do employ fuzz tools where appropriate (compression and decompression) but they are not detected by the tool.

Maintained: Isa-l is actively maintained and not sure why we miss the activity trigger levels.

Token-Permissions: If I understand this correctly the only token we use in ci github actions is the main actions/[email protected]. Is it better to pick a version in this case and perhaps get bug fixes or pin to a hash value?

Pinned-Dependencies: As before we really don't have any dependencies so nothing to pin.

SAST: We do run static analysis as part of CI testing just none of the tools that ossf detects.

Signed-Releases: We don't push any binaries ourselves so nothing to sign. Not sure how we get 10 on Binary-Artifacts for this same reason but not here.

@giantcroc
Copy link
Author

@gbtucker Hi, thanks for your help! I think it is enough to make the score bigger than 5.0, and there is no need to fix all items listed, so we can choose some eaiser one to improve the score.

  1. Add security policy: the simplest one, just place a security policy file SECURITY.md in the root directory of the repository.
  2. Branch-Protection: we can add some constraints on main branch like disabling force pushes and allow deletion.
  3. Token-Permissions: I don't konw if it's possible to make token read-only. If it's ok, we can add permissions for the GITHUB_TOKEN as add permissions.

After finish fixing these items above, I think the score should be able to reach 5.0 or more. Thanks!

@rhpvorderman
Copy link
Contributor

@giantcroc. I have integrated ISA-L as a dependency in some other projects and my experiences are as follows:

  1. ISA-L not having dependencies is great. This means ISA-L never causes a dependency conflict.
  2. ISA-L provides only one feature over zlib: speed. This means that functionally-wise, it is redundant. Which in turn means you can add the dependency by adding optional code path for speed instead of building your whole library on top of ISA-L.

Reason 1 means ISA-L is very unlikely to become a liability in your project and reason 2 means that if for some reason it does, there is very little cost to removing the code that uses ISA-L. The speed benefits are non-trivial so integrating is worth it when compression/decompression is a bottleneck.

@giantcroc
Copy link
Author

@rhpvorderman Thanks! Your sharing is very helpful to me.
And I think the most difficult thing is building nasm in bazel because envoy uses bazel as the build tool.
Do you have any experience to build isa-l using bazel?

@rhpvorderman
Copy link
Contributor

  1. I build nasm first: https://github.com/pycompression/python-isal/blob/develop/.github/workflows/ci.yml#L168
  2. Then I simply use the commands as provided on the readme:
./autogen.sh
./configure --prefix /somewhere/in/build/directory
make
make install

You need to statically link isa-l_static.lib.

@gbtucker
Copy link
Contributor

Added security policy file in 2bcbaf4.

@giantcroc
Copy link
Author

  1. I build nasm first: https://github.com/pycompression/python-isal/blob/develop/.github/workflows/ci.yml#L168
  2. Then I simply use the commands as provided on the readme:
./autogen.sh
./configure --prefix /somewhere/in/build/directory
make
make install

You need to statically link isa-l_static.lib.

OK, thanks!

@giantcroc
Copy link
Author

giantcroc commented Jul 19, 2022

Added security policy file in 2bcbaf4.

@gbtucker Look good. Thanks!
I have ran the scorecard again and the score is already up to 4.6.
And I think it's easy to get to 5.0+ with more small improvements.

Aggregate score: 4.6 / 10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants