Downloading vulnerability DB gets blocked by rate limiter in Trivy 0.56.0

[mgortzak]

Description

I just upgraded Trivy to version 0.56.0, mainly for the fix in #7592. Unfortunately I still get that error in a Bitbucket pipeline.

I deleted the Bitbucket Trivy cache (which seems to me equivalent to a trivy clean --all), that didn't solve the problem. I don't know if I was to fast, the bug was fixed, or... ?

Desired Behavior

That the vulnerability DB was downloaded.

Actual Behavior

A TOOMANYREQUESTS from a the request GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2.

Reproduction Steps

We run Trivy in a Bitbucket pipeline with this command:

trivy filesystem . --format json --output trivy_cve_results_source.json --scanners vuln,misconfig,secret,license --no-progress --debug --timeout 15m

Bitbucket is running on the Docker image maven:3.9.9-eclipse-temurin-21-jammy.

### Target

Filesystem

### Scanner

Vulnerability

### Output Format

JSON

### Mode

Standalone

### Debug Output

```bash
+ trivy image ${IMAGE_VERSION} --format json --output trivy_cve_results_image.json --no-progress --debug
2024-10-03T10:07:51Z	DEBUG	No plugins loaded
2024-10-03T10:07:51Z	DEBUG	Default config file "file_path=trivy.yaml" not found, using built in values
2024-10-03T10:07:51Z	DEBUG	Cache dir	dir="/root/.cache/trivy"
2024-10-03T10:07:51Z	DEBUG	Cache dir	dir="/root/.cache/trivy"
2024-10-03T10:07:51Z	DEBUG	Parsed severities	severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-10-03T10:07:51Z	DEBUG	Ignore statuses	statuses=[]
2024-10-03T10:07:51Z	INFO	[vulndb] Need to update DB
2024-10-03T10:07:51Z	INFO	[vulndb] Downloading vulnerability DB...
2024-10-03T10:07:51Z	INFO	[vulndb] Downloading artifact...	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-03T10:07:51Z	WARN	See https://aquasecurity.github.io/trivy/v0.56/docs/references/troubleshooting#db
2024-10-03T10:07:51Z	FATAL	Fatal error
  - init error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.Run
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:367
  - DB error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.NewRunner
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:119
  - failed to download vulnerability DB:
    github.com/aquasecurity/trivy/pkg/commands/operation.DownloadDB
        /home/runner/work/trivy/trivy/pkg/commands/operation/operation.go:40
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).Download
        /home/runner/work/trivy/trivy/pkg/db/db.go:158
  - failed to download vulnerability DB:
    github.com/aquasecurity/trivy/pkg/db.(*Client).downloadDB
        /home/runner/work/trivy/trivy/pkg/db/db.go:207
  - failed to download artifact from ghcr.io/aquasecurity/trivy-db:2:
    github.com/aquasecurity/trivy/pkg/oci.Artifacts.Download
        /home/runner/work/trivy/trivy/pkg/oci/artifact.go:236
  - OCI repository error:
    github.com/aquasecurity/trivy/pkg/oci.(*Artifact).populate
        /home/runner/work/trivy/trivy/pkg/oci/artifact.go:96
  - 2 errors occurred:
	* GET https://ghcr.io/token?scope=repository%3Aaquasecurity%2Ftrivy-db%3Apull&service=ghcr.io: DENIED: denied
	* GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2: TOOMANYREQUESTS: retry-after: 761.181µs, allowed: 44000/minute

Operating System

Ubuntu 22.04

Version

+ trivy --version
Version: 0.56.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-09-27 12:15:19.89802692 +0000 UTC
  NextUpdate: 2024-09-27 18:15:19.898026629 +0000 UTC
  DownloadedAt: 2024-09-27 13:18:26.035779376 +0000 UTC
Java DB:
  Version: 1
  UpdatedAt: 2024-09-27 08:58:37.220624453 +0000 UTC
  NextUpdate: 2024-09-30 08:58:37.220624023 +0000 UTC
  DownloadedAt: 2024-09-27 13:18:50.878512827 +0000 UTC
Check Bundle:
  Digest: sha256:ef2d9ad4fce0f933b20a662004d7e55bf200987c180e7f2cd531af631f408bb3
  DownloadedAt: 2024-09-27 13:18:52.161650017 +0000 UTC

Checklist

• Run trivy clean --all
• Read the troubleshooting

[mgortzak]

An other pipeline in a different repository did succeed.

[nikpivkin]

Hi @mgortzak !

This fix can only reduce the number of problems with DB downloading, but not get rid of it completely.

[nikpivkin]

Duplicate of #7591