Performances issues with API v2 bulk_search

[tdruez]

The current implementation of the package bulk_search in API v2 is not usable.
It takes over 2 minutes to fetch a single purl, and searching for several purls ends in a timeout.

from timeit import default_timer as timer
import requests

purls = [
    "pkg:alpm/archlinux/[email protected]",
    "pkg:composer/guzzlehttp/[email protected]",
    "pkg:composer/guzzlehttp/[email protected]",
    "pkg:maven/org.apache.commons/[email protected]",
    "pkg:maven/org.elasticsearch/[email protected]",
    "pkg:nuget/[email protected]",
    "pkg:nuget/[email protected]",
    "pkg:nuget/[email protected]",
    "pkg:pypi/[email protected]",
    "pkg:pypi/[email protected]",
    "pkg:pypi/[email protected]",
    "pkg:pypi/[email protected]",
    "pkg:pypi/[email protected]",
    "pkg:pypi/[email protected]",
    "pkg:pypi/[email protected]",
]
data = {
    "purls": purls,
}

# v1, 15 purls
url = "https://public.vulnerablecode.io/api/packages/bulk_search"
start_time = timer()
response = requests.post(url, data)
print(timer() - start_time)  # 7 seconds
print(len(response.json())) # -> 15 entries

# v2, 15 purls
url = "https://public.vulnerablecode.io/api/v2/packages/bulk_search"
start_time = timer()
response = requests.post(url, data) # -> TIMEOUT
print(timer() - start_time)

# v2, single purl
url = "https://public.vulnerablecode.io/api/v2/packages/bulk_search"
data = {"purls": "pkg:pypi/[email protected]"}
start_time = timer()
response = requests.post(url, data)
print(timer() - start_time)  # -> 138 seconds

[yawningwinner]

Hi, I'am interested in helping out, could you please provide more details