Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parametrize test_benchmarks.py test by criterion benchmarks #4832

Open
roypat opened this issue Oct 2, 2024 · 3 comments
Open

Parametrize test_benchmarks.py test by criterion benchmarks #4832

roypat opened this issue Oct 2, 2024 · 3 comments
Labels
Good first issue Indicates a good issue for first-time contributors Priority: Low Indicates that an issue or pull request should be resolved behind issues or pull requests labelled ` Status: Parked Indicates that an issues or pull request will be revisited later

Comments

@roypat
Copy link
Contributor

roypat commented Oct 2, 2024

Currently, test_benchmarks.py contains a single test that runs all criterion benchmarks. With increasing number of benchmarks, we thus increase the duration of this test, and need to adjust its timeout. Instead, we want to parameterize the test by the list of criterion benchmarks we have in the repository. This means introducing a fixture that yields the benchmarks listed by cargo bench --all -- --list individually. Then the test itself would only run the benchmark it is provided. This should also make it easier to see which benchmarks are failing.

Originally posted by @pb8o in #4830 (comment)

@kalyazin kalyazin added Good first issue Indicates a good issue for first-time contributors Priority: Low Indicates that an issue or pull request should be resolved behind issues or pull requests labelled ` Status: Parked Indicates that an issues or pull request will be revisited later labels Oct 9, 2024
@MacOS
Copy link

MacOS commented Oct 18, 2024

If someone has not done it yet, I would take it over.

@bchalios
Copy link
Contributor

Hey @MacOS, please start working on it if you want.

@cm-iwata
Copy link
Contributor

@MacOS
CC: @bchalios

Are you still working on this?
If you are busy, may I handle it instead you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Good first issue Indicates a good issue for first-time contributors Priority: Low Indicates that an issue or pull request should be resolved behind issues or pull requests labelled ` Status: Parked Indicates that an issues or pull request will be revisited later
Projects
None yet
Development

No branches or pull requests

5 participants