Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Policy change: don't retry benchmarks that time-out #54

Open
rrnewton opened this issue Oct 16, 2014 · 0 comments
Open

Policy change: don't retry benchmarks that time-out #54

rrnewton opened this issue Oct 16, 2014 · 0 comments

Comments

@rrnewton
Copy link
Member

Benchmarks that timeout are by definition the biggest time consumers... retrying those can be really bad on the bottom line for job runtime.

One option is to not retry on time-out rather than crash. Retrying is more for testing programs that are still buggy and have some chance of crashing.

However, nondetermainistic bugs can also manifest themselves as a livelock. That's one argument for retrying but perhaps having an exponential decay of the timeout, so that the benchmark cannot consume too much total time via retries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant