Benchmark every release

[paneq]

http://dak1n1.com/blog/14-http-load-generate

[mostlyobvious]

Any idea how to create/where to host reproducible benchmark environment? In other case effort is not justified.

[paneq]

I think we do not need a reproducible benchmark env. Just something that can be executed for every release in current env. I hope it is clear. It would be cool to have separate machines to generate the load etc but I am not gonna cry if I am unable to find such location :)

[mostlyobvious]

So you want to have numbers that cannot be compared. Why bother than?

[paneq]

Compared to what ? I have no idea what you are talking about. Sorry :(

If I execute benchmark of three releases on my local computer then those values can be compared. In case my notebook broke I can execute a benchmark of four releases on my next local computer and still compare those values to get a hint if things are going in proper direction.

What usecase do you have in mind ?

[mostlyobvious]

Ok I have misunderstood it based on too general title of this issue.