Inductor dashboard features tracking

[huydhn]

Here are some of the request features to Inductor dashboard:

• Sort the results Per model view in compiler view sort does not work #4136, Improved per-model difference view #4138
• Add full support for blueberries and dynamic subgroups #6020
• There are many options which makes the dropdown list hard to use in small screen
• Way to swap between compiler type in drilldown page #4258
• Better log view and error matching for inductor benchmark jobs #6021
• Add a toggle to show everything on the same summary page Compilers HUD should show all graphs on page #4031
• Don't use accuracy CSVs to drive dashboard printing; support custom CSV names #4002
• [Inductor Benchmark]Show different precision dtypes in the same chart/table  #6019

Other bugs:

• Should fail_to_run models be included in numbers? #4310
• Should not use workflow id to filter workflows in range #4192
• Last quarter compiler HUD view does not work #4123

cc @ezyang

Tasks

Preview Give feedback

No tasks being tracked yet.

[eellison]

Another request is an option to compare two different backends in the per-model view , e.g. to see which dynamic models have a significant slowdown compared to normal inductor.

[ezyang]

Add a toggle to show everything on the same summary page #4031

This is the single most important thing, from the perspective of the oncall

[huydhn]

cc @yangw-dev as you are planning to pick up this project

[yangw-dev]

sounds good! thx!

[yangw-dev]

Add a toggle to show everything on the same summary page #4031

This is the single most important thing, from the perspective of the oncall

Implemented
cc: @ezyang

[ezyang]

Oh, there's another problem, which is that there are also a lot of configs (inference/training, the difference dtypes), and being able to see the graphs + these in one page also would help the oncall a lot.

[yangw-dev]

there are also a lot of configs (inference/training, the difference dtypes), and being able to see the graphs + these in one page also would help the oncall a lot.

Hi, thank you! Is there any issue is created for this? Mind provide the issue, and add some description(what list of things
to expect) would be super helpful! So we can track the work! Feel free to ping me too :)! @ezyang

[huydhn]

Hi, thank you! Is there any issue is created for this? Mind provide the issue, and add some description(what list of things to expect) would be super helpful! So we can track the work!

There is no issue for this atm, but please feel free to create one. I think the ask here is to see if we can remove the precision and mode pickers so that we can have all charts in one big page. You did this for the suite picker in #6009.

On the other hand, I think adding more charts blindly might not be ideal. I'm thinking maybe we could show different precision dtypes in the same chart/table to start with.