Refactor benchmark dashboard codebase for new usecases

[huydhn]

With the exception of PT2 inductor dashboard, all benchmark dashboards on HUD are base on:

• https://github.com/pytorch/test-infra/blob/main/torchci/pages/benchmark/llms.tsx
• https://github.com/pytorch/test-infra/tree/main/torchci/components/benchmark/llms
• and https://github.com/pytorch/test-infra/tree/main/torchci/lib/benchmark

With the recent additions of new dashboards like TorchAO, vLLM, and CacheBench, there are some ugly hard-coding parts in the code base that will become harder to maintain when we try to onboard new benchmarks in the future. The key issue here is that, although they share the same code base, different dashboards have slightly different requirements on what to show there. For example, we want to show is_dynamic field on CacheBench.

This is an open end issues to document some thoughts I have on how to improve this. We could:

• Figure out a generic and intuitive way to display the information on how a benchmark is setup on the dashboard. Note that this can include lots of information about parameters, devices, dependencies, etc. Although these information are recorded in the database, they don't generally have a coherent structure (to give devs a degree of flexibility there)
  • Enforce a structure on how a benchmark setup is hard because there are lots of use cases out there with widely different requirements, i.e. mobile benchmarks
• Refactor the code base into better reusable components
• Customize the dashboards using config files
• More ideas?

The goal is to make it easier to add new dashboard and/or reconfigure existing ones.

cc @yangw-dev

[yangw-dev]

Since we will introduce new UI, I think the code base would be different