You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What would you like to be added:
Add the ability for PreSchedule, Filter, Scorer, Picker, and PostSchedule scheduler plugins to add/modify the set of headers that are sent to the chosen inference pod with the request.
Why is this needed:
Some implementations of disaggregated Prefill/Decode processing require the use of additional headers in the request that is sent to the inference server.
If acceptable, I am happy to contribute the code.
The text was updated successfully, but these errors were encountered:
What would you like to be added:
Add the ability for PreSchedule, Filter, Scorer, Picker, and PostSchedule scheduler plugins to add/modify the set of headers that are sent to the chosen inference pod with the request.
Why is this needed:
Some implementations of disaggregated Prefill/Decode processing require the use of additional headers in the request that is sent to the inference server.
If acceptable, I am happy to contribute the code.
The text was updated successfully, but these errors were encountered: