Skip to content

Add the ability for scheduling plugins to add/modify requests during request processing #791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shmuelk opened this issue May 7, 2025 · 1 comment
Assignees
Labels
needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@shmuelk
Copy link

shmuelk commented May 7, 2025

What would you like to be added:
Add the ability for PreSchedule, Filter, Scorer, Picker, and PostSchedule scheduler plugins to add/modify the set of headers that are sent to the chosen inference pod with the request.

Why is this needed:
Some implementations of disaggregated Prefill/Decode processing require the use of additional headers in the request that is sent to the inference server.

If acceptable, I am happy to contribute the code.

@shmuelk shmuelk added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label May 7, 2025
@shmuelk
Copy link
Author

shmuelk commented May 7, 2025

/assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
1 participant