-
Notifications
You must be signed in to change notification settings - Fork 257
Open
Description
Problem
Streaming tests only measure maximum QPS by running searches as fast as possible. No option exists to test performance under controlled, specific QPS loads.
Current vs Proposed
- Current: insert_rate controllable, search rate runs at maximum speed
- Proposed: Add optional search_qps parameter for controlled search rates
Use Cases
- SLA Testing: "Can DB maintain 95% accuracy at 1000 QPS?"
- Resource Planning: "What's latency at target 500 QPS?"
- Load Comparison: "DB A vs B performance at same controlled load"
- Bottleneck Analysis: "At what QPS does performance degrade?"
Expected Behavior
- When search_qps not specified β existing max QPS measurement (backward compatible)
- When search_qps specified β test performance at that specific rate
- Results show both target QPS and achieved metrics
Questions
- Does this feature align with project roadmap?
- Any preferred approach or naming conventions?
- Should this extend to other test types beyond streaming?
Next Steps
I'm planning to contribute this feature to VectorDBBench. Should I go ahead with the implementation?
Any guidance on preferred approach or design considerations would be appreciated before I start working on it.
Metadata
Metadata
Assignees
Labels
No labels