Skip to content

Warn on large upsert requests #216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Warn on large upsert requests #216

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,18 @@ prost-types = "0.13.3"
anyhow = "1.0.89"
serde = { version = "1.0.210", features = ["derive"], optional = true }
serde_json = { version = "1.0.128", optional = true }
reqwest = { version = "0.12.8", optional = true, default-features = false, features = ["stream", "rustls-tls", "http2"] }
reqwest = { version = "0.12.8", optional = true, default-features = false, features = [
"stream",
"rustls-tls",
"http2",
] }
futures = { version = "0.3.31" }
tokio = { version = "1.40.0", features = ["rt-multi-thread"] }
futures-util = { version = "0.3.31", optional = true }
derive_builder = { version = "0.20.2" }
thiserror = "1.0.64"
semver = "1.0.24"
log = { version = "0.4.27", default-features = false }

[dev-dependencies]
tonic-build = { version = "0.12.3", features = ["prost"] }
Expand Down
12 changes: 11 additions & 1 deletion src/qdrant_client/points.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ use crate::qdrant::{
};
use crate::qdrant_client::{Qdrant, QdrantResult};

/// Max amount of points that should be upserted in a single request/chunk.
const TOO_MANY_POINTS: usize = 50_000;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels a bit arbitrary.

I mean, the request object can basically be infinitely large no matter the number of points.

Do you know if it's possible to render a proper error message instead on the tonic side of things when we surpass limits?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit for gRPC we set in Qdrant is usize::MAX (~2.3 Exabytes) and the error we get on too large requests is not a limit exceeded or similar error.

The one I was facing when uploading 500k points was
gRPC /qdrant.Points/Upsert unexpectedly failed with Internal error "Missing request message."
Which I can't find in tonics documentation.

To not risk throwing wrong errors, I'd prefer not to map this error to some RequestTooLarge error.

In order to get a limit exceeded error (assuming this exists and gets thrown reliably), we'd need to set a lower limit in the gRPC API, which is arguably a breaking change.

I thought getting some warning is always better than a non descriptive tonic error and a timeout in the client.
If you feel 50k points to be too random, feel free to suggest a better limit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if it's possible to render a proper error message instead on the tonic side of things when we surpass limits?
gRPC /qdrant.Points/Upsert unexpectedly failed with Internal error "Missing request message."

Could it be improved with hyperium/tonic#1682?
It's landed in tonic 0.12, but Qdrant still uses 0.11.


/// # Point operations
///
/// Manage points and vectors.
Expand Down Expand Up @@ -103,7 +106,14 @@ impl Qdrant {
&self,
request: impl Into<UpsertPoints>,
) -> QdrantResult<PointsOperationResponse> {
let request = &request.into();
let request: &UpsertPoints = &request.into();

if request.points.len() > TOO_MANY_POINTS {
log::warn!(
"Trying to upload more than 50k points. Consider using upsert_points_batch instead."
);
}

self.with_points_client(|mut points_api| async move {
Ok(points_api.upsert(request.clone()).await?.into_inner())
})
Expand Down