Skip to content

CacheCapabilities.max_batch_total_size_bytes refers to the size of the serialized protobuf messages, not the blobs #345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

CacheCapabilities.max_batch_total_size_bytes refers to the size of the serialized protobuf messages, not the blobs #345

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions build/bazel/remote/execution/v2/remote_execution.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -363,8 +363,8 @@ service ContentAddressableStorage {

// Upload many blobs at once.
//
// The server may enforce a limit of the combined total size of blobs
// to be uploaded using this API. This limit may be obtained using the
// The server may enforce an upper limit on the size of the serialized
// request message for this API. This limit may be obtained using the
// [Capabilities][build.bazel.remote.execution.v2.Capabilities] API.
// Requests exceeding the limit should either be split into smaller
// chunks or uploaded using the
Expand All @@ -381,7 +381,9 @@ service ContentAddressableStorage {
//
// Individual requests may return the following errors, additionally:
//
// * `RESOURCE_EXHAUSTED`: There is insufficient disk quota to store the blob.
// * `RESOURCE_EXHAUSTED`: There is insufficient disk quota to store the blob,
// or if the size of the serialized BatchUpdateBlobsResponse message exceeds
// [CacheCapabilities.max_batch_total_size_bytes][build.bazel.remote.execution.v2.CacheCapabilities.max_batch_total_size_bytes].
Comment on lines +384 to +386
Copy link
Contributor Author

@mostynb mostynb Sep 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unsure about this part- I need to check what gRPC implementations actually return in this case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we got something like code = ResourceExhausted desc = gRPC message exceeds maximum size 4194304: 4196713

// * `INVALID_ARGUMENT`: The
// [Digest][build.bazel.remote.execution.v2.Digest] does not match the
// provided data.
Expand Down Expand Up @@ -1883,14 +1885,23 @@ message BatchReadBlobsResponse {
// The digest to which this response corresponds.
Digest digest = 1;

// The raw binary data.
// The raw binary data, assuming that the status is `OK`.
bytes data = 2;

// The format the data is encoded in. MUST be `IDENTITY`/unspecified,
// or one of the acceptable compressors specified in the `BatchReadBlobsRequest`.
Compressor.Value compressor = 4;

// The result of attempting to download that blob.
// The result of attempting to download the blob.
//
// If the blob could not be included in the response because doing so
// would cause the serialized BatchReadBlobsResponse message to exceed
// [CacheCapabilities.max_batch_total_size_bytes][build.bazel.remote.execution.v2.CacheCapabilities.max_batch_total_size_bytes]
// then the server MUST NOT include the data and MUST set this field
// to `RESOURCE_EXHAUSTED`. Clients can retry the request with fewer or
// smaller blobs in the
// [ContentAddressableStorage.BatchReadBlobsRequest][build.bazel.remote.execution.v2.ContentAddressableStorage.BatchReadBlobsRequest]
// or download individual blobs with the [ByteStream API][google.bytestream.ByteStream].
google.rpc.Status status = 3;
}

Expand Down Expand Up @@ -2228,8 +2239,9 @@ message CacheCapabilities {
// Supported cache priority range for both CAS and ActionCache.
PriorityCapabilities cache_priority_capabilities = 3;

// Maximum total size of blobs to be uploaded/downloaded using
// batch methods. A value of 0 means no limit is set, although
// Maximum size of serialized request and response messages that
// this server supports when using the BatchUpdateBlobs and
// BatchReadBlobs APIs. A value of 0 means no limit is set, although
// in practice there will always be a message size limitation
// of the protocol in use, e.g. GRPC.
int64 max_batch_total_size_bytes = 4;
Expand Down