Skip to content

feat(javascript): add replaceAllObjectsWithTransformation #5008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

shortcuts
Copy link
Member

@shortcuts shortcuts commented Jun 16, 2025

🧭 What and Why

🎟 JIRA Ticket: https://algolia.atlassian.net/browse/DI-3894

Changes included:

this adds the helper to replace all of the content of an index by leveraging the transformation pipeline, i've inlined the chunkedPush logic but actually could be reused in the saveObjects and partialUpdateObjects methods as well, lmk

@shortcuts shortcuts self-assigned this Jun 16, 2025
@shortcuts shortcuts force-pushed the feat/javascript-replaceAllObjectsWithTransformation branch from 142a5c0 to bc5ed6c Compare June 17, 2025 07:35
@algolia-bot
Copy link
Collaborator

algolia-bot commented Jun 17, 2025

✔️ Code generated!

Name Link
🪓 Triggered by 38a6b17f362aece382a9479c5ec2e967f55e1d43
🍃 Generated commit d3f2c2bcc5f2d88eabc334f277a518161e795790
🌲 Generated branch generated/feat/javascript-replaceAllObjectsWithTransformation
📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language Req/s
go 1769
javascript 1539
php 1449
csharp 1231
java 1100
python 1092
ruby 846
swift 415
scala 23

@shortcuts shortcuts force-pushed the feat/javascript-replaceAllObjectsWithTransformation branch from bc5ed6c to 405c5e4 Compare June 17, 2025 07:36
@shortcuts shortcuts marked this pull request as ready for review June 17, 2025 09:35
@shortcuts shortcuts requested a review from a team as a code owner June 17, 2025 09:35
@shortcuts shortcuts requested review from morganleroi and millotp June 17, 2025 09:35
@shortcuts
Copy link
Member Author

've inlined the chunkedPush logic but actually could be reused in the saveObjects and partialUpdateObjects methods as well, lmk

ah actually that would mean breaking the signature, I need to check if there is some usage first

@shortcuts shortcuts force-pushed the feat/javascript-replaceAllObjectsWithTransformation branch from 0ee2dc0 to 4be66cf Compare June 17, 2025 11:35
@shortcuts
Copy link
Member Author

've inlined the chunkedPush logic but actually could be reused in the saveObjects and partialUpdateObjects methods as well, lmk

ah actually that would mean breaking the signature, I need to check if there is some usage first

breaking all of them at once in #5011

Copy link
Collaborator

@millotp millotp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so nice, gg

* @param chunkedPush.indexName - The `indexName` to replace `objects` in.
* @param chunkedPush.objects - The array of `objects` to store in the given Algolia `indexName`.
* @param chunkedPush.action - The `batch` `action` to perform on the given array of `objects`, defaults to `addObject`.
* @param chunkedPush.waitForTasks - Whether or not we should wait until every `batch` tasks has been processed, this operation may slow the total execution time of this method but is more reliable.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does waitForTasks make sense here ? should it be waitForPush ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied chunkedBatch so it's usable the same way but since it's internal I can rename it if you want

const responses: Array<WatchResponse> = [];

const objectEntries = objects.entries();
for (const [i, obj] of objectEntries) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about https://stackoverflow.com/a/8495740 ? I feel like it's more efficient and more readable

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've actually just copied chunkedBatch 🥶

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's IE compatible as well? I can't remember what version we should support but I think it's 11?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slice is available everywhere

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll update both implems in a follow up pr!

if (waitForTasks) {
for (const resp of responses) {
if (resp.eventID === undefined || !resp.eventID) {
throw new Error('received unexpected response from the push endpoint, eventID must not be undefined');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this error checking should be in loop above, to early exit.
Also in that case there should be an error message in the response

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually had to put it here to satisfy the type checking (same for Go and other languages), I can duplicate it, but this one is required

@shortcuts shortcuts requested a review from millotp June 17, 2025 15:08
@shortcuts shortcuts enabled auto-merge (squash) June 17, 2025 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants