Loro synchronization using Websockets

[zolero]

Thank you for creating such an amazing CRDT library! Loro's stable data format, excellent performance, and cross-platform support make it truly impressive.

I was wondering if you could provide an example or guide for implementing real-time synchronization with WebSockets? Specifically:

• Exporting changes from a Loro document.
• Transmitting those changes via WebSockets.
• Importing and merging updates into remote documents.

Right now, it’s quite hard to figure out the best practices for implementing these kinds of synchronizations. An example or guide would greatly help developers like me integrate Loro into real-time collaborative applications and showcase its full potential. I hope to see something similar to what Hocuspocus offers, as it provides a great reference for WebSocket-based synchronization.

Thank you again for your amazing work!

[zxch3n]

Thanks! We're working on that

[trusktr]

If you can at least describe:

• Exporting changes from a Loro document (these could be sent to somewhere centralized, could be via WebSockets or anything else)
• how you would track this change in a centralized place (assume JavaScript, f.e. Node.js, and no matter how it got there, could be WebSockets or anything else really)
• Merging an update into a doc (f.e. sent in any matter, could be WebSockets, doesn't matter)

Basically, what's the best practice for managing multiple documents around a centralized process (around a centralized state).

For sake of example, a reference example can be implemented inside a single JavaScript file that runs in a browser, something like this:

// Example: 3 clients will be in a "room" editing the same "document"
const doc1 = new LoroDoc()
const doc2 = new LoroDoc()
const doc3 = new LoroDoc()

// Example: the centralized location (it could be on a remote server, but that's no matter to the example).
// Is a LoroDoc even what we want here? I don't know (that's why I landed here)
const centralizedDoc = new LoroDoc()

// ...Example: doc1 is modified at any time ...
setTimeout(function client1UpdatesDoc1() {
  // send update for doc1
  sendUpdate(doc1.export(...))

  setTimeout(client1UpdatesDoc1, Math.round(500 + Math.random() * 1000))
}, Math.round(500 + Math.random() * 1000))

// ...Example: doc2 is modified at any time ...
setTimeout(function client2UpdatesDoc2() {
  // send update for doc2
  sendUpdate(doc2.export(...))

  setTimeout(client2UpdatesDoc2, Math.round(500 + Math.random() * 1000))
}, Math.round(500 + Math.random() * 1000))

// ...Example: doc3 is modified at any time ...
setTimeout(function client3UpdatesDoc3() {
  // send update for doc3
  sendUpdate(doc3.export(...))

  setTimeout(client3UpdatesDoc3, Math.round(500 + Math.random() * 1000))
}, Math.round(500 + Math.random() * 1000))

async function sendUpdate(update) {
  // pretend this function receives a "payload" (a network app could be sending it over WebSockets or anything else)

  await new Promise(r => setTimeout(r, 200 + Math.round(Math.random() * 100))) // emulate network delay

  const updateToSync = // ...update centralizedDoc ... centralizedDoc.import(...), or something

  syncClients(updateToSync)
}

function syncClients(update) {
  // update all clients (doc1, doc2, doc3), they might be updated in differing order depending on network conditions.
  // Maybe this also needs to skip updates from the original client that sent the update.

  new Promise(r => setTimeout(r, 200 + Math.round(Math.random() * 100))).then(() => { // emulate network delay
    doc1.import(...) // or similar
  })

  new Promise(r => setTimeout(r, 200 + Math.round(Math.random() * 100))).then(() => { // emulate network delay
    doc2.import(...) // or similar
  })

  new Promise(r => setTimeout(r, 200 + Math.round(Math.random() * 100))).then(() => { // emulate network delay
    doc3.import(...) // or similar
  })
}

What would this reference example look like?

After having a working version of a reference example like that, we can easily extrapolate the non-Loro parts (such as sending data via WebSockets instead of merely emulating something with async functions).

[lopezm94]

Is there any new developments on this? I was just searching for an alternative for yjs+hocuspocus and this is the only conversation about this that i have found.

[ununian]

Hi all,

I've modified the Hocuspocus code to add experimental support for Loro. This was possible because Loro's synchronization protocol is fundamentally similar to that of Yjs.

You can find my repository here: https://github.com/ununian/hocuspocus-loro

The key changes are in the frontend-loro directory and the LoroProvider.ts file. You can check the diffs of the recent commits for more details.

Please note that this is currently just a proof-of-concept (POC) to demonstrate feasibility. It has not been thoroughly tested and is still missing some functionality.

However, I believe it's a solid starting point. If there's interest from the community, I'd be happy to continue developing and refining it.