perf: optimize event_loop_tick and async ops

[CyanChanges]

Superseded by

• perf: resolve Promises natively for async ops #1163
• perf: faster timer #1164

Summary

34% more performant async ops + less memory usage
(by resolve Promises natively)

const ops = Deno.core.ops;
// ops.op_print(Object.keys(ops).join('\n')+'\n')
function tImm(tag, p) {
  const symbol = Symbol('k.notImm')
  function check(v) {
    const imm = v !== symbol
    // ops.op_print(`@${tag} ${imm}\n`)
  }
  const p1 = Promise.any([p, Promise.resolve(symbol)])
  p1.then(check, check)
}
for (let i = 0; i < 50000; i++) {
tImm("op_void_async", ops.op_void_async())
tImm("op_void_async_deferred", ops.op_void_async_deferred())
tImm("op_error_async", ops.op_error_async())
}
// ops.op_print("Hello World\n")

[CyanChanges]

Faster setTimeout

// ported from Bun: 
// https://github.com/oven-sh/bun/blob/c82e4fb485808a7582c227c1eb4dd8cc25ad7b86/bench/snippets/set-timeout.mjs#L10
const ops = Deno.core.ops;

let count = 20_000_000;
const batchSize = 1_000_000;

const beginTime = Temporal.Now.instant()
ops.op_print("Run\n");

let { promise, resolve, reject } = Promise.withResolvers();
let remaining = count;

if (batchSize === 0) {
  for (let i = 0; i < count; i++) {
    setTimeout(() => {
      remaining--;
      if (remaining === 0) {
        resolve();
      }
    }, 0);
  }
  await promise;
} else {
  for (let i = 0; i < count; i += batchSize) {
    let batch = Math.min(batchSize, count - i);
    const bbt = Temporal.Now.instant()
    ops.op_print(`Batch ${i} - ${i + batch}`);
    let { promise: batchPromise, resolve: batchResolve } = Promise.withResolvers();
    let remaining = batch;
    for (let j = 0; j < batch; j++) {
      setTimeout(() => {
        remaining--;
        if (remaining === 0) {
          batchResolve();
        }
      }, 0);
    }
    await batchPromise;
    const bet = Temporal.Now.instant()
    ops.op_print(`: ${bet.since(bbt).toLocaleString("en-US")}\n`);
  }
}

const fmt = new Intl.NumberFormat();
ops.op_print(`Executed ${fmt.format(count)} timers\n`);
const endTime = Temporal.Now.instant();
ops.op_print(`Duration: ${endTime.since(beginTime).toLocaleString("en-US")}\n`)

[CyanChanges]

Timer in deno is way slower than Node.js and Bun. (the result is better in dcore with that dcore ported version above tho)

(https://canary.discord.com/channels/684898665143206084/778060818046386176/1393869232562765886)

I added traces for time consumed for queue timers, poll timers, and call timers,
both poll and call is actually very fast,
and seems the bottle-neck is at queue timers

^{time it took each `op_queue_timer` (debug build, tracked inside of the op)}

(https://canary.discord.com/channels/684898665143206084/688040863313428503/1393599894022783038)

If I just took the worst case, and multiplied by the timer count, and it's a value about the same to the actual result.

It needs two inserts to the BTree, and it's probably the reason.

Take a look at the implementation of Node.js
using a map for timeout value, and each value is a linked list for a list of timers

It's a constant time insert `O(1)`

Take a look at Bun's Timer implementation, it's using a Pairing Heap ^[1]

The time complexity is O(1) Amortized (according to AI tho)

While for BTree (that deno_core uses), it's O(log(N)) for most cases

https://en.wikipedia.org/wiki/Pairing_heap#Summary_of_running_times

Pairing Heap seems to be a good choice tho

[Hajime-san]

Cool!

• A question about code generating by core/rebuild_async_stubs.js #917

[bartlomieju]

Thanks, lots of interesting stuff in here. I'd like to ask you to split this PR into smaller chunks though - it's quite hard to review (and merge) with confidence with so many moving pieces. I think it should be split into 3 distinct PRs in following order:

• changes to handling of promises in ops
• changes to op2 macro (once I understand why they are needed anyway)
• changes to timers implementation

Let me know if you need any help with that

[bartlomieju]

Could you actually store these callbacks/bindings as separate functions instead of additional struct?

[CyanChanges]

I thought put them together will be a good option, or I have to wrap them in RefCell<Option<...>> individually, i don't think it's necessary to have many these wrappers

[bartlomieju]

I don't think this collects all callbacks into the struct, leading to bifurcation. Let's keep them separate and once these changes are done we can refactor all callback handling into a single struct.

[bartlomieju]

This should be kept - if you don't drop these globals before snapshotting, it will most likely error out

[CyanChanges]

Oh, I forgotten

[bartlomieju]

Can you explain why changes to the op2 macro are needed?

[CyanChanges]

TL;DR Since I can just resolve and reject natively, I changed the macros to resolve reject the promise instead of scope.throw or retval.set. Avoid the cost of throwing errors.

Oringinally, dispatch async uses the same code as dispatch slow, if it's Err immediately, it throw errors directly to scope, while in async, the errors should result in Promise.reject.

So In the async wrap, it does try { ... } catch (err) { return Promise.reject(err) }.

Now I removed the async wrap, the error will be thrown directly, that might cause problem.
Since I can just resolve and reject natively, I changed the macros to resolve reject the promise instead of scope.throw or retval.set

[CyanChanges]

Thanks, lots of interesting stuff in here. I'd like to ask you to split this PR into smaller chunks though - it's quite hard to review (and merge) with confidence with so many moving pieces. I think it should be split into 3 distinct PRs in following order:

• changes to handling of promises in ops
• changes to op2 macro (once I understand why they are needed anyway)
• changes to timers implementation

Let me know if you need any help with that

These two needs to be done together tho:

• changes to handling of promises in ops
• changes to op2 macro (once I understand why they are needed anyway)

since I needs needs resolve promise natively, currently the op2 macro needs the async wrap to behave correctly.

This is still WIP, I guess I would use a Pairing Heap or something and check if there's performance improvements:

• changes to timers implementation

[bartlomieju]

since I needs needs resolve promise natively, currently the op2 macro needs the async wrap to behave correctly.

Yeah, that's fine. 2 PRs then 👍

[CyanChanges]

Superseded by

• perf: resolve Promises natively for async ops #1163
• perf: faster timer #1164