|
| 1 | +# `ReadableStream` Async Iteration Explained |
| 2 | + |
| 3 | + |
| 4 | +## Introduction |
| 5 | + |
| 6 | +The streams APIs provide ubiquitous, interoperable primitives for creating, composing, and consuming streams of data. |
| 7 | + |
| 8 | +This change adds support for the [async iterable protocol](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_async_iterator_and_async_iterable_protocols) |
| 9 | +to the `ReadableStream` API, enabling readable streams to be used as the source of `for await...of` loops. |
| 10 | + |
| 11 | +To consume a `ReadableStream`, developers currently acquire a reader and repeatedly call `read()`: |
| 12 | +```javascript |
| 13 | +async function getResponseSize(url) { |
| 14 | + const response = await fetch(url); |
| 15 | + const reader = response.body.getReader(); |
| 16 | + let total = 0; |
| 17 | + |
| 18 | + while (true) { |
| 19 | + const {done, value} = await reader.read(); |
| 20 | + if (done) return total; |
| 21 | + total += value.length; |
| 22 | + } |
| 23 | +} |
| 24 | +``` |
| 25 | + |
| 26 | +By adding support for the async iterable protocol, web developers will be able to use the much simpler |
| 27 | +`for await...of` syntax to loop over all chunks of a `ReadableStream`. |
| 28 | + |
| 29 | +## API |
| 30 | + |
| 31 | +The [ReadableStream definition](https://streams.spec.whatwg.org/#rs-class-definition) is extended |
| 32 | +with a [Web IDL `async iterable` declaration](https://webidl.spec.whatwg.org/#idl-async-iterable): |
| 33 | +``` |
| 34 | +interface ReadableStream { |
| 35 | + async iterable<any>(optional ReadableStreamIteratorOptions options = {}); |
| 36 | +}; |
| 37 | +
|
| 38 | +dictionary ReadableStreamIteratorOptions { |
| 39 | + boolean preventCancel = false; |
| 40 | +}; |
| 41 | +``` |
| 42 | + |
| 43 | +This results in the following methods being added to the JavaScript binding: |
| 44 | + |
| 45 | +* `ReadableStream.prototype.values({ preventCancel = false } = {})`: returns an [AsyncIterator](https://tc39.es/ecma262/#sec-asynciterator-interface) |
| 46 | + object which locks the stream. |
| 47 | + * `iterator.next()` reads the next chunk from the stream, like `reader.read()`. |
| 48 | + If the stream becomes closed or errored, this automatically releases the lock. |
| 49 | + * `iterator.return(arg)` releases the lock, like `reader.releaseLock()`. |
| 50 | + If `preventCancel` is unset or false, then this also cancels the stream |
| 51 | + with the optional `arg` as cancel reason. |
| 52 | +* `ReadableStream.prototype[Symbol.asyncIterator]()`: same as `values()`. |
| 53 | + This method makes `ReadableStream` adhere to the [ECMAScript AsyncIterable protocol](https://tc39.es/ecma262/#sec-asynciterable-interface), |
| 54 | + and enables `for await...of` to work. |
| 55 | + |
| 56 | +## Examples |
| 57 | + |
| 58 | +The original example can be written more succinctly using `for await...of`: |
| 59 | +```javascript |
| 60 | +async function getResponseSize(url) { |
| 61 | + const response = await fetch(url); |
| 62 | + let total = 0; |
| 63 | + for await (const chunk of response) { |
| 64 | + total += chunk.length; |
| 65 | + } |
| 66 | + return total; |
| 67 | +} |
| 68 | +``` |
| 69 | + |
| 70 | +Finding a specific chunk or byte in a stream also becomes easier (adapted from |
| 71 | +[Jake Archibald's blog post](https://jakearchibald.com/2017/async-iterators-and-generators/#making-streams-iterate)): |
| 72 | +```javascript |
| 73 | +async function example() { |
| 74 | + const find = 'J'; |
| 75 | + const findCode = find.codePointAt(0); |
| 76 | + const response = await fetch('https://html.spec.whatwg.org'); |
| 77 | + let bytes = 0; |
| 78 | + |
| 79 | + for await (const chunk of response.body) { |
| 80 | + const index = chunk.indexOf(findCode); |
| 81 | + |
| 82 | + if (index != -1) { |
| 83 | + bytes += index; |
| 84 | + console.log(`Found ${find} at byte ${bytes}.`); |
| 85 | + break; |
| 86 | + } |
| 87 | + |
| 88 | + bytes += chunk.length; |
| 89 | + } |
| 90 | +} |
| 91 | +``` |
| 92 | +Note that the stream is automatically cancelled when we `break` out of the loop. |
| 93 | +To prevent this, for example if you want to consume the remainder of the stream differently, |
| 94 | +you can instead use `response.body.values({ preventCancel: true })`. |
| 95 | + |
| 96 | + |
| 97 | +## Goals |
| 98 | + |
| 99 | +* Permit `ReadableStream` to be used as the source of a `for await...of` loop. |
| 100 | + |
| 101 | + |
| 102 | +## Non-goals |
| 103 | + |
| 104 | +N/A. |
| 105 | + |
| 106 | + |
| 107 | +## End-user benefit |
| 108 | + |
| 109 | +* Reduces boilerplate for developers when manually consuming a `ReadableStream`. |
| 110 | +* Allows integration with future ECMAScript proposals, such as [Async Iterator Helpers](https://github.com/tc39/proposal-async-iterator-helpers). |
| 111 | +* Allows interoperability with other APIs that can "adapt" async iterables, such as |
| 112 | + Node.js [Readable.from](https://nodejs.org/docs/latest-v20.x/api/stream.html#streamreadablefromiterable-options). |
| 113 | + |
| 114 | + |
| 115 | +## Alternatives |
| 116 | + |
| 117 | +* It was [initially suggested](https://github.com/whatwg/streams/issues/778#issuecomment-371711899) |
| 118 | + that we could use a `ReadableStreamDefaultReader` as an `AsyncIterator`, by adding `next()` |
| 119 | + and `return()` methods directly to the reader. However, the return values of `return()` and |
| 120 | + `releaseLock()` are different, so the choice went to adding a separate async iterator object. |
0 commit comments