Streamed JSON Lines

Instead of streaming a single JSON (which it seems does not realy stream with the StreamedJsonResponse class from Symfony/Laravel), we proposed Streamed JSON Lines (https://jsonlines.org/).

Example:

return new \MacropaySolutions\LaravelCrudWizard\Responses\StreamedJsonResponse(
    Operation::query()->lazyByIdDesc(1000, 'id'),
    encodingOptions: JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES
 );

The above will stream each row from DB as a single JSON per line:

{"id":17009,"value":"92.00","created_at":"2024-01-17 09:17:11","updated_at":null,"primary_key_identifier":"17009"}
{"id":17008,"value":"87.00","created_at":"2024-01-17 09:17:11","updated_at":null,"primary_key_identifier":"17008"}

In browser, the following Javascript code could be used to parse the response and display each row as it arrives, thus not needing to wait for the whole JSON stream to finish:

<script>
     function requestStream(e,form) {
         e.preventDefault();
         const outputElement = document.getElementById("streamed_operations_response");

         outputElement.innerHTML = '';
         fetch(form.action, {method:'post', headers: {
                 'Accept': 'application/json'
             }, body: new URLSearchParams(new FormData(form))})
             .then(response => {
                 const reader = response.body.getReader();
                 const decoder = new TextDecoder();
                 let leftOver = '';
                 return new ReadableStream({
                     start(controller) {
                         function push() {
                             reader.read().then(({ done, value }) => {
                                 if (done) {
                                     controller.close();
                                     return;
                                 }
                                 const chunk = decoder.decode(value, { stream: true });
                                 chunk.split(/\n/).forEach(function (element) {
                                     let row;
                                     try {
                                         row = JSON.parse(element);
                                     } catch (e) {
                                         console.log('e');
                                         if (leftOver === '') {
                                             leftOver = element;

                                             return;
                                         }
                                         try {
                                             row = JSON.parse(leftOver + element);
                                             leftOver = '';
                                         } catch (ex) {
                                             console.log('ex');
                                             leftOver += element;
                                             console.log('This leftOver should not happen: ' + leftOver);

                                             return;
                                         }
                                     }

                                     let child = document.createElement('p');
                                     child.innerHTML = JSON.stringify(row);
                                     outputElement.appendChild(child);
                                 });
                                 controller.enqueue(value);
                                 push();
                             });
                         }
                         push();
                     }
                 });
             })
             .then(stream => new Response(stream))
             .then(response => response.text())
             .then(data => {
                 console.log("Streaming complete");
             })
             .catch(error => {
                 console.error("Streaming error:", error);
             });
     }
 </script>

A demo can be found here https://laravel-crud-wizard.com/laravel-9/laravel-lumen-crud-wizard#operations. Just select Streamed Json in the Pagination drop-down and submit.

See also StreamedJsonResponse class.