Skip to content

Conversation

@williamsyang-work
Copy link
Contributor

@williamsyang-work williamsyang-work commented Nov 10, 2025

Export Selection Range as CSV Command

This is a draft that I'd like some feedback and support on.
This exports table rows to a CSV file based off of the selection range.

SUPPORT NEEDED ON:

  • Currently only works for Events Table. When trying to fetch the start/end event indexes from timestamps for any other table, I receive a 400 error BAD REQUEST - INCORRECT QUERY PARAMETERS. I commented on the related lines.
  • General feedback

Feature Demo

This writes ~100k lines to a CSV file from the selection without crashing the app or lagging out main usage.

Screencast.from.11-10-2025.11_39_42.AM.mp4

Why A Command vs UI Integration?

People have lots of opinions about UI. I wanted verification on the core functionality before building out UI.

Algorithm

I kept it simple.

The function exports table data to a CSV file within a selected time range.
It retrieves column headers, resolves start and end indices for that range, and opens a writable CSV stream.
It then requests rows from the server in fixed-size batches, prefetching the next batch while writing the previous one to disk.
The process repeats until all rows have been written, after which the file stream is closed.

Fixed memory usage (1k lines)
No FE -> BE data pipline (lots of code overhead / slow JSON RPC)

A diagram if it helps:

sequenceDiagram
    participant App
    participant API
    participant FS as "File"

    %% Setup phase
    App->>API: "Fetch column headers"
    API-->>App: "Header list"
    App->>API: "Fetch start index for START_TIME"
    API-->>App: "Start index"
    App->>API: "Fetch end index for END_TIME"
    API-->>App: "End index"
    App->>FS: "Open CSV stream and write header"

    %% Prime first fetch
    App->>API: "Fetch batch 1 (startIndex, count)"
    Note over App,API: "ongoing = promise for first batch"

    %% Batched pipelined loop
    loop "while rowsLeft > 0"
        App->>API: "Await previous fetch (batch N)"
        API-->>App: "Return lines (batch N)"

        App->>API: "Start next fetch (batch N+1)"
        App->>FS: "Build and write CSV batch (await drain if needed)"
        App->>App: "Update progress and decrement rowsLeft"
    end

    %% Final flush
    App->>API: "Await final fetch (batch N+1)"
    API-->>App: "Final lines"
    App->>FS: "Write final batch"
    App->>FS: "End stream and wait for finish"

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant