Firestore API encourages waterfalls

[iwikal]

I have a use case where I want to subscribe to two queries simultaneously.

const aSnap = useFirestoreCollection(aQuery)
const bSnap = useFirestoreCollection(bQuery)

This will result in an unnecessary waterfall where we wait for the first query to unsuspend before sending off the next. I resolved this by writing this helper function:

export interface Resource<T> {
  read(): T
}

/**
 * A higher order function that will call a given function, and wrap the return
 * value in a Resource. If the callback throws (or suspends), we catch that and
 * throw (or suspend) upon Resource.read instead.
 *
 * This makes it easier to have more concurrent requests, and avoid unnecessary
 * waterfalls by suspending too eagerly.
 */
export function resourcify<A extends any[], T>(func: (...args: A) => T) {
  return function(...args: A): Resource<T> {
    try {
      const t = func(...args)
      return {
        read() {
          return t
        }
      }
    } catch (err) {
      return {
        read() {
          throw err
        }
      }
    }
  }
}

function MyComponent() {
  const [aSnap, bSnap] = [
    resourcify(useFirestoreCollection)(aQuery),
    resourcify(useFirestoreCollection)(bQuery),
  ].map(resource => resource.read())
  return null
}

But it would be nice if the API nudged us in this direction by default.

[jhuleatt]

Hi @iwikal, great observation, and cool solution!

We actually have an experimental function for this, though it currently isn't documented:

reactfire/reactfire/firestore/index.tsx

Line 17 in e1daab4

export function preloadFirestoreDoc(

The preloadFirestoreDoc function allows you to kick off a non-blocking request that can later be subscribed to via useFirestoreDoc. The other nice thing is that you can call it before the Firestore SDK is loaded. I'm thinking that this will be the pattern for the other hooks in reactfire, but definitely open to feedback.

[jhuleatt]

I've started documenting this here. Maybe I should rename the section to "Preventing Waterfalls"

[iwikal]

I didn't find a preload function for collection queries, is that in the works?

Also, I don't really understand how this preloading works. Does it register a listener? If so, when does that get unsubscribed? If not, wouldn't that result in twice the reads? (Once to preload, once to register the actual listener)

[iwikal]

I think the resource as return type pattern would work pretty well for the whole api too. If you do want to suspend immediately, you can always call read right away.

const { docs } = useFirestoreCollection(query).read()

You can also enforce preloading on the type level, if you say that a component wants a resource as a prop. That tells the parent component that they need to start fetching the resource before rendering the child.

Maybe there's room for both. useFirestoreCollectionDataResource is a pretty long name, though.

[jhuleatt]

Thanks for the thoughts and suggestions, @iwikal, and sorry for the winter holiday delay!

I've had some time to think on this some more, and I think this is solvable without increasing ReactFire's API surface. Example:

somewhere up the component tree when we're pretty sure we'll need some data later

// loads the Firestore SDK (doesn't block render or suspend)
preloadFirestore().then(firestore => {
  // preload two collections using the vanilla Firestore SDK
  // non-blocking
  // get() will close the connection to firestore automatically
  // if we end up not using this farther down the tree
  firestore()
    .collection("animals")
    .get();
  firestore()
    .collection("plants")
    .get();
});

in the component that actually needs the data

// hopefully, preloadFirestore has already loaded this in.
// if not, it will suspend
const firestore = useFirestore();

// both these requests are already in flight (managed by Firestore SDK)
// each will suspend if they're not ready, but that's ok, because this component
// needs both to be ready to render
const animals = useFirestoreCollectionData(firestore().collection("animals"));
const plants = useFirestoreCollectionData(firestore().collection("plants"));

This is similar to Relay's preloadQuery pattern and the React team's Start Fetching Early example, with the benefit that I think we can lean on the Firebase SDK to cache between the preload and the useFirestore* call instead of passing the query down to the component.

It also means that if someone just wants to get an app working and doesn't yet care about waterfalls + preloading, they don't need to change any code down in their component later when they decide they want to increase perf by adding a preload.

If you'd like to see it in action, I'm going to open a PR soon to try this in the sample directory. I'm interested in hearing your feedback!

[iwikal]

I can totally see that this "opt-in to preloading" approach would be desirable.
I still think that mechanism of sending off a request, awaiting it, and then subscribing after it has been resolved, even when it's just used behind the scenes like here is what's causing the double reads issue #196.

Me and my team have been experimenting with a different design forgoing the preloads altogether, cutting away a whole bunch of features to get a proof of concept working. It's not really ready, yet, but that's probably what we'll end up using.

We've discovered an issue with the Resource pattern though: you have to be careful where you call read so that you don't try to suspend from an event handler or similar.

We're also trading the double reads for a risk of having more long-running requests that don't end up being used. It's hard to say which is going to be cheaper in the general case.