SuspenseCache / autoDisposeTimeoutMs should be documented

[metatick]

I just spent quite some time scratching my head as to why useSuspenseQuery was unexpectedly returning cached data.

In our case, our graphql server implementation calls out to an unreliable external service, and returns a graphql error if that call fails.

Unmounting/remounting to trigger a "fresh" useSuspenseQuery did not work as anticipated due to the suspense cache, which doesn't appear to be documented anywhere.

I'd suggest documenting:

1. The fact that the suspense cache exists, what it does, and what it's for
2. How to configure autoDisposeTimeoutMs

Additionally, perhaps the behavior of the suspense cache should be reconsidered - in my opinion calling useSuspenseQuery with a fetchPolicy of 'network-only', 'no-cache', etc.. should probably do what it says on the tin, rather than there being a whole extra layer of hidden caching involved...

[phryneas]

Hi @metatick, I'm sorry for the frustration!

Generally, SuspenseCache should be an implementation detail and while we added the autoDisposeTimeoutMs option, it should not really be necessary to ever change that. We did have some reports about this kind of scenario, though, so we'll have to revisit that.

Additionally, perhaps the behavior of the suspense cache should be reconsidered - in my opinion calling useSuspenseQuery with a fetchPolicy of 'network-only', 'no-cache', etc.. should probably do what it says on the tin, rather than there being a whole extra layer of hidden caching involved...

You might read too much into the name "cache" here. Really, what SuspenseCache does is keep a reference to a promise around for a certain amount of time while your component is still trying to mount. Once it is mounted, it discards of that reference.

In React, components don't have an identity, state or lifecycle before they fully mounted for the first time - and suspense prevents that "first successful mount" until suspense has finished completely.

Until then, the component could try to mount any number of times for any amount of reasons. Without SuspenseCache, if your component would try to mount 5 times, you would end up with 5 network requests.

Even worse, this component would probably never mount:

function MyComponent(){
  const result1 = useSuspenseQuery(query1, { fetchPolicy: 'no-cache' })
  const result2 = useSuspenseQuery(query2, { fetchPolicy: 'no-cache' })
}

It would try mounting, make a network request for query1. Once that's done, it would try to mount again, go past the first useSuspenseQuery (realistically, not even this would work without SuspenseCache, but let's assume it does) and make a network request for query2. Once that is done, it would try to mount again, but have forgotten about query1 already being resolved (the component never finished mounting, has no local state). So it would start the circle again, and you'd end up with a never-mounting component and indefinite network requests.

So, yeah, unfortunately we need SuspenseCache because React itself doesn't have a mechanism to solve this.

But as I said, it should be an implementation detail, so we'll have to look more into this issue of retrying after an error.

[metatick]

Hey @phryneas

Thanks for replying and clarifying, I appreciate it!

I had guessed that the intent behind the SuspenseCache was to handle request waterfalls, where multiple components within the same suspense boundary could suspend in series.

I think documenting the behavior, and the autoDisposeTimeoutMs option could help others understand what is happening if they run into the same situation as I did :)

Just a thought...

Wouldn't the current implementation also never allow a component to mount if there were two or more calls to useSuspenseQuery that both took longer than autoDisposeTimeoutMs to resolve?

Using your example code, if both queries each took longer than autoDisposeTimeoutMs, wouldn't query1 be dropped from SuspenseCache by the time query2 resolves, causing query1 to suspend again, etc..?

I wonder if it would be practical to, do something like:

1. Have autoDisposeTimeoutMs configured with a smaller value by default
2. Have useSuspenseQuery restart the timer on each call to keep the data available in the SuspenseCache (I'm assuming there's a call to setTimeout under the hood somewhere)
3. Resolve the promise thrown by useSuspenseQuery after autoDisposeTimeoutMs / 2 (or some other magic number) if the query is still in flight

I'd love to hear your thoughts :)

[jerelmiller]

I had guessed that the intent behind the SuspenseCache was to handle request waterfalls, where multiple components within the same suspense boundary could suspend in series.

@metatick this isn't quite right. The suspense cache is really more of a "promise" cache (I used quotes because it actually stores query refs, but for simplicity, you can think of it like a promise cache). Suspense works by throwing promises. Because it uses throw as the mechanic behind suspense, we need some way to ensure we can reference that same promise instance when React tries to rerender your component again. And that is what @phryneas is saying here:

In React, components don't have an identity, state or lifecycle before they fully mounted for the first time - and suspense prevents that "first successful mount" until suspense has finished completely.

To explain further, this means that we can't use useState, useMemo, useId, useRef, etc. to store those promises because this hooks do NOT store values until the component has fully mounted (fully mounted meaning it can render your component without suspending or throwing an error). An external store is the only thing we can use to store/lookup those promises when React renders your component after it suspends. And that is what the suspense cache is. Its that external store that ensures we lookup the right promise after suspending your component.

(side note: the key used to lookup values in the suspense cache is a combination of query + variables. This is why the queryKey option exists in case you have two components that use the same query + variables and you want to ensure they get their own promise query ref instances, otherwise those 2 components share the same promise state query ref).

As @phryneas said, it really is an implementation detail that end users shouldn't have to know/care about so we choose not to talk about it for that reason.

I think documenting the behavior, and the autoDisposeTimeoutMs option could help others understand what is happening if they run into the same situation as I did :)

Yes this is a good callout! We should document autoDisposeTimeoutMs. I thought I had done so when I first added that option, but apparently not. My apologies 🤦‍♂️.

Using your example code, if both queries each took longer than autoDisposeTimeoutMs, wouldn't query1 be dropped from SuspenseCache by the time query2 resolves, causing query1 to suspend again, etc..?

Yes this is very possible, though this is why we allow you to configure the duration of the timeout. We set it to 30s by default (which should be plenty of time for 99.9% of queries), but if for some reason you have queries that last longer than 30s, you can adjust it yourself to ensure the promise doesn't get evicted from the suspense cache before your component tries to render again (which would result in another fetch + suspend). You can configure this as such:

new ApolloClient({
  // ...
  defaultOptions: {
    react: {
      suspense: {
        // set to 60s instead of 30s
        autoDisposeTimeoutMs: 60_000,
      },
    },
  },
});

Again, definitely something that should be documented! We will be releasing 4.0 soon and we will try and get this change in after that.

I wonder if it would be practical to, do something like:

1. Have autoDisposeTimeoutMs configured with a smaller value by default
2. Have useSuspenseQuery restart the timer on each call to keep the data available in the SuspenseCache (I'm assuming there's a call to setTimeout under the hood somewhere)
3. Resolve the promise thrown by useSuspenseQuery after autoDisposeTimeoutMs / 2 (or some other magic number) if the query is still in flight

I'll try and respond to each of these in turn:

1. Have autoDisposeTimeoutMs configured with a smaller value by default

The autoDisposeTimeoutMs really only exists to ensure that the suspense cache (again think of it like a promise cache) doesn't end up with a memory leak if a component suspends, but then never mounts/renders again (for example, when the component that suspends is rendered conditionally). We need a way to clean up the promise query ref that caused the component to suspend in the first place. That is really its only purpose.

Once a component fully mounts, we cancel that auto dispose timer. Being fully mounted means we can also detect when the component unmounts and immediately remove the promise query ref from the suspense cache. The dispose timer doesn't affect anything after a component is fully mounted. Again, its only purpose is to ensure that if nothing reads that promise within the given autoDisposeTimeoutMs, it gets cleaned up and evicted from the suspense cache.

2. Have useSuspenseQuery restart the timer on each call

Again, once the component fully mounts, we have no need for that timer so we cancel it immediately.

to keep the data available in the SuspenseCache (I'm assuming there's a call to setTimeout under the hood somewhere)

The suspense cache isn't a data cache. Thats the job of InMemoryCache (or whatever cache you configured with new ApolloClient(...). Its a "promise" cache that exists to ensure your component doesn't suspend indefinitely because we would have no other way to store the thrown promise. A restartable timer is unnecessary.

3. Resolve the promise thrown by useSuspenseQuery after autoDisposeTimeoutMs / 2 (or some other magic number) if the query is still in flight

We don't want to do this either because the suspense hooks read data off of that promise. If we resolve early, we don't have any data to resolve since the network request hasn't finished. That breaks our guarantees about data returned from the hook and thats not something we want to do.

---

What I think you're experiencing here is the same as this comment: #449 (comment). Here is that comment (mostly) copied:

So it looks like this is a known "issue" and something that I had originally forgotten about in regards to Suspense. What is happening here is that the query ref (which stores the promise from the query) is subject to the autoDisposeTimeoutMs behavior from the suspense hooks. By default this timeout is set to 30 seconds and will automatically dispose of the query ref if a component hasn't mounted in that time. In fact, if you try your reproduction but wait 30s, you can actually see the remount of the component work as expected. This is something we noted in our tests, but I had forgotten about this behavior: https://github.com/apollographql/apollo-client/blob/820f6e656377fbc4f7d870976fe3563d32ec1ec1/src/react/hooks/__tests__/useSuspenseQuery.test.tsx#L3690-L3693

What's happening is that since the component suspends immediately, then returns an error, the component isn't ever mounted so we have no way to tell our suspense cache to dispose of the query ref that caused the error. That query ref is then retained in the suspense cache for 30s (or whatever is configured for that auto timeout value) so that when the component runs againit finds that query ref in the suspense cache and the error gets rethrown.

Unfortunately due to the mechanics of suspense, this one is very very difficult to fix. If you'd like a more technical breakdown of what's happening, I'd be happy to explain more. We'll need to get creative in order to find a solution for this 😞

@phryneas and I met this morning and we are experimenting with apollographql/apollo-client#12773 that will evict the promise query ref from the suspense cache when we detect that the promise has rejected. This should avoid the issue you're experiencing which is that rerendering your component within that autoDisposeTimeoutMs after an error makes it seem like we are returning some sort of cached data value (in reality we just haven't evicted that promise from the suspense cache, so your component is rerendering it again!).

We will be testing apollographql/apollo-client#12773 a bit more, but I think this is a promising direction that should hopefully resolve this issue once and for all.

Sorry for the long explanation, but hopefully this makes sense! At the very least, hopefully this shows why Suspense is actually a pretty complicated feature!

[metatick]

Hey @jerelmiller,

Thanks for replying - I'm aware of how Suspense works, though I think my proposed idea on an alternative implementation was not properly explained.

Here's a (very crude) code sample for a pseudo useSuspenseQuery based on what I was meaning, with the following caveats:

1. The simple example doesn't cache the promise, so it wouldn't work for multiple components at all
2. Errors aren't cleared from the cache - not really necessary for the basic example in my opinion
3. It's very crude, and probably littered with bugs!

The crux of the idea is that the promise thrown can be resolved by either the query completing, or by a "remount timer". When the components attempt to remount, useSuspenseQuery resets the dispose timer and returns the data if available, throws an exception if the query failed, or throws a new promise if the query is still pending.

The benefit of this approach is that the auto dispose timeout can be fairly small, as it only needs to larger than both the auto-remount timer and the longest time a non-useSuspenseQuery based suspension takes to resolve (Eg. a React.lazy component).

I'd love to hear your thoughts :)

import {useEffect} from "react";

const AUTO_DISPOSE_TIMEOUT = 500;
const AUTO_REMOUNT_TIMEOUT = 250;
const SUSPENSE_CACHE = {}

async function executeQuery(key, simulatedQueryLatency, reject) {
    await new Promise(resolve => setTimeout(resolve, simulatedQueryLatency));
    if (reject) {
        throw new Error("Something went wrong")
    }
    return `Hello World: ${key}`;
}

function resolveQuery(key, data) {
    SUSPENSE_CACHE[key].status = 'resolved';
    SUSPENSE_CACHE[key].data = data;
    SUSPENSE_CACHE[key].resolve(data);
}

function rejectQuery(key, error) {
    SUSPENSE_CACHE[key].status = 'error';
    SUSPENSE_CACHE[key].error = error;
    SUSPENSE_CACHE[key].reject(error);
}

function disposeQuery(key) {
    if (!SUSPENSE_CACHE[key]) {
        return;
    }
    console.log(`disposeQuery ${key}`)
    if (SUSPENSE_CACHE[key].remountTimer) {
        clearTimeout(SUSPENSE_CACHE[key].remountTimer)
    }
    if (SUSPENSE_CACHE[key].disposeTimer) {
        clearTimeout(SUSPENSE_CACHE[key].disposeTimer)
    }
    delete SUSPENSE_CACHE[key];
}

function remount(key) {
    throw new Promise((resolve, reject) => {
        if (SUSPENSE_CACHE[key].remountTimer) {
            clearTimeout(SUSPENSE_CACHE[key].remountTimer)
        }
        SUSPENSE_CACHE[key].resolve = resolve;
        SUSPENSE_CACHE[key].reject = reject;
        SUSPENSE_CACHE[key].remountTimer = setTimeout(() => {
            console.log(`remounting ${key}`)
            resolve()
        }, AUTO_REMOUNT_TIMEOUT);
    })
}

function restartDisposeTimer(key) {
    if (SUSPENSE_CACHE[key].disposeTimer) {
        clearTimeout(SUSPENSE_CACHE[key].disposeTimer);
    }
    SUSPENSE_CACHE[key].disposeTimer = setTimeout(() => {
        disposeQuery(key);
    }, AUTO_DISPOSE_TIMEOUT);
}

function startQuery(key, simulatedQueryLatency, reject) {
    console.log(`startQuery ${key}`)
    const query = executeQuery(key, simulatedQueryLatency, reject)
        .then(data => resolveQuery(key, data))
        .catch(data => rejectQuery(key, data));

    SUSPENSE_CACHE[key] = {
        status: 'pending',
        data: null,
        error: null,
        query
    };
}

function getOrCreateQueryCacheEntry(key, simulatedQueryLatency, reject) {
    if (!SUSPENSE_CACHE[key]) {
        startQuery(key, simulatedQueryLatency, reject);
    }
    return SUSPENSE_CACHE[key];
}

function useSuspenseQuery(key, simulatedQueryLatency, reject) {
    let entry = getOrCreateQueryCacheEntry(key, simulatedQueryLatency, reject);
    restartDisposeTimer(key);

    useEffect(() => {
        disposeQuery(key);
    }, [key]);

    if (entry.status === 'pending') {
        remount(key)
    } else if (entry.status === 'error') {
        throw entry.error;
    } else {
        return entry.data;
    }
}

export function Example() {
    console.log('attempting mount')
    const data = useSuspenseQuery('example', 1000, false);
    const data2 = useSuspenseQuery('example2', 1000, false);
    return (
        <div>
            <div>{data}</div>
            <div>{data2}</div>
        </div>
    )
}

[phryneas]

That solution is missing a key requirement:

For Suspense, you have to cache the promise. Your component will "try" to mount about 2-10 times before it actually mounts for the first time, and the use hook expects to get the same promise instance back every single time.

This is something that has gotten murky between React versions - React 18 gets a lot of problems if the promise reference isn't stable, where in React 19 there are some heuristics where it's kinda "okay" for the promise reference to be unstable (althought definitely not recommended) and it just ignores all promises except for the first one (which can cause problems on it's own).
Since we support React 18 and 19 with suspense, we have to keep stable promises cached.