DEV Community

Cover image for Normalize your React Query data with MobX State Tree
Mateo Hrastnik for Lloyds digital

Posted on • Updated on

Normalize your React Query data with MobX State Tree

Fetching data in React is deceptively hard. You start off with a simple useEffect + useState combo and you think you're done.

"This is great!" you think to yourself...
But then you realize you didn't handle errors. So you add a bunch of code to handle that.
Then you realize you have to add a refresh button. So you add a bunch of code to handle that.
Then your backend developer tells you the data is paginated. So you add a bunch of code to handle that.
Then you want to trigger a refresh automatically every N seconds. So you add a bunch of code to handle that.
By this time, your data fetching code is an absolute nightmare and managing it becomes a headache, and we haven't even touched the subject of caching.

What I'm trying to say is that React Query is awesome. It handles all of the complexity listed above, and much more. So if you haven't yet, you should definitely give it a shot.

However, at Lloyds, we haven't always been using React Query. Not so long ago, we had a custom useQuery hook that tried real hard to serve all our data fetching needs. It was good, but not nearly as good as React Query. However, as our useQuery was tightly coupled with MobX State Tree, we had a couple of benefits that we really liked:

  • Typed models
  • Data normalization at response time
  • Data denormalization at access time
  • Actions on models

Note - you can check out my article on how we used MST here: Why you should use MST

Typed models

With MobX State Tree, you're required to define the shape of your data. MST uses this scheme to validate your data at runtime. Additionally, as MST uses TypeScript, you get the benefit of having IntelliSense autocomplete all of the properties on your data models while you're writing code.

Data normalization and denormalization

What do I mean by this? Well, to put it simply - this ensures that there's only one copy of any given data resource in our app. For example, if we update our profile data this ensures that the update will be visible across the app - no stale data.

Actions on models

This is a great MST feature. It enables us to attach actions on the data models in our app. For example, we can write something like

  onPress={() => {
      article.createComment("I love this!");
  }}
Enter fullscreen mode Exit fullscreen mode

instead of the much less readable alternative

  onPress={() => {
      createCommentForArticle(article.id, "This doesn't feel good");
  }}
Enter fullscreen mode Exit fullscreen mode

or the even more complicated version

  onPress={() => {
      dispatch(createCommentForArticle(getArticleIdSelector(article), "I'm sorry Mark, I had to"));
  }}
Enter fullscreen mode Exit fullscreen mode

Moving to React Query meant getting the new and improved useQuery hook, but losing the great MST features we just couldn't do without. There was only one option...

Combining React Query and MST

Turns out it's possible to get the best of both worlds, and the code isn't even that complicated.
The key is to normalize the query response as soon as it gets back from the server and instead of the raw resource data, return the MST instance from the query function.

We'll use the MST stores to define the data fetching methods and the methods for converting raw network response data to MobX instances.

Here's an example... First, let's define two models. These will define the shape of the resources we will fetch.

const Author = model("Author", {
  id: identifier,
  name: string,
});

const Book = model("Book", {
  id: identifier,
  title: string,
  author: safeReference(Author),
}).actions((self) => ({
  makeFavorite() {
    // ... other code
  },
}));
Enter fullscreen mode Exit fullscreen mode

Next we'll define the stores to hold collections of these resources.

const BookStore = model("BookStore", {
  map: map(Book),
});

const AuthorStore = model("AuthorStore", {
  map: map(Author),
});
Enter fullscreen mode Exit fullscreen mode

Let's add a process action that will normalize the data and return the MST instances. I added some logic to the action so that it can handle both arrays and single resources and additionally merge the new data with the old - this way we avoid potential bugs when different API endpoints return different resource shapes (eg. partial data when fetching a list of resources vs full data returned when fetching a single resource).

We'll also add an action that will perform the HTTP request and return the processed data. We will later pass this function to useInfiniteQuery or useQuery to execute the API call.

const BookStore = model("BookStore", {
  map: map(Book),
})
  .actions((self) => ({
    process(data) {
      const root: StoreInstance = getRoot(self);
      const dataList = _.castArray(data);
      const mapped = dataList.map((book) => {
        if (isPrimitive(book)) return book;

        book.author = getInstanceId(root.authorStore.process(book.author));

        const existing = self.map.get(getInstanceId(book));
        return existing
          ? _.mergeWith(existing, book, (_, next) => {
              if (Array.isArray(next)) return next; // Treat arrays like atoms
            })
          : self.map.put(book);
      });

      return Array.isArray(data) ? mapped : mapped[0];
    },
  }))
  .actions((self) => ({
    readBookList: flow(function* (params) {
      const env = getEnv(self);
      const bookListRaw = yield env.http.get(`/books`, {
        params,
      });
      return self.process(bookListRaw);
    }),
  }));

const AuthorStore = model("AuthorStore", {
  map: map(Author),
}).actions((self) => ({
  process(data) {
    const dataList = _.castArray(data);
    const mapped = dataList.map((author) => {
      if (isPrimitive(author)) return author;

      const existing = self.map.get(getInstanceId(author));
      return existing
        ? _.mergeWith(existing, author, (_, next) => {
            if (Array.isArray(next)) return next; // Treat arrays like atoms
          })
        : self.map.put(author);
    });
    return Array.isArray(data) ? mapped : mapped[0];
  },
}));

const Store = model("Store", {
  bookStore: BookStore,
  authorStore: AuthorStore,
});
Enter fullscreen mode Exit fullscreen mode

That's basically it, we can now use the readBookList method in our components with useQuery or useInfiniteQuery... Almost.
If you try it at this point, you'll get an error. That's because React Query internally uses something called structural sharing to detect if the data has changed. However, this is not compatible with MobX State Tree so we need to disable it. We can configure this using a top-level query client provider.

import { QueryClient, QueryClientProvider } from "react-query";

const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      structuralSharing: false,
      // ... other options
    },
  },
});

function App() {
  // ... other code

  return (
    <QueryClientProvider client={queryCache}>
      {/* ... other providers ... */}
      <Router />
    </QueryClientProvider>
  );
}
Enter fullscreen mode Exit fullscreen mode

All that's left to do is to actually try running the query.

function BookListView() {
  const store = useStore();
  const query = useQuery("bookList", (_key, page = 1) =>
    store.bookStore.readBookList({ page })
  );

  // Convert array of responses to a single array of books.
  const bookList = _.flatMap(query.data, (response) => response.data);

  return (
    <div>
      {bookList.map((book) => {
        return (
          <BookView
            book={book}
            onPress={book.makeFavorite} // We have access to methods on the Book model
          />
        );
      })}
    </div>
  );
}
Enter fullscreen mode Exit fullscreen mode

We get the flexibility of React Query without sacrificing the benefits of MobX State Tree.

You can check out the complete example on Code Sandbox here:

LINK TO CODE SANDBOX

In the example, the API calls are mocked. In production, this would be replaced with the real fetch calls. You can notice how, when you enable the "Show author list" checkbox, it updates the author on the "Book list" section. There is only one instance of author-2 in the app, and everything stays in sync. We don't have to refetch the whole list.

Summary

React Query and MobX State Tree are great tools. But together, they are unstoppable. React Query gives us the flexibility to fetch data from the server just the way we want it. MST + TypeScript provide the type safety + intuitive way of adding methods and computed properties on the data models. Together they provide a great developer experience and help you build awesome apps.

Thank you for reading this! If you've found this interesting, consider leaving a ❤️, 🦄 , and of course, share and comment on your thoughts!

Lloyds is available for partnerships and open for new projects. If you want to know more about us, check us out.

Also, don’t forget to follow us on Instagram and Facebook!

Top comments (14)

Collapse
 
robertcoopercode profile image
Robert Cooper

With this setup, do you have a cache in both MobX as well as React Query? If yes, how do you determine which cache you should read from? Or does the React Query cache reference the MobX cache so whenever you update data in MobX, the change propagates to the React Query cache?

Also, how do you update data in your server data with a MobX action? In your linked code sandbox you use .makeFavorite on the Book model to update the book in the MobX cache, but would you also need to use a react-query mutation to update your server data?

Collapse
 
topcat profile image
Misha

I also would like to see how to make caching and mutations work. It seems pointless to me to use React Query with Mobx without these two features

Collapse
 
hrastnik profile image
Mateo Hrastnik • Edited

Or does the React Query cache reference the MobX cache so whenever you update data in MobX, the change propagates to the React Query cache

Exactly. The data lives only in MobX. React Query caches only the references to the MST instances, so if you update the data anywhere, the change is visible everywhere.

how do you update data in your server data with a MobX action

You can use react-query mutations, and then update the MobX data once the request is successful.
Mutations are really practical as they handle the IDLE > LOADING > SUCCESS | ERROR state transitions and support retries and other react query goodies. However, you can just as simply run your async actions in useEffect.

Collapse
 
vinhnguyenhq profile image
Vinh Nguyen

great article man, appreciated.!

Collapse
 
rdewolff profile image
Rom • Edited

Thanks for the write up. Am wondering about the exact downside of disabling the structuralSharing param from React Query. For whoever might be interested in this, see [1].

For info, the queryCache has been deprecated [2]. QueryClient should now be used [3].

[1] react-query.tanstack.com/guides/im...

[2] react-query.tanstack.com/guides/mi...

[3] react-query.tanstack.com/guides/mi...

Collapse
 
hrastnik profile image
Mateo Hrastnik

I've updated it!

Collapse
 
ntucker profile image
Nathaniel Tucker

What are the advantages of this over using a store that has normalization built in like resthooks.io/docs/getting-started/... ?

It seems like a lot of extra work vs

function TodoDetail({ id }: { id: number }) {
  const todo = useSuspense(TodoResource.get, { id });
  const ctrl = useController();
  const updateWith = title => () =>
    ctrl.fetch(TodoResource.partialUpdate, { id }, { title });
  return (
    <div>
      <div>{todo.title}</div>
      <button onClick={updateWith('🥑')}>🥑</button>
      <button onClick={updateWith('💖')}>💖</button>
    </div>
  );
}
Enter fullscreen mode Exit fullscreen mode
Collapse
 
klis87 profile image
Konrad Lisiczyński

Nice article. I also missed automatic normalisation and data updates like in apollo client (but for anything including rest), so I recommend you to try normy - the library I recently created - github.com/klis87/normy

It already has react-query binding, so it basically allows you to use react-query just like you do without any extra code (apart from several lines of code to setup normalisation store) and you can enjoy automatic data updates.

And bonus - it works with trpc too!

Collapse
 
zavdev profile image
Igor Zaviryukha

I love the idea of storing a reference to the mobx-state-tree instances inside the react-query cache because whenever you update your data from actions it's also updated in react-query cache automatically. But when we have this kind of code:

const PostsStore = t
  .model("PostsStore", {
    posts: t.array(PostModel),
  })
  .actions((self) => ({
    fetch: flow(function* () {
      const posts = yield* toGenerator(apiPosts.getAll());
      self.posts.replace(posts);
      return self.posts;
    }),
  }));
Enter fullscreen mode Exit fullscreen mode

where we return an array of posts from fetch action, this array is going to be stored inside react-query cache and if we update it with setQueryData function like:

queryClient.setQueryData([QUERY_KEYS.posts], (prev) => {
     return [...prev, { id: 100, title: "New Post", body: "Body" }];
});
Enter fullscreen mode Exit fullscreen mode

changes won't be tracked by state tree and we'll get unsynchronized data. But if we push it from actions it will be available in both (react-query cache and inside mobx-state-tree model).

To fix this we can return a "self" instance from fetch action so we can store the reference to the whole postsStore inside react-query cache and then any manual changes from outside (from react-query) should be performed by calling actions. But, I'm wondering if it's a correct solution at all to return a reference to the whole storage because it can possibly contain more fields that we don't really want to cache (even references to them).

So, my question is how did you solve this problem? Because in case we return an array it becomes very easy that someone (if you don't work alone) can change rq cached data like I did before with setQueryData and this array won't be observable anymore so any other changes to it from actions won't make any effect.

Collapse
 
ekeijl profile image
Edwin • Edited

Nice article! If I understand correctly, using this method, each object only occurs once in the store, but you still fetch a whole page per query? The creator of react-query mentioned that with normalized caching you may end up replicating all the filtering, sorting, paging logic on the client side and you potentially still have stale data.

twitter.com/tannerlinsley/status/1...

In your opinion, does this still outweigh just refetching the data (and potentially overfetching) with react-query?

Collapse
 
hrastnik profile image
Mateo Hrastnik

Nice article! If I understand correctly, using this method, each object only occurs once in the store, but you still fetch a whole page per query?

That's correct. I don't have to recreate the server-side logic because I use react-query to sync. The normalization makes sure that each object occurs only once in the store. This way, a resource on a list view and the same resource on the details view is always in sync. Also, updating the resource means that it will update both the list and details view.
a
This alone is already a great benefit with no downsides compared to plain react-query. But when you add to it all other benefits of MST...

Actions on models

you can do stuff like article.like() instead of like(article)

Transparent denormalization

you can do stuff like article.author.name (Here author is a MST object with actions and views). Or even article.author.update({name: "John Doe" }). The API is as clean as it gets.

Type checking

All of the properties, actions, views on the model are typechecked. When you type in article. you editor will list all the available props on the model, and it goes all the way down - type in article.author. and you get all the props on the author model.

As I see it - there no downsides when using RQ+MST combo - only benefits.

Collapse
 
kode profile image
Kim Ode

Interesting! I've thought a lot about this problem, and ended up writing a query library on top of mobx-state-tree: github.com/ConrabOpto/mst-query

Not as many features as react-query yet of course, but I personally think first class support for mobx-state-tree models is worth it.

Collapse
 
dilmodev profile image
Dillon Morris

Great article!

Are there any benefits react-query provides that we lose by storing our results in the state tree? In the tweet below, Tanner Linsley says he doesn't combine his global state tree (in his case zustand) until he needs them in the same place:

twitter.com/tannerlinsley/status/1...

Collapse
 
hrastnik profile image
Mateo Hrastnik

You don't lose any benenfits. You have to write a bit more code for mst, but thats just the types that you have to write anyway if you use TS