In the last post, I described how I was going about rendering a backwards compatible react admin datagrid component. This part focuses on the data fetching logic.

How is it supposed to work?

The user should be able to scroll around the whole component, and the data should be fetched only when required, i.e. when the user slows down or stops at some point. Thankfully the when to fetch the data is handled by react virtualized. So we just need to tie it together to react admin. React Virtualized however, does not store our data anywhere so we need our own data structure somewhere to keep track of that.

Making it work

The render tree of our infinite scrolling component looks something like this

<InfiniteLoader
  isRowLoaded={({ index }) => false}
  loadMoreRows={
    ({ startIndex, stopIndex }) => Promise.resolve('Done')
  }
  rowCount={/* number of rows */}
>
  {({ onRowsRendered, registerChild }) => (
    <AutoSizer>
      {({ width, height }) => (
        <Table 
          width={width}
          height={height}
          rowGetter={({ index }) => null}
          ... 
        />
      )} 
    </AutoSizer>
  )}
</InfiniteLoader>

We need to tap into react admin to implement the following:

• isRowLoaded
• loadMoreRows
• rowCount
• rowGetter

We use react admin’s dataProvider to handle custom fetches. The data fetched through this is also cached in the redux store, as a id -> object map so we can access the data easily anywhere.

The ordering of the data displayed is determined by another array ids present in the store. For our use case, however we can’t use this since the data need not be linear. For example, the user loads item 1-20 and scrolls to the bottom to load 80-100. If we use the ids from the store, we’ll need to populate everything in between. Hence, we use a map data structure to store the ids, called loadedIds, which maps an integer key to a id which is loaded.

We can extract the data from the store as follows

const { resource } = useListContext()

const loadedIds = useRef({})

const { data, ids, total } = useSelector((state) => ({
  ids: state.admin.resources[resource].list.ids,
  data: state.admin.resources[resource].data,
  total: state.admin.resources[resource].list.total,
}))

function rowGetter ({ index }) {
  return data[loadedIds[index]]
}

function isLoaded ({ index }) {
  return !!loadedIds[index]
}

React Admin’s data provider expects data to be fetched in pages. Hence, since the number of items perPage is fixed, we can find the page to be fetched using the formula startIndex / perPage. Note that this might fetch less items than requested because of the perPage limit. But we need not worry about that since React Virtualized will call our function again for the missing data when required.

const { resource, currentSort, filterValues } = useListContext()

const loadMore = ({ startIndex, stopIndex }) => {

  const page = Math.floor(startIndex / perPage) + 1

  // All fetched data is stored in RA's redux store
  return dataProvider
    .getList(resource, {
      pagination: { page, perPage },
      sort: currentSort,
      filter: filterValues,
    })
    .then(({ data, total }) => {
      // Populate loadedIds with the new data
      const newStopIndex = Math.min(total - 1, startIndex + perPage - 1)
      for (let i = startIndex; i <= newStopIndex; i++) {
        loadedIds.current[i] = data[i - startIndex].id
      }
    })
}

Handling Sort, Filter and Optimizing Network Calls

For handling sorting and filtering, one way is to watch the corresponding props and trigger a side effect which clears all previous data and initiates a new request for the data. This however, would result in duplicate network calls. Similarly, when mounting, react admin fetches the first page of data for us, which we are fetching again.

To eliminate these, we come back to ids. Notice that ids always contains the ordered ids for the first page of data. Even when updating the sorting or filter, this value updates.

Hence, we can run an effect to synchronize ids with our loadedIds as follows

// ids change only on first mount, or an external trigger
// like currentSort, filterValue, delete etc
useEffect(() => {
  // reset data
  loadedIds.current = {}
  for (let i = 0; i < ids.length; i++) {
    loadedIds.current[i] = ids[i]
  }
}, [ids])

Also, since we always have the data for the first page, we can eliminate the need to load it

const loadMore = ({ startIndex, stopIndex }) => {
  // React Admin always fetches the first page for us, so we
  // don't need to fetch it again
  if (startIndex < perPage) 
    return
  ...
}

Sidenote - Why is loadedIds a ref and not a state variable ?

If loadedIds was a state variable, updating it would trigger a rerender which would be unnecessary since InfiniteLoader is not a stateful component.

Thats about it. I went through multiple solutions and iterations to arrive at this, history of which can be found here

Until next time!