Reselect v5 Roadmap Discussion: Goals and API Design

[markerikson]

Reselect has been loosely maintained over the last couple years. There's been a lot of PRs filed that have been sitting around, including some that require a new major version. The goal of this discussion is to:

• List what use cases Reselect v4 does not cover, and what pain points users have with it
• Compare Reselect's APIs with other similar libraries in the ecosystem and identify capabilities that would be worth borrowing
• Nail down a desired set of capabilities for Reselect v5 and determine what the API should look like
• Add any discussion necessary to figure out implementation details

I'd like to thank @ellbee, who's been the primary maintainer. Real life has taken up his time lately, so he's given myself and @timdorr publish rights on NPM and a green light to work on PRs.

I already have more than enough on my todo list with the rest of the Redux family of libraries, so I need to limit my involvement in maintaining Reselect. However, I'm happy to help shepherd the conversation here, define some vision, and provide guidance.

I'd love to see some folks in the community volunteer to help do whatever work's needed here, and even come on board as an active maintainer for Reselect.

Prior Reselect v5 Planning

In a discussion with @ellbee earlier today, he said:

The version 5 release was just going to be fixes to the TypeScript bindings resolving the problem where we had to use overloads so there was a maximum number of parameters that could be typed correctly. As I recall we tried changing the position of the combining function and enforcing that dependencies had to be given as an array, which would have been a problematic breaking change so were also going to look into the feasibility of a code mod. I think the whole thing is moot now with variadic tuple types in TypeScript 4.0 and it looks like people are actively working on it at the moment.

The other thing that was being considered was #401

Oh, and it was going to be written in TypeScript to side step the whole where should the bindings live issue

I do think that josepots approach in the linked issue is good, but I wonder if reselect should be deprecated rather than fundamentally alter how it works. I worry about the amount of projects using it in its current form and don’t want it to be the cause of lots of (potentially subtle) breakage or performance problems.

[To clarify]: I just mean trying to move it away from being the default/recommended option, especially now that es6 is so widely supported and things like proxies are feasible to use.

So, as a starting point, it seems reasonable to assume that we'd rewrite Reselect's source to TypeScript, update the types to work better with variadic args, and try to address known pain points and additional use cases.

Current Reselect Pain Points and Problems

Cache Size and Selector Instance Reuse

These two problems go hand-in-hand. Reselect only has a cache size of 1 by default. This is fine when a selector is only being given state as its only argument. However, it's very common to want to reuse a selector instance in a way that requires passing in varying arguments, such as a "filter items by category" selector:

const selectItemsByCategory = createSelector(
  state => state.items,
  (state, category) => category,
  (items, category) => items.filter(item.category === category)
)

selectItemsByCategory(state, "a"); // first call, not memoized
selectItemsByCategory(state, "a"); // same inputs, memoized
selectItemsByCategory(state, "b"); // different inputs, not memoized
selectItemsByCategory(state, "a"); // different inputs from last time, not memoized

In cases like this, multiple components all call the same selector with different arguments one after the other. So, it will never memoize correctly.

The current workaround here, when used with React-Redux, is to create unique selector instances per component instance. With connect, this required a complex "factory function" syntax for mapState:

const makeMapState = (state) => {
  const selectItemsForThisComponent = makeUniqueSelectorInstance();

  return function realMapState(state, ownProps) {
    const items = selectItemsByCategory(state, ownProps.category);

    return {items}
  }
};

With function components, this is a bit less obnoxious syntax-wise, but still annoying to have to do:

function CategoryItems({category}) {
  const selectItemsForThisComponent = useMemo(makeUniqueSelectorInstance);
  const items = useSelector(state => selectItemsForThisComponent(state, category));
}

Clearly this is a major use case that is difficult to work with right now.

It's possible to customize Reselect's caching behavior by calling createSelector(customMemoizer), but that's an extra level of complexity as well.

Optimizing Comparison Behavior

Reselect works by:

• Passing all parameters to all "input selectors"
• Saving all the input selector results to an array
• Checking to see if any input results changed by reference
• If so, passing all input results to the output selector

However, the use of shallow equality / reference checks here can lead to calculating a new output result in cases where it wasn't truly necessary. Take this example:

const selectTodoDescriptions = createSelector(
  selectTodos,
  todos => todos.map(todo => todo.text)
)

This recalculates the result any time the todos array changes. However, if we dispatch(toggleTodo(3)), we create a new todo object and todos array. That causes this selector to recalculate, but none of the todo descriptions changed. So, we end up with a new descriptions array reference even though the contents are shallow equal. Ideally, we'd be able to figure out that nothing really changed, and return the old reference. Or, even better, not even run the final calculation, because it might be relatively expensive. (Issue ref: #451)

Related to this, it's also possible to write poorly-optimized selectors that have too broad an input (such as using state => state as an input selector) and thus recalculate too often, or may just not be well memoized.

Finally, Reselect doesn't do anything to help with the output itself taking a long time to calculate (Issue ref: #380 ).

Debugging Selector Recalculations

Reselect was made to work with selectors acting as inputs to other selectors. This works well, but when multiple selectors are layered on top of each other, it can be hard to figure out what caused a selector to actually recalculate (see the selectors file from the WebAmp project as an example).

Other Issues

• The TS typings are very complex, and also possibly broken as of TS 3.1 for some cases
• The use of the multi-argument form (createSelector(input1, input2, output)) was bad for TS usage previously. This might not be an issue now with TS 4.x.
• People want to be able to customize more of the API, including additional memoization checks and resetting cache
• Also better error handling, like detecting an undefined selector (which can happen due to circular imports)

Existing Ecosystem Solutions and Addons

Open Reselect PRs

There's a bunch of open PRs that are trying to add various small changes in functionality and behavior. Some relevant ones:

• cache the subsets of a collection when the other parts changes cause the collection reference change #217: equality check for result
• Added a simple FIFO cache to defaultMemoize, with a cacheSize option #238: cache for defaultMemoize
• add memoizedResultFunc to output selector interface #358: expose memoizedResultFunc so you can clear it
• cacheSize option for defaultMemoize #210: another cache size option for defaultMemoize
• Expose selector's result to allow adhoc caching #448: expose the last result so an output selector can reference and return it
• Add resultCheckMemoize #456: adds a resultCheckMemoize memoizer that can be used instead of defaultMemoize

Ecosystem: Caching

There are a bunch of different packages that either wrap Reselect directly, or implement similar behavior separately.

The biggest one is https://github.com/toomuchdesign/re-reselect , which specifically creates a customized memoization function that supports multiple cached keys so that one selector instance can be reused in multiple places.

Meawhile, @josepot came up with an approach for keyed selectors, submitted it as #401 , and also published it as https://github.com/josepot/redux-views .

There's also https://github.com/ralusek/reselectie , which is an alternative lib with a similar API.

Ecosystem: Comparisons

The best option I found for dealing with cases that return arrays and such is https://github.com/heyimalex/reselect-map , which has specialized wrappers like createArraySelector that deal with one item at a time.

Ecosystem: Debugging

The biggest piece here is https://github.com/skortchmark9/reselect-tools , which adds a wrapper around createSelector that tracks a dependency graph between created selectors. It also has a really neat browser DevTools extension that visualizes that dependency graph.

While searching NPM for Reselect-related packages, I also ran across:

• https://github.com/AaronBuxbaum/analyze-reselect : wraps createSelector to keep some stats on call durations
• https://github.com/techstack-nz/reselect-lens : creates a Redux store and uses your selectors as "reducers" to allow visualizing them in the standard Redux DevTools Extension

Alternative Selector Libraries

There's also other selector-style libraries with varying approaches and APIs:

The one I find most intriguing is https://github.com/dai-shi/proxy-memoize. @dai-shi has been doing amazing work writing micro-libs that use Proxies. I think that proxy-memoize actually does solve some of Reselect's pain points, and I want to start officially recommending it as another viable option. I suggest reading reduxjs/react-redux#1653 , which has discussion between myself, @dai-shi, and @theKashey regarding how proxy-memoize works and whether it's sufficiently ready.

@theKashey previously wrote https://github.com/theKashey/kashe , which uses WeakMaps to do the caching behavior.

https://github.com/taskworld/rereselect and https://github.com/jvitela/recompute both use their own internal forms of observables to track dependencies and updates.

https://github.com/pzuraq/tracked-redux uses the Ember "Glimmer" engine's auto-tracking functionality to provide a tracked wrapper around the Redux state.

Ecosystem: Library Summaries

Since I was researching this, I threw together a table with some of the more interesting selector-related libs I found. Some are wrappers around Reselect, some are similar to Reselect API-wise, and some are just completely different approaches to sorta-similar problems:

Repo	Type	Domain	Notes
https://github.com/toomuchdesign/re-reselect	Wrapper	Cache size	cache size and varying inputs, mapped by cache key
https://github.com/heyimalex/reselect-map	Wrapper	Comparisons	specialized selectors to handle arrays/objects
https://github.com/theclinician/selectors	Wrapper	Comparisons	Various wrapper selectors for mapping over collections
https://github.com/techstack-nz/reselect-lens	Wrapper	Debugging	Creates a Redux store with selectors as "reducers" for DevTools viewing
https://github.com/AaronBuxbaum/analyze-reselect	Wrapper	Debugging	Wraps createSelector to track debugging stats
https://github.com/skortchmark9/reselect-tools	Wrapper	Debugging	Creates graph of selectors; has its own DevTools UI
https://github.com/trufflesuite/reselect-tree	Wrapper	Output structure	create trees of selectors with deps between leaves
https://github.com/liitfr/relational-reselect	Wrapper	Output structure	Constructs joins between state sections
https://github.com/jvitela/recompute	Competitor	Cache size	custom observables; unbounded cache size, any num args, can be shared across components
https://github.com/theKashey/kashe	Competitor	Cache size	relies on WeakMaps
https://github.com/ralusek/reselectie	Competitor	Cache size	"Smaller and faster"; has cache key parameters
https://github.com/josepot/redux-views	Competitor	Cache size	Nearly API-compat. Created to inspire Reselect 5.0
https://github.com/taskworld/rereselect	Competitor	Inputs	dynamic dependency tracking instead of static; debug introspection
https://github.com/spautz/dynamic-selectors	Competitor	Inputs	Dynamic selector construction; also can wrap Reselect selectors
https://github.com/pzuraq/tracked-redux	Alternative	Tracking	Uses Glimmer's tracking with Proxies
https://github.com/dai-shi/proxy-memoize	Alternative	Tracking	Uses custom Proxies for tracking dependencies
https://github.com/theKashey/beautiful-react-redux	Alternative	Tracking	Wraps React-Redux; wraps and double-calls your mapState, tracks deps
https://github.com/dai-shi/reactive-react-redux	Alternative	Tracking	Uses Proxies to track state access; also see reduxjs/react-redux#1503

Conclusions

Reselect is Widely Used

For reference, Github shows 1.4M+ repos depending on Redux, and 400K+ depending on Reselect. So, any changes we make should try to keep the API similar to minimize breakage.

Biggest Issue: Caching and Output Comparisons

This seems like the main problem people are concerned about and is the biggest annoyance working with Reselect right now.

Reselect Should Be Updated Even If Other Options Exist

I really like how proxy-memoize looks and I think it's worth us promoting it officially. That shouldn't stop us from improving Reselect while we're at it.

Rewrite Reselect in TypeScript

We might as well unify the code and the types so they don't get out of sync, and start building Reselect against multiple versions of TypeScript.

Coordinate on API Tweaks

There's a bunch of overlapping PRs with small tweaks, and we should try to figure out a coordinated and coherent approach to updating things vs just randomly merging a few of them.

Final Thoughts

So, here's the questions I'd like feedback on:

• What use cases does Reselect not cover sufficiently now?
• What other improvements can be made to Reselect?
• What other changes should be made to Reselect?
• What other pain points have you run into?
• What should a final Reselect v5 API design look like?

I'd like to tag in a bunch of folks who have either contributed to Reselect or are likely to have relevant opinions here:

@ellbee, @timdorr, @josepot, @OliverJAsh, @dai-shi, @theKashey, @faassen, @Andarist, @eXamadeus

I'd like to get feedback from them and the rest of the Redux community!

I'd specifically recommend reading through the "Reselect v5.0" proposal by @josepots and the proxy-memoize discussion over in the React-Redux issues as background for this.