Ask HN: Resources on application data preloading and caching strategies?

by martijn_himself on 5/4/2021, 12:00:53 PM with 2 comments
Intuitively it seems to me that the major issue with perceived slowness of large business applications (such as CRM systems) is the absence of any pre-loading / caching of data for the user.

In my experience most users only work on a small subset of data and work on the same or similar records and processes every day, so I'd be interested to have the application learn what subset this is for a particular user or group of users and then do 'pre-emptive' loading to increase the responsiveness of the application and improve the user experience.

If I want to avoid re-inventing the wheel, what are some good resources that deal with strategies on how to achieve this? What are common pitfalls?

  • by bem on 5/4/2021, 12:51:06 PM

    One powerful way to deal with these problems is event sourcing. It's a reasonably elegant way to materialize a single application-specific cache based on many different data sources. Two great resources:

    https://engineering.linkedin.com/distributed-systems/log-wha...

    https://queue.acm.org/detail.cfm?id=3321612

  • by ianpurton on 5/4/2021, 12:42:50 PM

    A good example is superhuman the email client. They cache data in the browsers local web sql database.https://en.wikipedia.org/wiki/Web_SQL_Database

    Then all searches etc are fast, due to not having to go to the server.

    I'm not sure how they implement that, but I would probably go with a websocket that sends the latest data to the client which then stores it in Indexed DB (the browsers local database).

    I don't know of any solution that does this out of the box.