Ask HN: How to implement caching for dynamic user data in sites like HN, Reddit?

2 points by rishav_sharan 7y ago ↗ HN

Hi guys. Need some thoughts on how can implement caching for sites like HN, Reddit etc which have a lot of dynamic user metadata (eg. liked comments)?

Earlier on, when I only had the user name ("Hi User Alpha") on the navbar, my plan was to send the entire HTML which looks the same to both logged in and non logged in users. Then to use Javascript to add the user info which I could get from the browser cookies.

However, Now I am looking to add more features like comments with likes and tags (with voting for tags), and I am confused as to how can I cache any of it. How do sites like HN, Reddit etc cache such dynamic user data?

3 comments

[ 3.4 ms ] story [ 15.3 ms ] thread

ioddly 7y ago ↗

Why would you start by caching it?

What are you storing the data in currently? If relational, I'd advise starting with simple relational tables (post_comments, user_comment_likes, etc). Render the HTML for logged in users based on querying that, and see how far you can get.

The source code for historical versions of both reddit and HN is available, so you could check that to see what they do. But caching probably won't be necessary for a long time.

majidazimi 7y ago ↗

Depends on the scale. With few users, you'd better query database directly. With enough users (when it makes sense to introduce complexity), I would adopt Change Data Capture pattern. A small process reads/follows database logs (journal) and when a new post or comment is added, updates the cache asynchronously.

cimmanom 7y ago ↗

Are you asking how they cache pages when logged in users have per-user data? You can use edge-side includes, or use a separate AJAX request to retrieve the user-specific data.