Show HN: Seen – Virtual list rendering with 1M+ notes (seen-v2.vercel.app)
Hello HN! I've been working on creating a new note-taking app called Seen. Right now, it's really just a preview: virtual-list rendering ~1,000,000 notes in a masonry layout, while trying to minimize CPU and memory usage as much as possible.
Seen currently has:
- A client-side search feature
- Creating, updating, deleting the topmost note (locally; no changes are saved to any server)
- Automatic repacking of the masonry layout when the window width changes
- A lot of bugs, I'm sure
I reached a point where I thought I was ready to show the Seen to the world, and this is it. It's pretty basic, I know, but I'm really proud of it.
Seen was born out of an experiment to see just how fast things can become if you use the right techniques, and will (hopefully) achieve this dream sometime in the near future. My vision for Seen is to create something that can handle a lot of notes while using the least amount of resources. If you've got any suggestions or bugs, they're welcome in the comments!
A little about how Seen works: during the first visit, Seen caches note heights for the specific window width. Seen also caches the HTML output for the Markdown notes. It caches height by the key SHA256(title + note content), and the Markdown-to-HTML renders by the key SHA256(note content). It uses this cache every other time instead of recalculating, meaning that if most of the content remains unchanged, Seen can render things pretty fast. Caches are saved to IndexedDB.
When window width changes, the amount of text that can fit in a line also changes. So, Seen caches heights for specific window widths as well, putting the width in the key. Scrolling triggers an event handler that calculates the current viewport and finds & renders all notes that are visible (or partially visible) with a buffer of a 600px.
When a note is deleted or inserted, the entire layout is re-rendered. I plan on optimizing this to only re-render the specific column, which is what happens anyway when a note is updated. Updated notes create new caches (as they should).
38 comments
[ 2.4 ms ] story [ 97.4 ms ] threadI think it would be a lot faster to do this with native HTML.
It seems currently the scroll struggles when you jump around, e.g. when you click to the bottom of the scroll bar area, the scroll will jump twice, first as you click and the second time as it renders. This is not a big issue if you continuously scroll, but is very noticeable if you use page down or click on the scroll bar. At least on my system, there is a noticeable delay between scrolling and the items being rendered on screen, that might be a related issue?
Those notes aside, I could definitely see value for this in cases when you want to get a glance overview or similar of a large number of notes, it's a cool idea.
Yes, I believe the two are related (and related to the scroll event handler). I'll look into that too!
Since you can't CTRL+F them, they're not rendered.
Also... who wants to render 1M notes? Maybe if you can CTRL+F them...
Also, Seen does have search. The problem with making it work with Ctrl+F would be to render all 1M notes (or at least some "shadow" version of them), which would really slow everything down. But again, maybe there is some other way - speed is one of the major concerns, and I'm working on it!
- Too much on band calculation blocking the renderer when the new items come into view.
- Not enough preloading of the items off screen
- Incorrectly calculating the virtual scroll height which causes inefficient re drawing during render.
Not sure what stack you’re running but it seems the table could use some optimization.
I would prefer a note taking app that can render maybe 100 notes but doesn't have that scrolling behaviour.
In a more general case where you can make no assumptions about what you're rendering, you first render whatever is on-screen (so that you can also calculate their heights and position other elements properly). Then you render everything else off-screen, measure the total height and set that as the height for your container (for your virtual list).
Then you could implement some sort of caching for the heights to reduce recalculation for future loads.
If you can make some assumptions, there are a lot of clever ways you could even skip the rendering-then-checking-height method completely. One example I can think of is for is images; the height of an image is already present in the metadata. You just have to multiply it with some scaling factor to make it fit in your UI. That's much faster compared to checking the height after rendering.
What Seen currently does is the first. But the next version is going to do the second: finding a way to set a note's aspect ratio constant across all configurations of screen sizes, so that you no longer have to wait for all that calculation to finish.