Launch HN: Athens Research (YC W21) – Open-Source Roam Research
The problem with notetaking is that we don’t look back at 99% of our notes. Organization systems like PARA, BASB, and GTD can extend the shelf-life of information, but they’re mechanical and tedious to maintain. On the other hand, without a system, we just hope that search will one day have the answers. What comes after search, tags, and folders?
With bidirectional links, you never have to worry about where you write a note. Bidirectional links allow you to connect any two notes together, creating a knowledge graph. Structure emerges organically, bottom-up. This way of joining any arbitrary entities together is a paradigm shift (sorry to be cliché). The graph-based approach subverts the tree-based hierarchy most notetaking apps (and most applications/OSes/computer systems) have used until now. This is fundamental because hyperlinks are primitives, used by every single internet user. Just imagine using computers and phones without links! This is why Athens is about more than just notetaking. I believe networked applications with bidirectional links and data could become a new category itself.
Of course, this bidirectional idea isn't new. In fact, it goes as far back as the origin of the Web. It's the original concept of hypertext and Xanadu, which Ted Nelson has been advocating for decades. More recently, aspects of it were attempted by the Semantic Web. Yet the adoption never really caught on, until perhaps now.
Something else that's interesting about the most powerful networked tools like Roam and Athens is that you can't really make these apps with JavaScript or plaintext/markdown. For maximum power, you want a true graph database. Both Roam and Athens leverage a front-end graph database called DataScript, which is written in Clojure(Script). JavaScript doesn't have a native analog, and Neo4j is only server-side. This matters because I believe this is the first consumer use case for graph databases. I believe both Roam and Athens are general-purpose platforms where individuals and organizations can centralize all of their knowledge and tasks. I believe the graph is the right data structure to do this with.
More broadly, I believe networked applications beyond just notetaking can emerge. These networked applications can develop unique interactions and patterns in-app. They will even be able to share data between one another—Datascript has a native interface for querying multiple databases at once—unlocking new ways for users to synthesize, analyze, and transform their data at scale.
I started working on Athens in March of 2020 after interviewing at Roam Research. At the time, I was certain Roam was going to change the world with their vision of networked thought. Pre-Roam, I was a power user of Notion, even giving a Notion productivity workshop to my friends (where I actually mention Roam https://youtu.be/4HXHk5C3bSc?t=5820). But it was clear to me that Roam was doing something different from Notion altogether, on the graph. It was clear to me that this was about more than just another productivity app.
Unfortunately, I had a glaring lack of Clojure experience. It didn't seem like I was going to get a call back from Roam, so I saw two options going forward. Option one was to find other applications of graphs and bidirectionality. I saw an opportunity in messaging to develop a "Slack Killer." Chat is another place where information was constantly getting lost in streams and siloed channels. Option two was to develop an open-source version of Roam. I wanted to prove I could learn Clojure fast (and hopefully get a return call from Roam). After prototyping these two options for about a wee...
152 comments
[ 2.6 ms ] story [ 219 ms ] threadThe great outliner features of the first two, and the backlinking and graphing of Obsidian. My only real reservation is that files are not in Markdown. Is there a plan to address that, or is a DB central to the architecture? Currently using LogSeq because of this but eager to switch.
CSV -> Excel -> SQL -> Distributed Cloud DBs
Furthermore, our database supports data types, including numbers, dates, etc. I don't think any networked notetaking app has executed well on tables and non-string types. UX for tables is generally not great for markdown.
I guess I can see perf becoming an issue for a company wide knowledge base or a wikipedia type of thing but for a personal knowledge base which is my main use case currently I'm not clear it's necessary to have a real database and the benefits of just a bunch of markdown files are more compelling to me right now. Maybe that would change if my knowledge base got big enough or my needs more complex regarding links and references.
That should be true, but so far it is Roam with the biggest problems with performance, from 30+ seconds of loading each time you load/refresh tab to constant lags in normal usage for some users :)
I personally like the most this mixed approach of Logseq - app internally uses DataScript, but data is ultimately stored in plain files.
Roam's performance suffers mainly on first-load because they are server-first, and they load the entire db into memory at the beginning (such that it's quite fast thereafter).
Once we have true local-first data structures with something like https://github.com/replikativ/datahike, we could still have fast in-memory, but also fast initial load.
That would be cool :)
> such that it's quite fast thereafter
Not always :) I saw that myself and I also see complaints on Roam Slack. In my experience it mostly depends on how many queries you have on given page, but sometimes it lags (long waiting time when switching between pages or opening note in sidebar) also with simple, "atomic" notes. But they overall struggle with polishing existing parts before adding new features, so maybe performance is still on its early days in Roam.
Anyway, I wish you good luck with your project :)
- https://docs.asciidoctor.org/asciidoc/latest/macros/inter-do...
- https://asciidoc.org/userguide.html#X60
- https://asciidoc.org/userguide.html#X73
I am currently using Obsidian, but would love a markdown-backed Workflowy interface that I could switch to if possible.
Perhaps logseq is that app?
1- Why did you choose Clojure?
2- Last I checked, Roam founder here on HN said something along the lines that "the self-host functionality is on their minds"...this was around late 2019 I think. Wonder what's the update on that...
3- Actually, personally if I want to use this approach, I definitely want to self-host. Wonder what would happen to the data if this tool gets updated. Is it just markdown + some syntax for bidi links?
> Something else that's interesting about the most powerful networked tools like Roam and Athens is that you can't really make these apps with JavaScript or plaintext/markdown. For maximum power, you want a true graph database. Both Roam and Athens leverage a front-end graph database called DataScript, which is written in Clojure(Script). JavaScript doesn't have a native analog, and Neo4j is only server-side.
3-Yes, we've already updated the database before within the application. DataScript makes it quite easy to update the schema.
There are a few major differences, but otherwise, the functionality is mostly similar.
[0] - https://www.roambrain.com/implementing-zettelkasten-in-roam/
[1] - https://medium.com/@anthonyclemons1/roam-research-the-digita...
We actually have people doing a live workshop right now on workflows for Athens in our Discord!
One question I have is: how much compatible will it be eventually with other tools like Roam/Obsidian/Logseq (when we want to import the graphs)?
That said, all the best to the Athens Research team.
[1] https://twitter.com/Conaw/status/1229978388787384320?s=20 [2] https://www.theinformation.com/articles/a-200-million-seed-v...
What are you thoughts on the other Roam alternative, Logseq[1]? It seems to be doing really well and has a lot of features, from github sync and encrypted data to custom themes and publishing, it looks like privacy is their main goal, while being local-first and opensource, do you think that such features will be implemented in the near future and will those features become somewhat a standard across PKM's?
Good luck with the launch!
[1] https://logseq.com
All features are possible, especially given similar technologies. The question is about design/product - whether we need to have all these features. Most notetaking apps are too bloated IMO.
From all the Note-taking tools, I liked TW best, as the "story river" is a genious visualization, but I found the text editor very cumbersome and annoying. It just did not feel good to write my notes.
The "tag-interface" always popped up and could only be closed with the mouse, there were some issues with tab, ...
---
I then moved to Dendron and finally to org-roam. I think markdown notes are superior to org-files*, but I also wanted VIM keys, as being able to write without hassle is the single most important thing when taking notes. VSC of course has a VIM extension, but it is very much a second-class citizen. There is also the option to emulate and use nvim, but then I would have to set-up and customize nvim. Why not move directly to Emacs, where Evil reigns superior?
*org-files are very parasitic. I only consider them plain-text in the sense that they can be opened and edited with any text editor, similar to SVG, JSON and XML. But they are not exactly plain, and pretty much unusable outside of emacs/org-mode.
If it will be my second brain (please note how important this sounds), what I want is to be able to be in full control of the data I generated with it.
I'd happily pay for an Open Source SaaS service from which I can download my data and use it in my own computer/server if I wish to do so, much like Sentry or Strapi.
Edit: Clarity, grammar and spelling
I am just pointing out that closed source doesn't necessarily mean 'not in control of your data'.
But if you're doing a complete read-only copy of roam, why not go all the way and make it editable, and compete with them instead?
You could say openness is “a feature, not a product”, and it’s a feature that could be retrofitted.
You could put in a lot of work, just for read-only feature parity.
Or you could work some more, and have a competing product.
You seem adamant, that the former is a better path, but I don't quite see why would you do all that work for naught.
(Roam already broke my backup script 2 times, and there are better uses of my time than interfacing with a proprietary product, if there are better alternatives.)
Do yourself a favor and check if the exports actually have what you need, before blindly trusting it.
Is the answer "host it as a web service" (and corollary "let us do it for you")?
To your second answer, yes and yes. I write in the post:
> As for how we will make money, most users, even technical ones who could self-host, don’t want to self-host (but they value that optionality and insurance against lock-in). They want a subscription SaaS, which will make features like backups, integrations, and collaboration much easier.
Would it be possible to deploy a web-based instance with the server handling storage and collaboration? If so, I might try my hand at packaging it for Sandstorm once that's available. Would make self-hosting Athens super easy for Sandstorm users.
Their page https://sandstorm.io/how-it-works writes:
> For example, when using Etherpad – a document editor app – on Sandstorm, every document lives in a separate container, isolated from the others. The front-end and database for that document live in the container. The container has a private filesystem for storage. JavaScript running in the user’s browser can talk only to the document container for which it was loaded, no others. All of this makes up a single “grain”.
Right now Athens works directly with the filesystem (and therefore filesync services), not a REST-ful/HTTP server. I'm not sure how Sandstorm apps "talk to the document container." If they allow read/write access to the filesystem from the web app, it could work, but I'm not sure how Sandstorm works.
I guess I was assuming that real-time collaboration would eventually involve some sort of synchronization server as a rendezvous point, in which case that would be the natural thing to run on the server side in Sandstorm. That may not be the route you're taking though!
Logseq eventually added an open source option, but that it fails to keep sub trees collapsed pretty much kills it.
My current favorite note applications are Dynalist and Workflowy, but during my use I have always had a bad stomach feeling about privacy, inability to use it for work related stuff and always thought "this needs some features breaking out of the tree structure".
I'm not entirely sure Athens is trying to be exactly what I'm thinking of here, but getting competition in the space is great.
How come? There's no single feature that you can implement in any other programming language that you can't also implement with JavaScript, right?
Main reason is access to datascript, one of the only client-side graph databases, which doesn't work the same with JavaScript.
I highly doubt using datascript will give you a significant advantage over competitors in the long term, and I say this as somebody building pretty much a competitor app with Electron, but that's more a matter of personal opinion so I can only wish you good luck about that.
I don't want to sound like a condescending snob, but I honestly wish someone tried to convince me years ago. I myself have spent over a decade dealing with JavaScript (and many other languages as well). Multiple times I had a chance to learn Clojure, but I dismissed every opportunity. I didn't like it. It seemed to me there was nothing "sexy" about it. And one day, I just decided to give it a try (it appeared that I tried many things, and finally I had to try this). The basics I learned pretty quickly. But the more profound philosophy of the language and its ecosystem took me quite some time.
It is incredibly challenging to attempt to explain in a single comment what makes Clojure so much nicer than everything else I have tried before. And I promise you, I did try using lots of different things before that. But please, I urge you to explore it wholeheartedly, and I promise, you will be rewarded. Accept it with all its intrinsic beauty and unavoidable flaws. I guarantee it - you will never regret that.
Long-term, open-source is great at building communities around a product/project, which is important for this space.
Do you have empirical validation of this claim? Anecdotally, Postgres has a pretty rich set of graph modeling features these days, and keeps getting more of them, and SQL databases are often operationally simpler than graph databases. Maybe this matters less in a client side app?
SQL databases are indeed operationally simpler but that's because they have been around for much longer and are more battle-tested.
[0]: https://youtu.be/tV4pHW_WOrY?t=1772
I realize this seems intuitive, but the relational model has proven itself to be surprisingly flexible in its ability to store multiple different data models efficiently. Similar arguments were made about the suitability of document stores to social networking websites ten years ago, and I think we know how that panned out.
> SQL databases are indeed operationally simpler but that's because they have been around for much longer and are more battle-tested.
I wouldn't discount the age and battle-testing, though. They're the first choice for data storage for many good reasons. Not least of which is the flexibility and power of the relational model.
It's clear that graph query languages are more expressive for graph databases. What's not clear to me is whether there's anything fundamental about that expressiveness, or whether it will eventually be replicated in SQL. Have you tried using recursive CTEs before?
Haven't tried recursive CTEs before, maybe I should dig into it more!
To be clear, I think Datalog and Clojure are both actually really cool, I'm just a huge SQL fan too. I think the architectural decisions you made in this case make perfect sense :)
It's possible that sometimes we have to let early ideas fully succeed to see how the market actually performs. Else one can end up in an arms-race situation that ultimately only leads to a tiny user base for each company.
Sometimes it's healthy when spaces heavily fragment due to hypercompetiton. It allows products with strong technical and theoretical groundwork to show up later and render the entire space effectively obsolete.
1. Market: If the pie doesn't grow, you may be right. If, however, these become new platforms, with the majority of use cases and user value occurring in the future, then there's certainly room for many existing players to succeed, and for brand new players to. Most people don't even take notes. What if knowledge management looks different in 10 years?
2. Consolidation: Notetaking is pretty singleplayer and subjective right now. Until there are "network effects", reasons why people would want to purposefully join the same network, a lot of it will simply come down to taste. Note: network effects does not simply mean real-time collaboration.
Another use case that goes beyond individual notetaking is collaborative knowledge management for organizations. So far Roam's collaborative features have been lacking to say the least.
So basically a Github vs Gitlab scenario. Both are extremely valuable companies. One is more for individuals, the other for enterprises. Knowledge management can go broad and deep, and we're only processing and collecting more information. Never underestimate a growing problem/market.
Could you talk about the clojure part of it? how instrumental was it to build Athens Research. could you have built it on Electron + Typescript for example
We actually do use Electron! ClojureScript compiles to JS at the end of the day.
Ultimately Clojure is pretty necessary to use our database, Datascript, a graph db with its own data query language, Datalog. Datalog is a descendant of prolog, and makes recursive/reverse queries trivially easy.
Hopefully will have a great roadmap too like Gitlab's: https://about.gitlab.com/direction/maturity/
Jira, Github issues/pull requests, all list the linkage from both sides?
eg:
* Which db is the source of truth?
* How will conflicts from offline editing be handled?
* Will you be able to see edits made in realtime across the devices?
Hope that this local first solution will make you avoid the performance problems those apps have
[1] https://github.com/MatthieuBizien/roam-to-git
[2] https://github.com/adithyabsk/keep2roam
I love this setup as there is no internet needed, full text search on all the documents and very less friction while starting to write something. I have tried other apps like obsidian, roam etc but keep coming back to this one due to its simplicity.
I'm not saying paper systems don't have a place - people have put them to use for centuries after all. They do have very real limitations though, and to be really useful require more ongoing work than many people are willing to put in, especially if they grow to an appreciable size.
Interesting way to monetize, not sure how I feel about that, but it would be nice if you'd at least list what data you are collecting (or link to your privacy policy/faq/what ever.)