Reactive updates is the big one, in my opinion. DataScript is a triumph and arguably is the reason why so many note-taking tools (Roam, Athens, Logseq, etc) are written in Clojure. But there are so many cases where it would be nice to react when some set of entities is changed.
I think what we need is to figure out how to combine DataScript with a rules engine. I wrote a rules engine and made a writeup that compares the two together: "Using O'Doyle Rules as a poor man's DataScript" https://github.com/oakes/odoyle-rules/blob/master/bench-src/...
Subscribing to individual entities is nice but with a rules engine you have so much more fine-grained control over your reactions. And with the RETE algorithm this can be done efficiently. Most libraries in this space just ignore it and make their own ad-hoc solution -- an informally-specified, bug-ridden, slow implementation of half of a rules engine.
The rules engine direction is definitely an interesting one, hopefully there’s more exploration into that in the future.
For my needs now I actually wrote something pretty hacky[1] to get a 'reactive' version of DataScript’s Entity API.
As Nikita mentions in the article, I’ve found that most of the time I don’t really need queries in the UI, and so the entity API ends up fitting quite well – except that it’s non-reactive.
The `reactive-entity` library implements a reactive version of (most of?) the entity API, so you can pass entities in/through components, and then the components only re-render when the attributes accessed within those components change (including attributes accessed on other entities through refs).
Loads of room for improvement but I’ve used it successfully in a couple of projects.
How come there is no SQLite of Datalog? You can find plenty of implementations of embedded datalog database in a specific language, where the query API is tied to the language. I want to write text datalog queries and access my database from multiple languages. Why doesn’t this exist?
> GNU Prolog accepts Prolog+constraint programs and produces native binaries (like gcc does from a C source). The obtained executable is then stand-alone. The size of this executable can be quite small since GNU Prolog can avoid to link the code of most unused built-in predicates. Beside the native-code compilation, GNU Prolog offers a classical interactive interpreter (top-level) with a debugger.
I’m looking for more of a library that integrates with other programs, not a standalone program itself. I’ve seen other prolog implementations that fit the bill but they seem to rely on parsing program text from a clean state—they don’t “remember” facts on their own.
No, the standard evaluation model of Prolog is top-down, while that of Datalog is bottom-up. As such, some queries will perform very differently between the two.
Very good pointers, match my thinking on the topics I had considered.
One thing though is that I'm not a fan of uuids, I think content addressing is sufficient if you build everything around that. Actually I feel all IDs can be content address and type/name/context. Ofc I'm approaching this slightly differently, I don't care about web browsers but I care about p2p and replication..
As great as content addressed IDs are, they only allow you to build DAGs.
Many domains require a graph to be properly modelled, a collision free string of random entropy is the best thing you can get, in terms of distributed-ness, for these scenarios.
There is a certain duality to unique identifiers and content addressed identifiers, both are best utilised together.
Unique identifiers are the antithesis of content addressing since they garble the advantage (digital commons or "torrents").
If you want a graph then you can just encode it in the data by having `n` content addresses and ca. `n^2` bits for the adjacency matrix.
The DAG of content addressess is only in the abstract when you have "unflattened" data and all you can do is chase the references to earlier content addresses. If you have some semantics for relating content addresses then you can flatten stuff into graph structures (or whatever you desire) by providing the relevant metadata (that relates the data in question).
Now you have more canonical data that can be content addressed.. This is the essence of what I am doing with datalisp.
Oh boy, their blog[1] is really painful to read for someone with a database background. They conflate so many different concepts, like datalog and triple-stores, apparently having seen only Datascript/Datomic.
The big question is: if you don't need queries (which I agree you don't), then why bother with DataScript at all? Why not just store your data in native Clojure/ClojureScript data structures, using the right structure for each thing and use a small number of data access functions, possibly maintaining an index or two?
This is what I did: migrated an app (arguably a fairly complex one) from DataScript to native Clojure data structures. Not because I didn't like DataScript: I actually liked the idea a lot, but because I couldn't justify the cost in performance and complexity (in my case, DataScript not handling nil values was a problem as well).
The "Optimized B-Trees" section I _think_ is suggesting to get rid of datoms, which I 100% agree with. I do not think they add anything at all; IME you can have a collection of all attributes indexed by entity ID and then have additional indexes on top of that collection.
My stupid question is: why even bother with B-Trees? I believe asami[0] stores everything in memory using Clojure maps & sets.
I'm curious about this too. I think the clojure maps are backed by RRBTs (relaxed radix binary trees), and I've wondered how possible it is to take advantage of that structure for writing algorithms on top of (vs it being sort of "behind the scenes" only, taking care of immutability/persistence).
23 comments
[ 1.4 ms ] story [ 57.6 ms ] threadI think what we need is to figure out how to combine DataScript with a rules engine. I wrote a rules engine and made a writeup that compares the two together: "Using O'Doyle Rules as a poor man's DataScript" https://github.com/oakes/odoyle-rules/blob/master/bench-src/...
Subscribing to individual entities is nice but with a rules engine you have so much more fine-grained control over your reactions. And with the RETE algorithm this can be done efficiently. Most libraries in this space just ignore it and make their own ad-hoc solution -- an informally-specified, bug-ridden, slow implementation of half of a rules engine.
For my needs now I actually wrote something pretty hacky[1] to get a 'reactive' version of DataScript’s Entity API.
As Nikita mentions in the article, I’ve found that most of the time I don’t really need queries in the UI, and so the entity API ends up fitting quite well – except that it’s non-reactive.
The `reactive-entity` library implements a reactive version of (most of?) the entity API, so you can pass entities in/through components, and then the components only re-render when the attributes accessed within those components change (including attributes accessed on other entities through refs).
Loads of room for improvement but I’ve used it successfully in a couple of projects.
[1]: https://github.com/austinbirch/reactive-entity
[1] https://github.com/pubkey/rxdb
https://github.com/ccorcos/tuple-database
E.g. GNU Prolog Compiler: http://gprolog.org/
> GNU Prolog accepts Prolog+constraint programs and produces native binaries (like gcc does from a C source). The obtained executable is then stand-alone. The size of this executable can be quite small since GNU Prolog can avoid to link the code of most unused built-in predicates. Beside the native-code compilation, GNU Prolog offers a classical interactive interpreter (top-level) with a debugger.
Strikes me as a basically sound idea and it would be lovely if someone picked up the ball.
you may be interested in https://github.com/sqwishy/owoof
One thing though is that I'm not a fan of uuids, I think content addressing is sufficient if you build everything around that. Actually I feel all IDs can be content address and type/name/context. Ofc I'm approaching this slightly differently, I don't care about web browsers but I care about p2p and replication..
As great as content addressed IDs are, they only allow you to build DAGs. Many domains require a graph to be properly modelled, a collision free string of random entropy is the best thing you can get, in terms of distributed-ness, for these scenarios.
There is a certain duality to unique identifiers and content addressed identifiers, both are best utilised together.
If you want a graph then you can just encode it in the data by having `n` content addresses and ca. `n^2` bits for the adjacency matrix.
The DAG of content addressess is only in the abstract when you have "unflattened" data and all you can do is chase the references to earlier content addresses. If you have some semantics for relating content addresses then you can flatten stuff into graph structures (or whatever you desire) by providing the relevant metadata (that relates the data in question).
Now you have more canonical data that can be content addressed.. This is the essence of what I am doing with datalisp.
[1]: https://www.instantdb.com/
[1]: https://www.instantdb.com/essays/datalogjs
This is what I did: migrated an app (arguably a fairly complex one) from DataScript to native Clojure data structures. Not because I didn't like DataScript: I actually liked the idea a lot, but because I couldn't justify the cost in performance and complexity (in my case, DataScript not handling nil values was a problem as well).
My stupid question is: why even bother with B-Trees? I believe asami[0] stores everything in memory using Clojure maps & sets.
[0] https://github.com/quoll/asami