68 comments

[ 3.5 ms ] story [ 129 ms ] thread
I bet if Datomic were open source it would get adopted at an incredible pace and become one of the go-to tools for working with data.
But would it put Rich Hickey's children through college?

This is just not something we've got a good, reliable answer for. E.g. I'm pretty confident that his creating and being the BDFL of Clojure wasn't sufficiently remunerative.

I think there are a lot of valid reasons for the current model and I'm pretty sure it (and the changes over time) are well thought through.

That said I also think it is worth to regularly revisit what an open sourced Datomic could do for adoption of Datomic itself as well as for Clojure (edit: not to mention what the impact would be like for the world at large). I'm optimistic that there are monetization opportunities through SaaS, consulting and other licensing concepts.

Ideas?

Being the BDFL of a rapidly growing programming language has its benefits. Even Rasmus Lerdorf has an impressive CV, despite a public attitude of negativity towards programming.

If Datomic achieves widespread success with both community and enterprise editions, Rich Hickey would have an impressive range of opportunities.

The man, however, is far more accomplished than I so I can only assume he's thought of that and firmly rejected it. Perhaps, although he wants to see his kids go through college, he also wants to be a big part of their lives. Perhaps 2 massive, popular over source projects would get in the way of that.

But would it put Rich Hickey's children through college?

I think it would. What it wouldn't do is give Rich Hickey the control he currently has over his life and his projects. He'd likely end up getting paid to work on Datomic by some big company instead of running his own as he does right now.

Isn't Cognitect mostly a consultancy already? Revenue wise, I mean. I don't mean to discount the incredible amount of work they pour into Clojure/Script and Datomic.
It would be nice if we had more well-understood funding models for software. The games industry has some interesting ideas eg:

Unity - free community version, pay monthly for extra features and to remove the splash screen, option to stop paying and keep license but stop receiving updates

UE4 - free, source code is on github (but not FOSS), takes 5% of your gross revenue after first $3000

Lumberyard - free, source code available (but not FOSS), you must either run your own private servers or use AWS

Given the change in licensing terms, I'd expect open-sourcing the db would be the next step. Enterprise support and professional services revenue would increase consummate with adoption.
This is kind of what I expect Datomic's path to be as well.

Early adopters will take most of the financial burden in order to recover the development costs, and open sourcing will happen once it's more strategic to use Datomic as a way to promote their knowledge and services broadly than it is for them to make money directly from the product.

Support tiers will remain even after open sourcing, of course. I don't expect it to turn into abandonware.

I hope Datomic does well and Rich easily puts his kids through college.

But as a developer I think the current license is a bit of a shame. I've played with Datomic, think it is fantastic, and would love to use it.

However:

- After the closure of products such as FoundationDB and Parse, I will never have a significant dependancy on a closed source product again. (maybe if the company was a behemoth like Microsoft or Oracle, but these days probably not even then.) [1]

- As a little guy, the $5000 I would need to pay after one year to keep getting updates to the product is just too much. One could argue that Datomic are looking for bigger fish, but something to consider is how important it is to get people using your technology (al la Microsoft 'developers developers developers!') Little guys can go on to be bigger guys making the important technical decisions. There is post above looking at the licensing done by game engines such as Unity, where larger companies pay more and the little guys are given a break until they are making a profit. Hopefully we will see more of this in the future.

[1] https://www.wired.com/2015/03/apple-pulls-plug-tech-company-...

$5000 doesn't seem like a lot of money for a commercial endeavour.

I'll just keep using the free edition for my toy/side projects.

Datomic free edition also sounds great for small open source projects.

It wouldn't be suitable for large scale free SaaS websites but I feel that's a tiny slice of developers.

Yep. Then we would also use it to build high-level research data processing tools at university (http://pharmb.io).

Right now, I'm sifting through the internet to find any other (up to date) similar system supporting datalog querying.

Let me know if you know about one!

(Only one I found so far being Cascalog [1], which is unfortunately only supporting old sluggish hadoop)

[1] http://cascalog.org

www.logicblox.com is one but it's also commercial/closed source afaik.
Thanks! Yea, we pretty strongly favor anything open sourceish, but it doesn't hurt to at least know about the other alternatives though.
Many programming languages are wonderful for working with data, are you sure you need to go full Datalog? Restricting yourself to a simple subset of Haskell or using something like Clojure itself (for which you can find Datalog libraries btw) to build your query engine seems like it would offer much of the same expressive declarativity of Datalog.
Thanks! Datalog library for clojure is something I should definitely look up.

Will think hard about the suggestion of using some functional language instead also ... you for sure got me thinking! :)

Feel free to share any pointers regarding this!

In the meanwhile, sharing my thoughts so far: What I have so far thought was pretty "unique" about datalog is that it support "named queries", which can be used as a kind of "duck typing" classification, to build up a model of your domain of interest that is always computed on-demand, leaving the raw data unchanged in it's default ("normal"?) form.

Because of the "named queries", you can combine lower-level queries in higher-level ones, to create a hierarchy of "dynamically generated concepts", which I haven't seen any other technology be able to do.

(While named queries are conceptually similar to views in SQL, they are so much more natural to work with.)

I know this by experience from my dabblings in prolog [1], but haven't found any large-scale prolog data system supporting distributed computing (I understood it has some features making it unpractical for distributed computing and more, in a way that datalog doesn't).

[1] http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A3988... (PDF: http://www.diva-portal.org/smash/get/diva2:398839/FULLTEXT01... )

Well, thinking about it, I guess one could achieve aspects of this by using lazy iterators in a functional programming language too, but it still seems to lack the declarative nature that you get with the unification / pattern matching of datalog/prolog ... I might be wrong though.
> Because of the "named queries", you can combine lower-level queries in higher-level ones, to create a hierarchy of "dynamically generated concepts", which I haven't seen any other technology be able to do.

This is the purpose of functions in most programming languages. A function is kind of like an HTML template, but for code. So rather than have `<h1 title="{{bar}}">{{foo}}</h1>` you have `function(foo, bar) { return h1({title: bar, text: foo}); }`. DOM tree, AST ("abstract syntax tree"), relational data, same thing: a graph, and most programming languages can represent raw graph nodes/edges easily as lists of strings. You can write a simple tuple-matching engine in Javascript: https://gist.github.com/notduncansmith/7858d27a2b875ff10b84

Obviously that's not a production-ready piece of software, but it highlights an important lack of distinction between "database" and "program" that wasn't readily apparent to me as a developer for a while.

HTML, source code, and user input are all the same thing (just data, really). One program's source code is another's input data, all the way down.

If you want to get existential, you can extend this even down to matter itself as simply a representation of information: imagine a physical machine in a factory that winds jack-in-the-box toys on their way out the door. Same thing as a higher-order function.

    function wind(toy) { 
      return toy.lever.rotateByDegrees(360); 
    }
So basically, code is data is matter, your source code is just electricity in a machine and whether you look at it as "code" or "data" is simply a matter of perspective. This is what Lisp folks mean by "code is data".

Moving from the abstract to your concrete case:

> build up a model of your domain of interest that is always computed on-demand, leaving the raw data unchanged in it's default ("normal"?) form. [...] (While named queries are conceptually similar to views in SQL, they are so much more natural to work with.)

SQL views don't compose at the same level that Datalog expressions do, but that's because they aren't typically expressed as raw graph data (i.e. lists of strings where each list item is a node in the graph). If you could just name arbitrary chunks of SQL AST and mash them together, it would be just as composable (though perhaps more cumbersome due to boilerplate).

One final thing before this already-too-long comment comes to a close: you might give CouchDB a look. It's a document datastore that offers map/reduce views written in Javascript (use as much or as little composition/abstraction as you want). The distributed features are pretty cool (bidirectional sync, versioned documents) and writing apps with it is awesome, especially if you can leverage supporting projects like PouchDB or Couchbase Lite. Not sure if it fits your use-case but it sounds like you might find it personally interesting.

Hopefully this has been helpful!

Awesome, thanks for the elaboration!
The big news for me is that now you can use all of Datomic's functionality at scale for free with Datomic Starter. You only pay if you want support. Until now, to go beyond 2 instances, you had to pay $3000 per process, which sounded like a nonstarter.
Datomic Starter is now limited to one year of updates, after which you can only use whatever was the current version when your maintenance expires. After that if you want updates then you need to pay $5,000.
> After that if you want updates then you need to pay $5,000.

Bummer.

$5k is less than my side project makes in a year.

And the average annual system maintenance budget for a small client.

you can go pretty far wo/ updates, for a side project could be totally OK.
I've deployed Datomic on AWS using cloud formation to spin up two transactors for when one goes down under pressure.

Throw out everything you ever thought about databases. It is from here you will learn a completely new way of dealing with data especially in environments where you need an immutable audit trail.

I almost feel like Datomic solves what blockchain does (immutability) really well without the drama.

Clojure was also another culture shock. I like it personally but the syntax at first was scary (but it's really simple and straightforward).

Having said that I'm not 100% sure whether I realized any productivity boost working with clojure. Finding clojure developers is tough & expensive vs Java or .NET or PHP.

Datomic has a bit of a learning curve and I still haven't fully crossed over, relying on google groups discussions and it's not a user friendly experience which I feel may be their limiting factor (they've made it far too difficult for the wandering developer to approach it) and the fact that it's closed source and proprietary but this is most likely because a lone developer hobbyist has no business using Datomic because it really makes sense in industries where you need a fully auditable trail, something that was typically available in event sourcing frameworks with some sort of CQRS wizardry that I've had trouble justifying using.

All in all, unlimited peer is a welcome news. However, datomic in all in it's shape or form, only applies to a small group of developers it seems, which is a shame because datalog queries blows SQL out of the water.

The stuff one can do with datalog on datomic tightly integrated with clojure, it's game changing, but won't apply to all use cases for folks on HN.

Can you elaborate on "datalog queries blows SQL out of the water"?
Not OP, but at my company we're heavy users of Datomic. Some examples:

- Declarative and pattern matching make it elegant to read.

E.g., find me the ids of all customers invited by a customer named "Joe":

    {:find [?customer]
     :where [
        [?inviter  :customer/name       "Joe"]
        [?customer :customer/invited-by ?inviter]]}
- Syntax more appropriate for code generation and introspection (since it's just a bunch of nested lists). A big plus when pairing w/ Clojure.

- Allows recursive queries.

- You can reference metadata associated to any transaction that ever touched any attribute in your database within the same query constructs.

E.g, find me the time customer name was last modified:

    {:find [?customer ?name ?last-modified]
     :where [
        [?customer :customer/name ?name ?tx]
        [?tx :db/txInstant ?last-modified]]}
- You can easily query the entire history of records.

E.g, find me the name every customer ever had, w/ date of modification

    {:find [?customer ?name ?last-modified]
     :where [
        [?customer :customer/name ?name ?tx true]
        [?tx :db/txInstant ?last-modified]]}
Almost the same as query above, only difference would be executed using a different Datomic API call (http://docs.datomic.com/clojure/#datomic.api/history).

- You can easily execute a query against how your database looked like months ago.

Same thing: would look like the query above, but executed using a different API call (http://docs.datomic.com/clojure/#datomic.api/as-of)

I would say these are the main ones, but there's more.

Awesome, thanks! The recursion looks compelling. I'd be interested to know if anyone has compared Datalog and Neo4j/Cypher.
Cypher doesn't have named queries (and thus not recursive ditto either), AFAIK. I asked some Neo4j guys on a meetup some time ago, and they didn't even seem to really understand the question at first.

It's a pity. I'd be very happy to use Neo4J if it had.

Interesting. It seems like to query a graph is implicitly recursive (especially compared to SQL), although I can see how something like pagerank might need more.
Neo4j focuses on modeling nodes and it's relationships, and allows you to attach metadata to the later, but doesn't really have transactions as first-class objects.

Datomic focuses on modeling facts (entity-attribute-value). This is a very stable data representation that allows modelling tables, documents or graphs [1], on top of carrying transaction explicitly on an additional index for the time-travel magic.

I guess Neo4j might make a good DB for ETL-ing into and running certain kinds of analysis.

[1] https://hashrocket.com/blog/posts/using-datomic-as-a-graph-d...

Awesome, thanks. I'm building a search engine (https://www.findlectures.com) and I've been looking at ETL-ing the data into/out of Neo4j to do some data augmentation, but the history feature of Datomic is really compelling if I used it as a primary data store.
Do you know if Clojure devs are expensive compared to Python and Ruby devs of similar experience? I feel the comparison to Java, .Net, & PHP maybe just a comparison of huge mainstream language to more "niche" languages.
Here in NYC, our clojure devs are not much more expensive then ruby devs. With the added bonus that clojure tends to yield more efficiency then ruby.

Disclaimer: 12 years ruby/rails experience 3-4 years clojure experience

> I almost feel like Datomic solves what blockchain does (immutability) really well without the drama.

Well said. Unless you need to solve the Byzantine general problem there are better ways to database audit trails than Blockchain. e.g. Datomic or time travel queries.

Datomic has a centralized transactor serializing what transactions, whereas Blockchain is decentralized. So they are at opposite ends of the spectrum, each trying to solve a completely different problem. Of course "centralized" is much less dramatic.
+1 on blockchain. Most of these "Blockchain" fintech company are just doing a distributed logs between trusted peers. nothing more.
Does the "Maintenance/Updates Limited to 1 Year" bit mean that you'd have to get a new starter license after that, or that you're expected to start paying for updates after one year?
If past experience continues, you can renew the starter license.

Edit: sounds like that's no longer the case. See sibling comment.

You will no longer be able to renew Starter after the 1 year period on the same account. Renewal only covers the ability to use newer versions and upgrade. The license is perpetual and as long as you are not upgrading versions -- you will be able to continue using your Starter license.

As was discussed in Datomic Slack:

"Starter was always intended as a path for customers to explore and use Datomic in a low-risk, low-cost approach as they developed their applications and moved toward production. We feel that a year is generally sufficient time to evaluate a product and develop a business application around that product. If you feel that you require a longer period to evaluate or develop against Datomic, please contact us."

Thanks for the clarification.
I wholeheartedly approve of the new licensing. The previous licensing was incredibly restrictive for microservices and for scaling out reads in an elastic way. The new model is much easier to understand and use in the ways that datomic was intended to be used.
(comment deleted)
Datomic is awesome. However, I am concerned about the Datomic Starter tier which states "The Datomic Starter license is perpetual, but is limited to 1 year of maintenance and updates." Previously we could download and use Datomic without registration with some sensible limitations. This was ideal for small projects that don't need support. We could update our Datomic instance to keep up with bug fixes and feature enhancements. Are we now locked into a 1 year window of code quality and features based on our registration?
I think you are talking about the"Datomic Free" option which is still available: http://www.datomic.com/get-datomic.html
Thanks. I didn't scroll down because it isn't listed with the others.
It's unfortunate that the "free" version only supports two peers. No scaling and/or convenience for open source projects, apparently!
(comment deleted)
> Are we now locked into a 1 year window of code quality and features based on our registration?

Yes. Datomic Starter (nee. Datomic Pro Starter) now gives you one year of updates for free after which you need to pay to receive further updates. Previously you could renew your (free) Datomic Pro Starter license perpetually.

Datomic Free is unchanged and is still free and receives continuous updates.

I've written a blog post summarising the changes here: https://danielcompton.net/2016/11/29/guide-to-datomic-licens.... In a nutshell:

* No limits on process count

* Datomic Pro Starter is now limited to one year of usage, then you need to pay $5,000 for Datomic Pro

* Riak, Couchbase, and Infinispan are now legacy storage and only available through Datomic Enterprise.

This whole thing is such a sad tale. Clojure has had lame adoption, starting with its hard to pronounce, hard to explain name. Datomic could have been a great booster for the language, but had a license that is counter to all that has been propelling the programming world. Now they have swapped that out for another confusing licensing scheme. It feels like just another chapter in the long weird saga of lisp non-adoption.

I don't understand why Rich doesn't understand that he'd make far more getting a tiny slice of the huge pie that open sourcing would provide. Yes, they've changed the licensing approach because it's obvious the previous one didn't work out at all. But they haven't done enough, and by the time they don't figure this out, people will be stuffing everything into cloud services, even if they are inferior approaches.

What is confusing about this licensing? Previously it was a mess and prohibited our use case, but this looks pretty straightforward.
"The Datomic Starter license is perpetual, but is limited to 1 year of maintenance and updates".

No updates after a year? To the new wave of developers, this is foreign. Most won't know what that means, but at first glance, it looks like you'll be paying $5,000 in a year. Even the most green developer knows you need updates. Also, it's unclear what a 'system' is without digging around, and many potential users won't bother.

The problem is that Cognitect knows how awesome Datomic is, but the world doesn't. Many potential users are not going to try software they know they're definitely going to have to pay $5,000 for in a year. Updates are critical. Cognitect should go for massive adoption and nothing else. They'll make far more money.

This thread shows continued confusion and uncertainty about the license, and if a potential user googles 'datomic license' they are likely to find this thread. After reading up and down it, they will be deterred from getting into it all. Why try Datomic and get into a potential mess with your employer over licensing issues/costs in a year when you could play it safe? I'm sure those who think this licensing approach is the right one are clear the advantages of Datomic are worth the risk, but they are not paying attention to how software is actually adopted at scale.

It's so sad it's almost 2017 and this issue's been around since 2014 ( https://news.ycombinator.com/item?id=7657701 ) and this is the solution. The word Datomic means nothing to 95% of mainstream tech. This change is a modification of their old model. They need an overhaul of how they are doing this.

No offense - but if $5K is an issue for your employer, after a year of essential magic, you might have the wrong employer and they might have the wrong employee...
It seems like the latest generations of software developers (which includes myself) is hazardously opposed to spending money on software, even when it can save/make you huge amounts of time and money.

I was lucky in that my first job was at a small non-software company that forked over huge amounts of cash for Oracle because it literally enabled their business to succeed (mysql and postgresql were nowhere close to being able to do it at the time). And yet I've seen fellow developers waffle over spending $60 on something they will use literally every day.

I like many things about datomic and was in a position to use datomic for a product twice. Each time when I looked at the license it basically said cognitect will make you your bitch now and you're not allowed to talk about it to anyone. We live in a world of ridiculous software licenses, but it was the most ridiculous license I remember ever having encountered.

This for a product which required a big leap of faith already (designing your data access around a radically different DB with enormous lock-in, supplied by a small consultancy).

Thankfully on superficial inspection it seems like some of the most egregious clauses have gone (Ctrl+F for "disrepute" and "immoral" here: https://blog.goodstuff.im/datomic_license to get a flavour).

I'm wondering though: has anyone here ever used datomic after some laywer looked at their previous license? In the unlikely case anyone from datomic cares to comment: did changing the license help sales?

I've only read the blog post you linked and scanned the current license. I don't find anything unusual in them for a proprietary license.

Most of the cited examples in the blog post are common standard language used in license agreements. I'd say the interpretation of what they means is idiosyncratic. One could argue that the clauses were not nice, but that's a different argument to the one presented.

Their current Eula you linked is pretty short, I'd probably argue with the indemnity, but not unless I was paying enough money that someone would care. Average legal fee to discuss a license ~600 USD an hour, their license is 2,500 USD. Most people have tougher license terms in their office lease!

Well, maybe I'm wrong and the license is in fact reasonable. I'd be more inclined to believe that if you are either a laywer or can provide some examples of software licenses for non-obscure enterprise software that have similar clauses that make you swear never to perform any act or omission(!) that might adversely affect Cognitect's interests or use the software for "immoral/unethical" purposes etc. I can't recall ever seeing that in any product I evaluated. There's a bunch of other problematic stuff that would make me wary of using such a vendor, but that's not atypical (benchmarking etc.).

To be clear, I don't have a bad opinion about Cognitect (other than their choice of license).

It's the Internet so who needs skills or experience to comment!

But, I did run a commercial team (lawyers and negotiators) for a software company that handled hundreds of incoming and outgoing licenses: including ones to major software vendors like Salesforce, Oracle, Microsoft and IBM. I am not a lawyer though ... but, sometimes I play one on the Web.

I'm not trying to tell you the terms are "reasonable", that's a matter of opinion. I'm telling you that the way in which they are interpreted in the blog post is not in line with how they would commonly be read. It is not a fair reading of the terms as far as I can see - consequently saying they have a "bad choice" of license makes no sense since the clauses are being cherry-picked.

Second, people often think licenses have to be 'reasonable', they don't. They have to be enforceable and in your businesses interests - Cognitect clearly thought those terms were in their interests.

I'm not going to comment on the clauses further, you've clearly read them one way and I was offering the observation that I would not have come at them from that perspective. I guess the best way to have found out their intent would have been to call them up and ask them, getting clarity added when you negotiated the deal terms where it was required. Most commonly, people have terms in their licenses because of specific issues they are concerned about and when you understand that something can be done to "fix" the difference of perspectives.

Finally, the new license looks eminently reasonable, so perhaps they noted the concerns and have now fixed them, no?

Wish the Datomic team would release a datastore-adapter API so we can add datastores without having to wait and hope for the team to release them.

There's been an open issue for years to add support for Google Datastore to bring Datomic Google Cloud support into parity with AWS.