> OpenSearch is a community-driven, open source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists of a search engine daemon, OpenSearch, and a visualization and user interface, OpenSearch Dashboards.
from the main page. So it’s essentially ES before they switched to a new license.
I believe it also includes some plugins that were not included in the open source version of ES. One being security, that allows you to create user accounts and grant access. Another being anomaly detection.
Yes, not a lot of changes. I've been digging through the commit logs to see if this project is attracting new developers. There's a little bit but most of it is trivial stuff. So, not a lot yet.
As a long term Elasticsearch user, I don't like their move away from OSS. I think its long term bad for them and it puts me on the spot when I have to consult my customers on what to use. And yes, I've already had a few conversations on this topic. People seem to prefer OSS and this whole thing makes them default to opensearch/aws.
It's hard arguing with that. Whatever Elastic says, it's a closed source license now and it largely cuts them off from external contributions. At least, I don't tend to make closed source contributions for free. Which is a shame.
Whether Amazon can turn this project in what it claims to be already: a thriving oss community, remains to be seen. As of yet it's basically a largely unmodified fork with not a whole lot of feature work happening (if any at all). Turning that around is going to require some skilled external contributors to join. At least, I don't see the current team achieving a whole lot as it appears to be rather small and most of their work so far has been tinkering with the build system and editing some licensing texts.
The good news is that 7.10.2 was a fine version. The bad news is that this fork is not getting the fixes, improvements, and performance work that mainline Elasticsearch is getting. Also, Elasticsearch 8.x at some point will become a thing and is likely to have some improvements. It's actually been unusually long since the last major release. Before 7.0 they had major releases every year. Of course, moving to new versions of Lucene will be a some non trivial work in terms of implementing and testing the changes. At some point the forks will diverge or opensearch will simply fall behind. The latter seems to be happening. Elasticsearch. 7.11-7.13 have added a few new features that opensearch won't have.
They forked the code but not the developer community. Not yet at least. That could just be a matter of time. IMHO Elastic should kill this fork by rolling back their license changes. There's nothing to backport from this that is worth keeping as far as I can see. So, not a big change technically. Not too late to admit they are simply wrong. They seem to have dug in on this so probably not going to happen. But one can hope.
With OpenSearch, you can start your own community and drive it, whereas with Elastic you can't unless you're willing to promise your fork won't get resold as a hosted service.
In all seriousness, is Amazon good about accepting PRs and stuff from volunteers for their oss projects (i.e. Firecracker)?
So far, Amazon have been good about accepting PRs from volunteers (I can only vouch for the OpenSearch project) and have committed to moving to a more open source model with active maintainers from other organisations. It's early days but it looks healthy so far.
Sorry, do not follow that area. Can someone explain to the uninvolved?
Who changed from which license to which?
The owner of the code can of course change the license. But in typical open source projects there are 100s of contributors, so changing the license can be anything between a major job and impossible.
If there’s only one owner, either through limiting the contributions or through insisting on a CLA, then that owner can do what they like moving forwards. They can’t change things retrospectively, though.
AWS used ElasticSearch; but it was a big headache for Elastic the company to support because it had custom bits which integrated AWS and they used an old version; support issues came in for these versions and it was a pain to manage. (There was also a money issue here).
As a result Elastic the company licensed the new versions of the elastic code with a licence AWS wouldn’t be able to use.
So AWS forked the last Apache* version and continued its development in the Apache license. This is the result of that.
Elastic requires contributors to sign a CLA. Their CLA does not reassign copyright but gives Elastic a license to distribute contributed code without restrictions. [1]
Elastic changed the license for Elasticsearch >7.10.2 from Apache to a more restrictive one to prevent cloud providers like AWS from providing a hosted version.
From the most recent news item on the link you posted: “In the coming weeks this website (https://opendistro.github.io/for-elasticsearch/) will be selectively decommissioned and/or forwarded to the corresponding OpenSearch.org content.”
Having used Open Distro for a little while, I really don't have high hopes for this project, although I would like to be proved wrong.
Open Distro was largely unloved - lots of Github issues remaining untriaged, and various small but poor technical decisions. I've seen FOSS projects that don't have massive corporate backing handle their projects better.
Perhaps the license change will force Amazon to put more resources into the project.
Elastic considers ElasticSearch to be its IP and has changed its license from Apache 2.0 to a more restrictive version to prevent AWS from monetizing it. In response, AWS forked ElasticSearch into OpenSearch at the most recent commit that was still on the Apache 2.0 license.
So this is Amazon rip-off (legal one, obviously) of the last Elasticseartch version having Apache license...
On the one hand, this is ok. Free software is taken and extended by someone who wants to do this... Yet, I am not sure if in that particular case this follows the spirit of the free software.
This move will probably kill company behind Elasticsearch sooner or later. Amazon is making money not on selling particular piece of software but on selling hosting of it. So they gladly give OpenSearch "for free".
Elasticsearch company produced pretty great product, they wanted to earn some bucks on additional features (I think alerts were only in paid version) and cloud hosting for rather reasonable price. And here came AWS, took their product to sell it as their own. This does not sound fair.
Long term consequences of this kind approach from big players are going to be bad. We will get less free software since everyone who wants to develop new product of this kind will have a choice:
- start with more restrictive license (like AGPL) and risk that nobody will even look on their new product
- start with "business-friendly" license like Apache, MIT and risk that if their product will become popular and profitable it will be cannibalized by AWS, etc.
I wish Elasticsearch people good luck, hopefully, with the competitive pricing and ability to work faster/leaner on their product they will survive, they definitely deserve that.
Elastic are a $13 billion company. They get little sympathy from me.
They built that value off the back of many other open source products, and with help from the community - all under licenses like Apache 2.0 and MIT.
And now, after building all that value by selling Enterprise and SaaS versions of the software, they decided to specifically target their competitor who was doing the exact same thing.
Without Lucene, there would have been no ElasticSearch. Why does ElasticSearch get to take all the hard work done by Lucene contributors and sell it as part of their own product? This does not sound fair... in the terms you put it.
If this hurts Elastic, they did it to themselves. They don't deserve your support.
Just remember that their valuation does not represent their free cash flow, whether they are profitable at all, etc. It could be that open sourcing something kills your profitability in the back, while boosting revenue.
In this case, it's understandable that they need to improve their profitability, and laudable that they are not doing so by closed sourcing the product.
"The Elastic projects were offered under the Apache license. Outside contributors donated time and energy with the understanding that their work was going towards the greater good, the public software commons. Now, instead, their contributions are embedded in a proprietary product. If they want to enjoy the fruits of their own and their co-contributors’ labor, they have to agree to a proprietary license or fork."
"Elastic’s current business model is inconsistent with what open source licenses are designed to do. Its current business desires are what proprietary licenses (which includes source available) are designed for."
Because they significantly innovated beyond the core lucene library to solve a major pain of their customers around scaling, analytics, etc., providing customers with extra value.
Amazon appear to be only making minor tweaks to the software and commoditising its running, capitalising on their entrenched audience. It's easy to see why Elastic are annoyed.
> Because they significantly innovated beyond the core lucene library to solve a major pain of their customers around scaling, analytics, etc., providing customers with extra value.
And they've decided that this extra value could be given away for $0, or they wouldn't have released this under an open source license.
> Amazon appear to be only making minor tweaks to the software and commoditising its running
They may be "minor tweaks" but apparently they have a lot of value for some people. I don't particularly care for them and run my own ES cluster. Just as enabling TLS and basic auth are "minor tweaks", but this is what Elastic charges for.
But if there are a non-negligible number of companies using the managed ES service, it means that it has at least some value and that it's good enough compared to the Elastic Cloud offering (I personally don't know how they compare, just that I find both expensive).
This value has been offered by AWS before Elastic's offering. Only now that "not enough people" want to buy the managed Elastic Cloud is it an issue.
To me, these are separate products. Sure, there would be no managed AWS ES without Elastic's product. But Elastic's cloud offering is something that came afterwards, and to me only this particular offering (the managed ES) competes with AWS's offering.
Now, it looks like Elastic will get the worst of both worlds. I only use ES for the logs and needed the authentication. I couldn't really justify the platinum tier just to get SSO, so we made do with basic LDAP. But now that Open Search has those features for free? I don't really have a good reason anymore to stick with Elastic's onerous plan. And no, I've never once called them for the "support" part of the offering, so I'm not really losing anything there.
Yeah AWS ES totally has value. I'm aware ES decided to go with an open licence but it doesn't take a rocket scientist to work out that they did that to try to gain traction and cross-sell other services, such as managed hosting.
If you have a friend round to stay for the weekend and say "stay as long as you want", wouldn't you be annoyed if they were still there 10 years later? There is such a thing as the spirit vs the letter of the law.
> it doesn't take a rocket scientist to work out that they did that to try to gain traction and cross-sell other services, such as managed hosting.
I sympathize with that, and would say that they took this chance and lost.
But again, what rubs me the wrong way is that AWS is not just selling a "packaged" version of ES for more money, the same exact thing that Elastic sells. They're selling the service around it. Which they built themselves. My client runs an ES cluster on a bunch of fairly fat EC2 instances, which would cost an arm and a leg if they went through Elastic Cloud. Are we the friend who's still there ten years later?
I probably still wouldn't agree with it, but I guess I would have less of an issue with this line of argument if it weren't just about ES but about every [Thing]Service that AWS sells, in particular RDS.
Hell, why is there no outcry about them selling EC2 instances with Ubuntu? Or even "worse", their "custom" distribution (Amazon Linux). Are they sending a check regularly to Linus and / or to the Linux Foundation (or whoever is in charge of Linux as a "product")?
This, to me, looks a lot like a sore loser situation. ES tried to take on AWS selling a service around a product and lost. I understand it stings even worse that said product is their product. But maybe they should've stuck to building an awesome service around Lucene that they could sell for profit. Wait... this sounds familiar, doesn't it?
I'm not saying this is a great situation for Elastic to be in. But I don't like the way they're trying to dance around it and not assume their errors. They wanted to build a second product (Elastic Cloud) around their first (Elasticsearch) and it failed to attract the expected success. Instead of asking themselves why that is, they decided to change the rules of the game. Now, they're of course allowed to do this, Elasticsearch being their product and all. But they can't expect for everybody to be on board with it. Maybe their product sucked compared to AWS' or simply wasn't good enough to warrant the hassle of going through a second supplier or whatever.
I think you nailed it in the last sentence. AWS will one day be able to extract near monopoly profits due to costs of migration.
I expect the ES didn't realise how popular it'd end up being so went open source, and later realised they'd shot themselves in the foot but knew they'd rub people up the wrong way if they went closed source so they kicked the can down the road... And here we are
Solr also addressed a lot of the same problems as Elastic, maybe not all of them. It was harder to configure but the documentation eventually got to be decent. That was some years ago. I don't know the state of things now.
My two cents.... trying to profit via the business of artificial scarcity on an open source project that's going to end up a commoditization anyway, is kinda crappy. If they offered support or implementations or a specific customization for profit, I get it. That way the people doing work get rewarded and people don't end up paying for access to a feature because it's a carrot being used to get money.
> They built that value off the back of many other open source products
Would you opinion change if everything in Elasticsearch was in-house?
I think there is a broader question about spirit of open source licensing here. If I'm building a new DB from scratch, should I choose a true open source license and risk giving up competitive advantage in my ability to monetize it, over Amazon-like companies who have the benefit of brand and scale? Or is it okay to just make the code source-available, which is still better than closed-source?
I like the balanced approach some companies are adopting: make the code source-available initially and automatically transition to a open source license after N years.
1. The "support pack", né x-pack, that adds features and support, is billed per node and for a minimum of 5 nodes, adding such "optional" features as centralized authentication, and, until recently, TLS (!) and basic auth (which have been integrated in the basic product a while ago).
2. The hosting part, where Elastic-the-company hosts the service for you. This does not include the "support pack" from 1.
Now, as I understand it, Elastic's issue with AWS is the hosting part. I've never used the managed AWS ES, but if I understand it correctly, if you wanted to use any of the "support" features, you would have had to purchase the pack from Elastic.
> Elasticsearch company produced pretty great product, they wanted to earn some bucks on additional features (I think alerts were only in paid version) and cloud hosting for rather reasonable price. And here came AWS, took their product to sell it as their own. This does not sound fair.
This doesn't sound right to me. AWS never sold "Elasticsearch, the product, as their own". They've sold a hosted version of it, the same way they sell a hosted version of Postgres, MySQL, Oracle, etc (RDS — Relational Database Service). The advanced features you talked about were separate.
In this, yes, they compete directly with one of Elastic's offerings, namely "hosted Elasticsearch".
I would understand the outcry against "AWS selling Elastic's product as their own" if there had been a similar outcry for PostgreSQL, Ubuntu, or any of the myriad other services they offer.
The thing is, Elastic put out a product, Elasticsearch, as open source. Now they've put out a second product, "Elastic Cloud", based on the first, and they don't like that other companies are able to use the first product in order to compete with them on the second.
It's like if Ford introduced a "managed car" service, with a driver, and suddenly started complaining that Uber drivers should pay extra for the same Ford car. This is also kind of like the whole "licensed repair shops" thing.
Yes, a Ford car costs money to buy, but it's the same price for everyone, be it a grandma looking to cart around her grandkids once in a while, or a professional driver offering to drive people around for money every day. The price has nothing to do with it, since AWS honored the Elasticsearch license.
Elastic Cloud was always behind what AWS already offerred. Not sure how it is like today, but it was one of blockers for us.
And what AWS offers today is not a huge bar, mainly too many reliability problems, at the 1.5x of bare EC2 cost, a stupid load balancer in front of cluster that abstracts the nodes so you cannot connect to the healthy one yourself and probably load balancer is too late to mark it unhealthy, many hours of blue-green deployment that increases latency considerably, overly complex Cognito for auth.. list continues. Still after all these years, like most half-managed offerings in AWS, ES offering sucks IMO.
Elastic happily used the free software Lucene, community contribution when it was convenient for more than 8 years, but when they surely lagged behind what AWS offers, they switched to this licence. ES cloud was not reasonable price either.
Why it's not fair? The license explicitly allows that. From another point of view, years of contribution is now behind another non-free license and outside contributors would not want to contribute to it.
Hating on AWS sounds cool in HN, but there are many projects starting AGPL now, however Amazon ES has been launched in 2015, there was plenty of time to make the license switch but they did not. They tried licensing parts of it, which made sense (I guess after the IPO, not sure.) but when AWS just implemented those on their own they went this license route and alienated everyone.
Finally, there is not just AWS ES, there were always another ones to offer managed ELK, i.o Logz.io and others who are also supportive of this fork now. Why there were so many providers? Because ES did not provide, and when they acquired another company to provide it, it was not on-par with others, so the situation today.
I must not have been the only one confused how exactly it counts as an Amazon rip-off rather than a community maintained version of the open source part of Elasticsearch. And frankly I'd still be interested if anyone could clarify how exactly this code can be considered Amazon's.
I did however notice that Amazon claims copyright on the github repo, which is interesting I suppose.
They dropped support for the pop-ups for installing OpenSearch plugins but still support the link relations method. The OpenSearch plugin repository at the Mycroft Project looks like it is still maintained: https://mycroftproject.com/
It’s still one of the most useful things in my opinion. And I think it’s also still working. To be able to have a custom short cut in Chrome’s bar and use it to seamlessly go to any website’s search results page is such a great feature.
At the end of the day, being truly committed to open source is difficult once you start feeling the revenue pressure. The very openness of the license that drives the growth of the project and the community in the early days becomes an albatross around your neck. Once you have raised a lot of money, there is tremendous pressure sooner or later to own the market and to justify the insane revenue multiples at which many companies are raising. It's a delicate path.
If you set the expectations right, I think the Cloud market is large enough to accommodate several companies in most verticals. However, "growth at all costs" requires walking back on your ideals and promises.
IMHO - This is on Elastic itself. AWS is not the only place or mandatory place to run Elastic search. There are tons of other cloud vendors, on premise farms and what not. That was more than enough for Elastic to be profitable in polishing their offerings for everything minus may be the 3 big cloud players.
This also goes for Mongo. The pricing at least was not clear to me for a deployment, I clearly and precisely articulated the amount of data and already 3 node MongoDB Cluster we have but the sales rep continued to insist on jumping on a call.
Here's the thing - what is so difficult about pricing that you cannot write a one liner saying XXX dollars per YY nodes per ZZ period?
I vaguely remember, it turned about to be around about 2.5k USD per month whereas infrastructure cost was to be yet paid by us.
I understand the reasoning behind the license change for elastic search / kibana and the rest of the server side products. But why did they have to change the license for Eui [0]. It is a real shame because it is amazing for building admin UIs.
58 comments
[ 1.1 ms ] story [ 131 ms ] threadfrom the main page. So it’s essentially ES before they switched to a new license.
As a long term Elasticsearch user, I don't like their move away from OSS. I think its long term bad for them and it puts me on the spot when I have to consult my customers on what to use. And yes, I've already had a few conversations on this topic. People seem to prefer OSS and this whole thing makes them default to opensearch/aws.
It's hard arguing with that. Whatever Elastic says, it's a closed source license now and it largely cuts them off from external contributions. At least, I don't tend to make closed source contributions for free. Which is a shame.
Whether Amazon can turn this project in what it claims to be already: a thriving oss community, remains to be seen. As of yet it's basically a largely unmodified fork with not a whole lot of feature work happening (if any at all). Turning that around is going to require some skilled external contributors to join. At least, I don't see the current team achieving a whole lot as it appears to be rather small and most of their work so far has been tinkering with the build system and editing some licensing texts.
The good news is that 7.10.2 was a fine version. The bad news is that this fork is not getting the fixes, improvements, and performance work that mainline Elasticsearch is getting. Also, Elasticsearch 8.x at some point will become a thing and is likely to have some improvements. It's actually been unusually long since the last major release. Before 7.0 they had major releases every year. Of course, moving to new versions of Lucene will be a some non trivial work in terms of implementing and testing the changes. At some point the forks will diverge or opensearch will simply fall behind. The latter seems to be happening. Elasticsearch. 7.11-7.13 have added a few new features that opensearch won't have.
They forked the code but not the developer community. Not yet at least. That could just be a matter of time. IMHO Elastic should kill this fork by rolling back their license changes. There's nothing to backport from this that is worth keeping as far as I can see. So, not a big change technically. Not too late to admit they are simply wrong. They seem to have dug in on this so probably not going to happen. But one can hope.
Somehow I have doubts about this part.
In all seriousness, is Amazon good about accepting PRs and stuff from volunteers for their oss projects (i.e. Firecracker)?
is a bit of a strange way of saying "selling ElasticSearch as a service".
Only to be expected, with that nick.
The owner of the code can of course change the license. But in typical open source projects there are 100s of contributors, so changing the license can be anything between a major job and impossible.
As a result Elastic the company licensed the new versions of the elastic code with a licence AWS wouldn’t be able to use.
So AWS forked the last Apache* version and continued its development in the Apache license. This is the result of that.
Elastic changed the license for Elasticsearch >7.10.2 from Apache to a more restrictive one to prevent cloud providers like AWS from providing a hosted version.
[1] https://github.com/elastic/elasticsearch/blob/master/CONTRIB...
https://opendistro.github.io/for-elasticsearch/
Open Distro was largely unloved - lots of Github issues remaining untriaged, and various small but poor technical decisions. I've seen FOSS projects that don't have massive corporate backing handle their projects better.
Perhaps the license change will force Amazon to put more resources into the project.
OpenSearch: AWS fork of Elasticsearch and Kibana - https://news.ycombinator.com/item?id=26780848 - April 2021 (424 comments)
On the one hand, this is ok. Free software is taken and extended by someone who wants to do this... Yet, I am not sure if in that particular case this follows the spirit of the free software.
This move will probably kill company behind Elasticsearch sooner or later. Amazon is making money not on selling particular piece of software but on selling hosting of it. So they gladly give OpenSearch "for free".
Elasticsearch company produced pretty great product, they wanted to earn some bucks on additional features (I think alerts were only in paid version) and cloud hosting for rather reasonable price. And here came AWS, took their product to sell it as their own. This does not sound fair.
Long term consequences of this kind approach from big players are going to be bad. We will get less free software since everyone who wants to develop new product of this kind will have a choice:
- start with more restrictive license (like AGPL) and risk that nobody will even look on their new product - start with "business-friendly" license like Apache, MIT and risk that if their product will become popular and profitable it will be cannibalized by AWS, etc.
I wish Elasticsearch people good luck, hopefully, with the competitive pricing and ability to work faster/leaner on their product they will survive, they definitely deserve that.
They built that value off the back of many other open source products, and with help from the community - all under licenses like Apache 2.0 and MIT.
And now, after building all that value by selling Enterprise and SaaS versions of the software, they decided to specifically target their competitor who was doing the exact same thing.
Without Lucene, there would have been no ElasticSearch. Why does ElasticSearch get to take all the hard work done by Lucene contributors and sell it as part of their own product? This does not sound fair... in the terms you put it.
If this hurts Elastic, they did it to themselves. They don't deserve your support.
FY2020 revenue was 430 million, and they spent 220 million on sales and marketing and 165 million on R&D, the total being net loss of 167 million.
This is what OSI had to say about it: https://opensource.org/node/1099
"The Elastic projects were offered under the Apache license. Outside contributors donated time and energy with the understanding that their work was going towards the greater good, the public software commons. Now, instead, their contributions are embedded in a proprietary product. If they want to enjoy the fruits of their own and their co-contributors’ labor, they have to agree to a proprietary license or fork."
"Elastic’s current business model is inconsistent with what open source licenses are designed to do. Its current business desires are what proprietary licenses (which includes source available) are designed for."
Amazon appear to be only making minor tweaks to the software and commoditising its running, capitalising on their entrenched audience. It's easy to see why Elastic are annoyed.
And they've decided that this extra value could be given away for $0, or they wouldn't have released this under an open source license.
> Amazon appear to be only making minor tweaks to the software and commoditising its running
They may be "minor tweaks" but apparently they have a lot of value for some people. I don't particularly care for them and run my own ES cluster. Just as enabling TLS and basic auth are "minor tweaks", but this is what Elastic charges for.
But if there are a non-negligible number of companies using the managed ES service, it means that it has at least some value and that it's good enough compared to the Elastic Cloud offering (I personally don't know how they compare, just that I find both expensive).
This value has been offered by AWS before Elastic's offering. Only now that "not enough people" want to buy the managed Elastic Cloud is it an issue.
To me, these are separate products. Sure, there would be no managed AWS ES without Elastic's product. But Elastic's cloud offering is something that came afterwards, and to me only this particular offering (the managed ES) competes with AWS's offering.
Now, it looks like Elastic will get the worst of both worlds. I only use ES for the logs and needed the authentication. I couldn't really justify the platinum tier just to get SSO, so we made do with basic LDAP. But now that Open Search has those features for free? I don't really have a good reason anymore to stick with Elastic's onerous plan. And no, I've never once called them for the "support" part of the offering, so I'm not really losing anything there.
If you have a friend round to stay for the weekend and say "stay as long as you want", wouldn't you be annoyed if they were still there 10 years later? There is such a thing as the spirit vs the letter of the law.
I sympathize with that, and would say that they took this chance and lost.
But again, what rubs me the wrong way is that AWS is not just selling a "packaged" version of ES for more money, the same exact thing that Elastic sells. They're selling the service around it. Which they built themselves. My client runs an ES cluster on a bunch of fairly fat EC2 instances, which would cost an arm and a leg if they went through Elastic Cloud. Are we the friend who's still there ten years later?
I probably still wouldn't agree with it, but I guess I would have less of an issue with this line of argument if it weren't just about ES but about every [Thing]Service that AWS sells, in particular RDS.
Hell, why is there no outcry about them selling EC2 instances with Ubuntu? Or even "worse", their "custom" distribution (Amazon Linux). Are they sending a check regularly to Linus and / or to the Linux Foundation (or whoever is in charge of Linux as a "product")?
This, to me, looks a lot like a sore loser situation. ES tried to take on AWS selling a service around a product and lost. I understand it stings even worse that said product is their product. But maybe they should've stuck to building an awesome service around Lucene that they could sell for profit. Wait... this sounds familiar, doesn't it?
I'm not saying this is a great situation for Elastic to be in. But I don't like the way they're trying to dance around it and not assume their errors. They wanted to build a second product (Elastic Cloud) around their first (Elasticsearch) and it failed to attract the expected success. Instead of asking themselves why that is, they decided to change the rules of the game. Now, they're of course allowed to do this, Elasticsearch being their product and all. But they can't expect for everybody to be on board with it. Maybe their product sucked compared to AWS' or simply wasn't good enough to warrant the hassle of going through a second supplier or whatever.
I expect the ES didn't realise how popular it'd end up being so went open source, and later realised they'd shot themselves in the foot but knew they'd rub people up the wrong way if they went closed source so they kicked the can down the road... And here we are
Would you opinion change if everything in Elasticsearch was in-house?
I think there is a broader question about spirit of open source licensing here. If I'm building a new DB from scratch, should I choose a true open source license and risk giving up competitive advantage in my ability to monetize it, over Amazon-like companies who have the benefit of brand and scale? Or is it okay to just make the code source-available, which is still better than closed-source?
I like the balanced approach some companies are adopting: make the code source-available initially and automatically transition to a open source license after N years.
There are two non-free parts to ES:
1. The "support pack", né x-pack, that adds features and support, is billed per node and for a minimum of 5 nodes, adding such "optional" features as centralized authentication, and, until recently, TLS (!) and basic auth (which have been integrated in the basic product a while ago).
2. The hosting part, where Elastic-the-company hosts the service for you. This does not include the "support pack" from 1.
Source: https://www.elastic.co/subscriptions/cloud
---
Now, as I understand it, Elastic's issue with AWS is the hosting part. I've never used the managed AWS ES, but if I understand it correctly, if you wanted to use any of the "support" features, you would have had to purchase the pack from Elastic.
> Elasticsearch company produced pretty great product, they wanted to earn some bucks on additional features (I think alerts were only in paid version) and cloud hosting for rather reasonable price. And here came AWS, took their product to sell it as their own. This does not sound fair.
This doesn't sound right to me. AWS never sold "Elasticsearch, the product, as their own". They've sold a hosted version of it, the same way they sell a hosted version of Postgres, MySQL, Oracle, etc (RDS — Relational Database Service). The advanced features you talked about were separate.
In this, yes, they compete directly with one of Elastic's offerings, namely "hosted Elasticsearch".
I would understand the outcry against "AWS selling Elastic's product as their own" if there had been a similar outcry for PostgreSQL, Ubuntu, or any of the myriad other services they offer.
The thing is, Elastic put out a product, Elasticsearch, as open source. Now they've put out a second product, "Elastic Cloud", based on the first, and they don't like that other companies are able to use the first product in order to compete with them on the second.
It's like if Ford introduced a "managed car" service, with a driver, and suddenly started complaining that Uber drivers should pay extra for the same Ford car. This is also kind of like the whole "licensed repair shops" thing.
Yes, a Ford car costs money to buy, but it's the same price for everyone, be it a grandma looking to cart around her grandkids once in a while, or a professional driver offering to drive people around for money every day. The price has nothing to do with it, since AWS honored the Elasticsearch license.
And what AWS offers today is not a huge bar, mainly too many reliability problems, at the 1.5x of bare EC2 cost, a stupid load balancer in front of cluster that abstracts the nodes so you cannot connect to the healthy one yourself and probably load balancer is too late to mark it unhealthy, many hours of blue-green deployment that increases latency considerably, overly complex Cognito for auth.. list continues. Still after all these years, like most half-managed offerings in AWS, ES offering sucks IMO.
Elastic happily used the free software Lucene, community contribution when it was convenient for more than 8 years, but when they surely lagged behind what AWS offers, they switched to this licence. ES cloud was not reasonable price either.
Why it's not fair? The license explicitly allows that. From another point of view, years of contribution is now behind another non-free license and outside contributors would not want to contribute to it.
Hating on AWS sounds cool in HN, but there are many projects starting AGPL now, however Amazon ES has been launched in 2015, there was plenty of time to make the license switch but they did not. They tried licensing parts of it, which made sense (I guess after the IPO, not sure.) but when AWS just implemented those on their own they went this license route and alienated everyone.
Finally, there is not just AWS ES, there were always another ones to offer managed ELK, i.o Logz.io and others who are also supportive of this fork now. Why there were so many providers? Because ES did not provide, and when they acquired another company to provide it, it was not on-par with others, so the situation today.
I did however notice that Amazon claims copyright on the github repo, which is interesting I suppose.
Eh, the spirit of free software is that the user has several important freedoms. This is true with OpenSearch, but no longer true with ElasticSearch.
The fact that a large percentage of the code was original developed by Elasticsearch B.V is entirely irrelevant to the “spirit of free software”.
https://en.wikipedia.org/wiki/OpenSearch
Alas, that thing called OpenSearch is just about dead now.
They dropped support for the pop-ups for installing OpenSearch plugins but still support the link relations method. The OpenSearch plugin repository at the Mycroft Project looks like it is still maintained: https://mycroftproject.com/
At the end of the day, being truly committed to open source is difficult once you start feeling the revenue pressure. The very openness of the license that drives the growth of the project and the community in the early days becomes an albatross around your neck. Once you have raised a lot of money, there is tremendous pressure sooner or later to own the market and to justify the insane revenue multiples at which many companies are raising. It's a delicate path.
If you set the expectations right, I think the Cloud market is large enough to accommodate several companies in most verticals. However, "growth at all costs" requires walking back on your ideals and promises.
This also goes for Mongo. The pricing at least was not clear to me for a deployment, I clearly and precisely articulated the amount of data and already 3 node MongoDB Cluster we have but the sales rep continued to insist on jumping on a call.
Here's the thing - what is so difficult about pricing that you cannot write a one liner saying XXX dollars per YY nodes per ZZ period?
I vaguely remember, it turned about to be around about 2.5k USD per month whereas infrastructure cost was to be yet paid by us.
i always suspect the pricing is dependent on your budget and they want to figure out the amount of your budget before giving a quote...
[0] https://github.com/elastic/eui/