I would argue that making Kafka more approachable and easy to deploy and use out of box is creating value for the developers, because it helps increase potential user adoption (and therefore increases the potential open source contributor base)
I agree. We started using Redis with AWS' managed offering (Elasticache), then migrated to running it on our own once we were comfortable with it. I can imagine the same path for Kafka use.
Then you couldn't run it on Linux, as you'd be unable to relicense the storage system that is the kernel file system or storage drivers from GPL to SSPL.
> Broker instances start at $0.21/hr and broker storage is $0.10 per GB-month
Kafka requires Zookeeper, do you pay for zookeeper instances too? Anyway, this pricing seems prohibitively costly unless you really need hands-off approach to your infra. Kafka is pretty low-maintenance and stable in my experience.
"You do not pay for Apache Zookeeper nodes that Amazon MSK provisions for you, or data transfer that occurs between brokers and nodes within clusters."
Just to note, the $.21/hr broker is on an m5.large (2 cpu, 8 GB mem), which goes for $.096/hr.
We run three nodes right now on m5.xlarge instances. At $.42/hr for the managed kafka, compared to $.192/hr self hosted kafka, I think we'll keep it self hosted for now...
It does. And until last year even the major support company's containers (for Kafka etc) were not very good, we didn't upgrade a k8s cluster purely because of the state of their stack. This stuff scared me and this is a shocking affair these days. In some ways it's a step back in reliability, and i'd have taken this AWS option in a heartbeat, price-gouging aside.
sorry that I answer here, this is totally unrelated to the topic being discussed. I've just seen that you replied me on an old HN post of mine and cannot reply there nor can I find another way to contact you on your profile.
It's regarding the habit tracker https://everyday.app I'm working on. You wrote "Your web-app is the best habit tracker ever created and I used it for a while after I paid for it very early on. :) However, I left it because mobile app is not as smooth of an experience. Currently using habitify.me across my devices but will switch back once mobile app catches up with other apps." and I'd totally love to hear in more detail what was lacking or what do you think I should get fixed to help the app catch up. Feel free to answer here or send me an email at joan@everyday.app.
Awesome stuff. Kafka has been out of reach for our small org because we simply don't have the resources to manage a prod Kafka+zookeeper cluster. Kinesis has never anywhere near feature parity with Kafka. Ktables on AWS here we come!
19 comments
[ 0.18 ms ] story [ 46.0 ms ] thread[0] - https://www.mongodb.com/licensing/server-side-public-license
Many vendors make money by selling services on top, and there are at least a dozen that offer managed Kafka right now.
Kafka requires Zookeeper, do you pay for zookeeper instances too? Anyway, this pricing seems prohibitively costly unless you really need hands-off approach to your infra. Kafka is pretty low-maintenance and stable in my experience.
"You do not pay for Apache Zookeeper nodes that Amazon MSK provisions for you, or data transfer that occurs between brokers and nodes within clusters."
We run three nodes right now on m5.xlarge instances. At $.42/hr for the managed kafka, compared to $.192/hr self hosted kafka, I think we'll keep it self hosted for now...
sorry that I answer here, this is totally unrelated to the topic being discussed. I've just seen that you replied me on an old HN post of mine and cannot reply there nor can I find another way to contact you on your profile.
It's regarding the habit tracker https://everyday.app I'm working on. You wrote "Your web-app is the best habit tracker ever created and I used it for a while after I paid for it very early on. :) However, I left it because mobile app is not as smooth of an experience. Currently using habitify.me across my devices but will switch back once mobile app catches up with other apps." and I'd totally love to hear in more detail what was lacking or what do you think I should get fixed to help the app catch up. Feel free to answer here or send me an email at joan@everyday.app.
Sorry for reaching out this way and thank you! :)
Right now I'm trying to debug this issue: https://community.hortonworks.com/questions/208764/kafka-err...
I'm not a fan of Kafka though but you sometimes have to managed stuf fyou don't like anyway.
So this is great news. AWS truly listen to their customers I think.