The data streaming world is saturated. There is a real cost associated with choosing between Spark, Flink, Akka, Heron/Storm, IBM Streams, Edgent, or even NodeRed — to the extent that its impossible to evaluate everything. Usually, developers start from the most commonly used framework and go down the list to evaluate how it plays.
A streaming framework can be kickass but see no real adoption because of this. Hopefully that isn’t what’s happening here, or why someone is posting to HN to ask “who’s using us and why?”
I work for an IT consultancy firm, and we have some customers asking for it right now (mostly in the banking sector).
The reason is Horton works deprecated Apache Flume and bundled Nifi instead.
Personally I don't like it. I prefer other ETL tools like Talend. I also find confusing the user interface.
The good part is that NIFi processed are buffered, and you can keep metadata (like data provenance) with the records which is very helpful for compliance.
Our company was desperate for an ETL solution like this, and we spent a good deal of time with NiFi - partly because Hortonworks was pushing it.
We found the UI crummy, the execution engine sloppy and overall, it came across as half-baked...
...we lost a lot of time trying to make it work, but gave up. So many other ETL tools out there, no need to put up with a half-baked platform.
Hortonworks probably picked Nifi up to differentiate themselves from Cloudera. I wonder if given the merge with Cloudera, they wont just scrap this whole NiFi thing.
With so many ETL tools out there, not enough mindshare to keep NiFi going...I do not recommend creating a dependancy in your stack on it.
4 comments
[ 3.1 ms ] story [ 21.6 ms ] threadA streaming framework can be kickass but see no real adoption because of this. Hopefully that isn’t what’s happening here, or why someone is posting to HN to ask “who’s using us and why?”
We view it as a middleware layer and integration engine. Tools like composable save enormous amounts of development time, and simplifies maintenance.
The reason is Horton works deprecated Apache Flume and bundled Nifi instead.
Personally I don't like it. I prefer other ETL tools like Talend. I also find confusing the user interface.
The good part is that NIFi processed are buffered, and you can keep metadata (like data provenance) with the records which is very helpful for compliance.
We found the UI crummy, the execution engine sloppy and overall, it came across as half-baked... ...we lost a lot of time trying to make it work, but gave up. So many other ETL tools out there, no need to put up with a half-baked platform.
Hortonworks probably picked Nifi up to differentiate themselves from Cloudera. I wonder if given the merge with Cloudera, they wont just scrap this whole NiFi thing.
With so many ETL tools out there, not enough mindshare to keep NiFi going...I do not recommend creating a dependancy in your stack on it.