Great news for them. I built a dashboard a few months ago using graphs which pull data from InfluxDB and I had a lot of fun with it. To me, it seem a whole lot easier to use than Graphite.
One of the founders here. Thanks to the community for all the support and encouragement. We couldn't have gotten this far without people getting excited about our software. 2015 is going to be a great year for open source time series databases!
How does your offering compare to KX ? http://kx.com/ . Do you see as a direct competitor or rather a completely different product? Do you see yourself competing in the financial sector soon?
KX seems to be much more focused on the financial sector use case. It's also closed source and has been around for a long time.
We're open source and want to make sure we hit use cases in DevOps, metrics, and sensor data. The financial market data use case is a nice to have for what we're building, but it's certainly not what we're optimizing for.
If we compete in the financial sector with them, it won't be for a while.
Disclaimers: I was CTO@Kx for a while, but also like InfluxDB :-)
It is definitely more finance oriented, though Kx are making moves towards other application areas. Another difference I'd highlight is that Kx concentrate on the core database itself, esp. performance and expressiveness of the query language, and leave things like GUIs and admin add-on tools to partners (like first derivatives and aquaq).
kdb does just fine with metrics and sensor data. Personally, I would argue that it's weaker on string handling though, which can hurt in certain use cases.
I doubt it'll go open source any time soon. However, it being around a long time is something salescritters can spin to wonderful effect re stability, support, etc etc. ;-)
I think there are fine application areas in finance that you should consider -- just consider the many areas where the core problem isn't related to juggling TB of market data ticks coming off the exchanges.
Thanks! That last use case you talk about is the one I'm definitely thinking is what we won't be pursuing. Lots of money there, but hard to come after an entrenched competitor that is heavily optimized for it ;)
If KX is making moves into sensors are they competing more with Informix? Or maybe Vertica?
It's been a while since I've been an insider, and these comments are purely from an outside perspective, interacting with such users.
Vertica is seeing use for more historical stuff, and where the time series queries are pretty simple. Informix time series is doing ok, and has better support for rich queries, but isn't really playing realtime. MemSQL has the realtime perf (hi guys!) but needs to beef up on expressiveness. SAP HANA could do it, but not seeing major uptake there.
Still seeing lots of ad hoc solutions, and the expected experimentation with the usual hadoop menagerie (spark is helping make that practical).
The sensor stuff gets interesting at scale. Individual sources may not be producing data that quickly, but in aggregate it can be entertaining volume. Esp. when it comes to mobile things, and correlations become interesting to look at.
Deep thoughts need to wait for the coffee to kick in.
I suspect we'll see a lot of reinvention of technology to cope with these problems; perhaps even open source..
I read this with great interest - I'm the founder of ApiAxle, an API proxy that actually has a timeseries system built in for the collection of various statistics. I've always had a little seed in the back of my mind about building a stand-alone TS system.
Looking forward to seeing how this all plays out. Best of luck!
I liked Influx so much in my current project at work, I couldn't help it and even used it for a little html5 game to track some metrics of the game and server.
http://www.mashed-games.com/stats.html
Already been done in 2001-2004: Sensage (http://www.sensage.com/, http://en.wikipedia.org/wiki/Sensage), followed by Vertica, etc. All the columnar-storage big-table timestamped data collectors and data aggregators. Sensage failed at the market. Splunk succeeded, perhaps with a different underlying data storage model, but certainly with great marketing.
I'm not familiar with Sensage and I know of Vertica. Open source is a key difference. Developers want to build their products and careers on open tools and we want to help them do it when it comes to time series data.
There will be more as we go along. Our project is a year old and had 2-3 people working on it for most of the time.
InfluxDB is one of our favorite database (we host and run open source DBs ...). Most people can get more from their data by recognizing time series problems, and it's nice to have an easily accessible DB to put that sort of information into.
In short no. In long, there have been attempts mostly failed ones. The current industry standard is making a big NAS and putting paths to log files in an SQL database, and make a nice java front end to chop up logs by time stamp.
The idea is from a testing perspective (I do data acquisition and analysis for test systems), is when a part breaks, or something critical in your test happens.
You want to find the source of failure. Often times analyzing every single data point is pointless, especially after a 10 hour test with 100 points per second per channel. And you only want to see ~2 minutes of data.
How most systems work is you need to open up a measurement file, and either use a stand alone tool to chop this (and pray it doesn't corrupt your measurement file). Or from what I understand Influx simply lets you query time code to time code and return channels via pattern matching within that time span.
Very simple.
Data management is another big issue. A single day's worth of road test can be ~5-10TB per vehicle (2 months of testing + 20 vehicle fleet). Also this total is project to like all things double every year -_-'
Also as far as I understand Influx doesn't support sound or video. Which is really what aerospace and automotive are looking for.
Been flowing InfluxDB for sometime now and used it to do some analytics work few times. Loved it and will use again every chance i get. Also plus points for being written in clean idiomatic Go.
Oh that's great news! I've been using Influx for about a year know and I'm really happy with it already. Hopefully the upcoming version 0.9 with BoltDB will even better than the current storage engines.
I'm also contributing to one of the drivers and most of the time it has been fun using it.
@Paul, are you still going to release a preview of 0.9 in december or is it delayed to celebrate the news? ;)
We're trying to get the 0.9.0 previews out as quickly as possible. We're trying to get something out for testing by the end of the year, but there's still a lot to do.
In software development there are lies, damn lies, and timeline estimates ;)
31 comments
[ 2.8 ms ] story [ 65.0 ms ] threadWe're open source and want to make sure we hit use cases in DevOps, metrics, and sensor data. The financial market data use case is a nice to have for what we're building, but it's certainly not what we're optimizing for.
If we compete in the financial sector with them, it won't be for a while.
It is definitely more finance oriented, though Kx are making moves towards other application areas. Another difference I'd highlight is that Kx concentrate on the core database itself, esp. performance and expressiveness of the query language, and leave things like GUIs and admin add-on tools to partners (like first derivatives and aquaq).
kdb does just fine with metrics and sensor data. Personally, I would argue that it's weaker on string handling though, which can hurt in certain use cases.
I doubt it'll go open source any time soon. However, it being around a long time is something salescritters can spin to wonderful effect re stability, support, etc etc. ;-)
I think there are fine application areas in finance that you should consider -- just consider the many areas where the core problem isn't related to juggling TB of market data ticks coming off the exchanges.
If KX is making moves into sensors are they competing more with Informix? Or maybe Vertica?
Vertica is seeing use for more historical stuff, and where the time series queries are pretty simple. Informix time series is doing ok, and has better support for rich queries, but isn't really playing realtime. MemSQL has the realtime perf (hi guys!) but needs to beef up on expressiveness. SAP HANA could do it, but not seeing major uptake there.
Still seeing lots of ad hoc solutions, and the expected experimentation with the usual hadoop menagerie (spark is helping make that practical).
The sensor stuff gets interesting at scale. Individual sources may not be producing data that quickly, but in aggregate it can be entertaining volume. Esp. when it comes to mobile things, and correlations become interesting to look at.
Deep thoughts need to wait for the coffee to kick in.
I suspect we'll see a lot of reinvention of technology to cope with these problems; perhaps even open source..
Looking forward to seeing how this all plays out. Best of luck!
What's different about InfluxDB?
There will be more as we go along. Our project is a year old and had 2-3 people working on it for most of the time.
The idea is from a testing perspective (I do data acquisition and analysis for test systems), is when a part breaks, or something critical in your test happens.
You want to find the source of failure. Often times analyzing every single data point is pointless, especially after a 10 hour test with 100 points per second per channel. And you only want to see ~2 minutes of data.
How most systems work is you need to open up a measurement file, and either use a stand alone tool to chop this (and pray it doesn't corrupt your measurement file). Or from what I understand Influx simply lets you query time code to time code and return channels via pattern matching within that time span.
Very simple.
Data management is another big issue. A single day's worth of road test can be ~5-10TB per vehicle (2 months of testing + 20 vehicle fleet). Also this total is project to like all things double every year -_-'
Also as far as I understand Influx doesn't support sound or video. Which is really what aerospace and automotive are looking for.
Custom processing against raw byte values (and other custom functions) should come sometime next year.
@Paul, are you still going to release a preview of 0.9 in december or is it delayed to celebrate the news? ;)
In software development there are lies, damn lies, and timeline estimates ;)
Now....any chance of open sourcing the retired errplane codebase?