Ask HN: Splunk Alternatives?

Elastic has been getting a lot of business from priced out Splunk users.

Elasticsearch (well, the whole ELK stack)

https://www.elastic.co/

Depending on your volume and situation, there are hosted options or you can roll your own on-prem.

I work at Datadog and we've been pretty successful at attracting customers over from Splunk depending upon the use case. Others in industry are SumoLogic, ELK etc. You can check us out here if you want https://www.datadoghq.com/

Scalyr is great, but I'm not sure if it's any cheaper. https://www.scalyr.com/

At my company, we used to pay 7 figures to splunk, until we realized how much data do we actually need. We cut down a lot of unnecessary logging and sent only the things that were needed.

Most of the data that was needed by business folks was sent to segment in terms of events and for genuine errors were sent to sentry with stack traces and converted most of the logs to metrics.

This helped us cut down a lot of $$ for logging and monitoring. Elastic search is a great alternative, but it takes an entire team to maintain it. I'm not sure, if its worth the time and effort.

AlienVault, or its FOSS version, OSSIM. [1]

[1] https://www.alienvault.com/products/ossim

I work at Google Cloud, and this is a common request. If you're on GCP, you can build your own using logging exports and BigQuery. Otherwise, I typically recommend Elastic.

It'll really depend on your use-case. If you're not doing _too much_ volume (less than 1-2TB) and you're willing to put in the work, ELK stack (as others mentioned) will be more cost-effective from a software cost perspective. You'll end up making up those costs in person-hours though, as you'll lose the benefits of managed/SaaS.

If you'd rather not spend the time managing an ELK stack, there's a lot of logging options (disclaimer: I work at one of them, https://logdna.com). It'd be helpful to unpack what you mean by "credible" competitors however. Our product, and most others in our space frankly, can't match Splunk in terms of feature-set today, so knowing what is most important to you would be helpful. For example, if you're looking for basic log storage and search, you'll have tons of options. If you need compliance, that would narrow the field a bit.

I'm happy to chat if you're interested in giving us (LogDNA) a shot or if you have other general questions, feel free to shoot me an email, I'm just peter@[ourcompanydomain]. I'm not in sales so I don't really have any incentive to push you towards us unless it makes sense, and a lot of our competitors have great products as well so happy to try and point you in the right direction.

I’d have to mention humio.com They’re mainly just for logs, but I really love their query style.

I had to go through this last year with my previous company. Splunk's licensing is outrageous. They cap the data transfer on a PAID license arbitrarily. Even getting coordinated with their sales to just pay them was a chore. They do nothing to foster open development on their platform. IMO, the open offerings actually seem designed to make it impossible to improve/contribute/extend in a cost effective way. It is also ill equipped to deal with Kubernetes even though they did release a reasonable if Windows Guy bloated example of a k8s deployment at the beginning of this year. I was completely baffled when I found out it was a Django project. It feels like something from the depths of enterprise Java.

Elasticsearch (or ELK/EFK) is really eating their lunch but if you have any experience managing an Elasticsearch cluster at scale you might have some reservations about it. If you'd like to audit it I might suggest using a Helm chart like this one[0] to deploy a full stack to a Kubernetes cluster. Even if you aren't using k8s for production it's a fast way to get a handle on it. You can deploy a test cluster easy on GKE quick for nearly no money. If that seems too inaccessible, there are a few good docker-compose[1] implementations too that can get you going right away.

I don't think it's quite ready for production, but we were already using Prometheus and Grafana so I was auditing Loki[2] and was pleasantly surprised. Though at the time I determined it would not be a substitute at scale for Splunk or ELK but could be viable for many people and the project is moving fast to do cool stuff.

[0] https://github.com/helm/charts/tree/master/stable/elastic-st...

[1] https://github.com/deviantony/docker-elk

[2] https://grafana.com/oss/loki

I work DevOps at a 60 person company. I wanted Splunk very badly, but due to pricing had to compromise.

We got SumoLogic. I'm very happy with it relative to ELK.

It's almost as good and much cheaper. The limitations would be that dashboards aren't as customizable and that some advance searches are harder.

It's important to consider whether your requirements include long-term retention and search of data, or only recent search and dashboards. Many of the open-source solutions such as Graylog-free or ELK don't retain information long-term by default, so if long-term data retention is your goal, you should take that into account (Graylog offers it, but only for $$$).

On the other hand, if your primary use-case is near-term searches and reviews of data (e.g. "we just need to see the last 90 days of information for troubleshooting and stats"), you'll be pleasantly surprised with the capabilities in both Graylog and ELK without the additional overhead of Splunk. I'll say, too, that I found working with Filebeat for ingestion to be a lot easier and automation-friendly than working with Splunk forwarders—so much so that we're basically adopting an "only use Splunk for indexing" approach to ingestion and using everything else to get data into it.

I'll give a non-orthodox suggestion: ClickHouse

You'll need to manage some stuff yourself, and assemble your own dashboards and stuff, so there will be some labor involved. That being said, I doubt it will be more painful than managing an ELK stack: there are just too many ways you can destabilize a cluster with it.

ClickHouse clusters from my experience are ridiculously scalable, fast, and stable. There are several other accounts to back that up, and a good case study is Cloudflare, which uses it to store and query all of their logs and metrics from all data centers (that's quite a few PB of data).

There are some projects on GitHub you can use to get inspired, but what you need is pretty much a ClickHouse cluster, Grafana, and a Log Shipper.

I work in the past on a big virtual ecommerce company and their solution was to move all their nosql(primary store)/log data to postgresql for reporting and analysis.

I think that is smart.

P.D: Better to not use nosql but well marketing of nosql back in the days...

p.D2: Most logs are pure NOISE. What I wish to have in some ways to reduce them when incoming and the distilled put in a rdbms. I think this is the best for most but not see much info about this.?

Elastic which used to be called Elasticsearch. There is a free community tier of this, though according to recent drama, not sure how much longer it will remain open as such.

No one mentioned GrayLog. They're a relatively new player in the game, but they show great promise. I should mention that their dashboards are probably not as mature as Splunk and some of the other platforms, but I've met a few people who use it and they seem happy! https://www.graylog.org/

Full disclosure: I work at Humio. Great, cost-efficient monitoring, sub-second ingest latency, on-prem or in the cloud. Feel free to check it out - humio.com

Runs smoother than the ELK stack.

Depending on what you're looking for, you might not want to spend a lot of time maintaining your own stack for dashboard.

For really good user/account dashboards, which is the most common first tool companies build, you check try windsor.io which does most things out of the box with no setup

If you're looking to build custom dashboard and are okay with spending time actually coding and maintaining it (just code snippets, not a full ELK stack) try retool.com

I work at Coralogix https://coralogix.com

We're a relatively new, relatively small player in this space, differentiating by real-time anomaly detection and tooling to help our customers keep costs low (by filtering out logs known to be irrelevant).

Graylog: https://www.graylog.org/

It can also integrate in your existing Splunk setup if you want: https://www.graylog.org/post/graylog-splunk-integration-is-n...

(which allows you to ship data to Graylog and/or Splunk and setup analytics in both as needed)