Self-hosted, OSS logfile analytics?

6 points by debacle 11y ago ↗ HN

After looking at a few solutions for web log analytics, I'm not closer to choosing a solution than I was a few days ago. I've looked at Webalizer, OWA, and Piwik but none of them really offer a logfile integration the way I was anticipating.

Right now I'm looking at Logstash and Kibana but don't really understand the use case for both solutions - Logstash gets my data into elastic search, and Kibana helps me analyze that data?

What's the best tool for the job right now for web analytics (no eCommerce, no ad tracking)?

7 comments

[ 5.2 ms ] story [ 34.1 ms ] thread

dozzie 11y ago ↗

Kibana, not Kobana. And no, it doesn't help you analyze that data. It only helps you browse/search that data, with a limited charting on top.

debacle 11y ago ↗

Thanks, fixed.

I'm really only looking for things like response times, server errors, etc. Will Kibana be able to provide that?

For example, I'd want to run a query for "All requests that took longer than 1.5 seconds that did not return 500 status codes." Is there a way to do that?

weddpros 11y ago ↗

yes, Kibana will do this and more

packetbeats 11y ago ↗

You can see a live demo of Kibana here: http://demo.elastic.co/packetbeat/

lennartkoopmann 11y ago ↗

Take a look at Graylog: www.graylog.org

gesman 11y ago ↗

If your log files are generating <500MB of log files per day you may use Splunk for free.

I wrote and published free splunk app - which i actually built specifically for my dedicated hosting server analytics:

https://splunkbase.splunk.com/app/2676/

This way you're fully in control of your data

sjscott80 11y ago ↗

Hello! I didn't quite understand -- are looking for a log analysis tool that also helps with related web analytics? Disclosure: I work for Logz.io (http://logz.io) and our log analytics platform can do exactly that.

To learn more about the intersection of log analytics and web analytics, you might want to see this Moz post I wrote a few weeks ago on using log data to analyze search engine crawl rates, crawl budgets, bad response codes, crawl priorities, duplicate URL crawling, and more:

https://moz.com/blog/technical-seo-log-analysis

I hope it might help! (Again, full disclosure repeated. :)