Ask HN: Making sense of your metrics data - what are your tools of choice?

We use flurry for mobile, google analytics for web and in house server logging for both. I use a custom stack of scripts, SQL and R to make sense of it.

What does everyone do for higher level analysis of their event logs? Lets say you have a concept of users, and a concept of most actions that they take, how do you build a picture of who you users are, what they are doing, what the main user segments are, etc?

Am I missing out on any specialized tools? Note, tools must be able to handle fairly large datasets, the tools I have seen that slice up SQL databases usually end up being too slow.

I'm currently developing a custom system to track all requests and events on my app that will allow me to map visits and see how users move through the site in addition to specific metrics.

We use google analytics but that doesn't provide the type of segmentation and exact event tracking I'm looking for. It may be possible, but I find the integration of specific event types will provide a lot of insight that google can't. Also I really want some specific data on demo users with the new registration less sign up I integrated.

We just starting using KISSMetrics http://kissmetrics.com/ started loving it, you actually describe your app funnel.. Also has a beautiful A/B Split Test API

Other than things mentioned here, we use an in-house graphing solution built on top of Postgres and custom views. This is great because we log almost everything and it's easy to join across users, etc.

Tableau also seems fairly promising, though it is really slow on live db queries. Does anyone here use Tableau?

Mozilla uses WebTrends for most of its web metrics like visitor and session analysis. We use SiteSpec for multivariate testing. For everything else we use Kimball's dimensional modeling to build data warehouses and then build reports, dashboards, and analysis off of them. We make extensive Pentaho's suite of tools for ETL, multi-dimensional querying of OLAP cubes, report creation/publishing, and a website to deliver interactive dashboards.

I would strongly recommend that anyone asking these questions do some extended web reading or pick up a few books on the topics of ETL, Data Warehousing, and Business Intelligence. I feel it is critical to have experience these fields if you need to derive insight from data. Using open source ETL/BI tools can help a startup without blowing the budget. When they discover the value for their company, they can look for specialists to join the team or they can investigate support and contracting options available.

In particular, a good ETL tool like Pentaho Data Integration is invaluable to any company that has to do anything with moving data from one place to another. Even if you don't use it for traditional BI.

For our Socorro project, we are making use of a Hadoop/HBase cluster to store large amounts of crash report data. The next two versions of Socorro will be pushing significantly more analysis into MapReduce and Hive type queries.

We are also working on a new data storage and analysis backend for Test Pilot which is based on Riak. The interesting part on the analysis side is that it is performed by submitting MapReduce jobs vis a simple HTTP POST to the REST API.