HN Explorer

Ask HN: When and how to argue with data driven decisions?

8 points by raymondgh ↗ HN
Sometimes you've got a philosophical, gut, or value-driven idea about how something should be that disagrees with an interpretation of data. In an increasingly developer-focused startup environment, people choose the most easily quantifiable metric to make decisions. At the same time, it's hard to ever claim you have enough data or the right data to make some decisions. I've struggled with the balance between Vision and Data as drivers for decision-making. Sometimes I feel like people, for lack of a better word, resort to data to make decisions. How have you dealt with this in the past and how would you in the future?

5 comments

[ 2.0 ms ] story [ 23.9 ms ] thread
gyardley
People who rely on vision alone have never had their vision be spectacularly, embarrassingly wrong yet. Make sure you don't have lucky-product-manager syndrome, where a couple of good decisions early in your career lead you to rely on your gut a lot more than you should.

If you're working with a team, you should probably deal with this by gradually earning the full trust of your team - people want to 'resort' to data to make decisions because they don't trust you enough to make a decision that goes against that data.

Failing that, you can get better data that supports what you want to do, cast enough doubt on the existing data that people are willing to ignore it, or use your organizational clout to just override them, in descending order of usefulness.

If you're working by yourself, you can make any decision you want to, whether the data supports it or not! But if you're the only person affected by the decision and you're still struggling with and having difficulty going against the data, that should tell you something right there - your gut might be speaking, but it's not speaking loudly enough to listen to.

Gustomaximus
Look at the quality and interpretation of the data as simply because data is presented, it doesn't mean it is correct, or relevant to a decision. E.g look for trends vs single data points. This is one of the most common errors I see in data based decision making.

Secondly if your looking at historic trends you can recognise out the past doesn't always predict the future. Be careful not to overuse this as I would say this is the exceptation to the rule, especially if looking at trends.

Thirdly challenge the gut feeling. If the data is overwhelmingly showing one things I'd be inclined to support it. I find with a gut feeling if you sit on it you will understand better why you feel that way.

Forth, take advise if you have doubt. Do you have a few go-to people who you respect their judgment or can help you clarify yours?

More generally, I think first point is pertinent. I work in a field that is very much part art, part science. In this area it's considered a must to say you're 'data driven' but the reality is only the minority really understand data driven decision making. Data and useful data are two entirely different things.

Does that help?

sjg007
the old Google vs Apple dichotomy.
mud_dauber
I've always wanted to be the 'data guy'. As a product manager, I can say that in 30 years I've never had a dataset that truly is a slam dunk.

People - sales managers, waffling customers, risk-adverse GMs - are always the primary sources of information. I've never had the luxury of simply pulling a SaaS analytics report and saying "there.".

I'm sorry that this doesn't answer your question. The only path out of the box is to clearly state the alternatives, advocate a preferred path going forward, and demand that people buy in or step the hell aside.

Otherwise you'll simply develop an ulcer.

psyklic
Data is typically not very conclusive by itself. So, people often make a number of behind-the-scenes decisions that make the conclusions seem stronger than they actually are (whether on purpose or not).

For a great example of this, try the interactive portion of the following article, where you'll use data to support an argument whether Democrats or Republicans are better for the US economy: http://fivethirtyeight.com/features/science-isnt-broken/.

Takeaway: Both conclusions are valid ... it all depends on you define "economy" and what subset of Democrats/Republicans you choose to count.