HN Explorer

Ask HN: As a software engineer, how do you leverage your knowledge in stats?

3 points by inquisitorial ↗ HN
I spend most of my spare time learning statistics since I found that jumping straight into ML left a big gap in my understanding of how inferences and predictions are made.

I would like to use what I'm learning in statistics (confidence intervals, inference, linear regression, sampling, classification, etc.) at work where I am tasked with designing, building, reviewing and maintaining highly scalable and available systems that aren't solving problems using ML or statistics. I find myself thinking really hard about where I could use these concepts in my day job.

As a software engineer, how do you leverage your knowledge in statistics?

Sorry for the vague nature of the question.

2 comments

[ 2.8 ms ] story [ 10.1 ms ] thread
hacknat
I think having a stats background helps a lot in the design of systems, but to be perfectly honest you don’t need much to be effective as an average software engineer.

The place I seem to use it the most is arguing against premature optimization or let’s-just-hack-it-together mentalities.

Two examples:

1. We had a new hire want to go through all of our database queries and optimize them. The only problem? The database in question was only a handful of megabytes large and saw very little usage. Even planning for inordinate success this fact would not have changed.

2. We had an upload feature that we were going to enable for our customers that people just want to quick and dirty hack together. The only problem is that it was going to be handling very large files right away. Even with “low” customer utilization it would be handling gigabytes per day.

As you can see these examples are more about statistical common sense than applying really rigorous statistical modeling to a situation.

If you’re really interested in applying math to your programming, I would recommend looking into getting into finance or science based programming fields.

fenier
I use knowledge of Stats and experiment design to work on A/B testing websites (Website Optimization).

It comes in handy to think through use cases to ensure you can actually learn from the experiment, and you can develop it in ways to avoid confounding variables, such as cell jumping.