Ask HN: How will automation affect data science jobs?

2 points by curo 10y ago ↗ HN

I'm rediscovering my love for math, and considering going from web dev to data science.

Some articles predict the decline of data science jobs because of automation and processing platforms like Platfora, Impala, Splunk, etc.

What does this mean for a data science career? Does this mean required skills will become high-level (e.g., analyst) and salaries will drop for most?*

(* realizing of course some data scientists will need to work on the platforms themselves)

3 comments

[ 3.7 ms ] story [ 9.7 ms ] thread

mswen 10y ago ↗

I have been doing applied statistics in combination with qualitative research and industry analysis; all pretty traditional project based kind of work since the mid 90s. In recent years as I added in NLP, SQL and web dev style programming skills to the mix I find myself building apps that in many ways automate analytical tasks.

With one product I developed for professional and trade associations I estimated that the average association would require as many as 10 full-time entry level analysts working with the typical Excel, Access and Powerpoint tools that so many of them use in order to even come close to replicating the personalized, graphical reporting that my platform could do for an association in real-time.

After feeling a bit guilty here is what I came to realize. The automation that I am offering lets the association offer a set of services to their members that previously would have never been considered because it was financially unthinkable. So if an association decides to use my platform they won't typically eliminate jobs. Instead they will add really substantial capabilities with an external platform supported service and no additional headcount.

This is just my example, but I think that might be true in many cases that automation of analytics (data science) will mostly add capabilities to organizations and not take away many jobs.

dagw 10y ago ↗

Data Science in my experience is 80% data collection, pre-processing, picking the right tool for the job, interpreting the results and communicating the results to the stakeholders. The actual crunching of data is just a small part of it in all but the most extreme cases, and those cases are far too specialized to simply be replaced by automated platforms.

tixocloud 10y ago ↗

If you want to provide additional value as a data scientist, then spend some time to understand the business context. There are some things that you can model with data but we live in a world where there are many other forces at hand that we don't have the capability of inputting it in our model yet.

Traditionally, it's usually a business manager working with the data analyst. If you're someone who can bridge the gap on both sides, your career is likely to be well set and you'll be in great demand.