Besides the demands of agile and corporate culture, is there other reasons why data science folks are leaving (if true)? For instance lack of cooperation, or not using the insights/recommendations?
Our data scientist left because instead of data science, he ended up doing more ETL and data engineering support than what he originally had signed up for.
A full-time data engineer should probably have been hired.
They both code. The engineer would pull the data from various sources and structure it in a way that the data scientist is happy with. (Tidy format) The DS would the code the models based on the engineer's work, which takes considerably less time.
In our environment, the scientist builds the models, the engineer makes them run. Both can code (the scientist to build models, the engineer to execute models).
I know someone who is hired as a data scientist and is mostly doing the same ETL work. The science is literally 10% of the work anyway, and all DS should know that.
You told me last year that one data scientist is gonna solve all my problems. Now you're telling me I need to babysit him with five data engineers, because he is not going to get his hands dirty with data??
Nobody ever gets to do the fun parts of their job.
Who would sign up if they told you up front that your main contributions would be shuffling email, filling out TPS reports, attending pointless meetings, and being roadblocked every time you tried to automate, eliminate, or improve that drudgery? You can smash your head against the wall trying to make things better, but all that ends up doing is pissing you off and giving you a headache.
Until recently I led a Data Science team (manager and IC) at a non-software Fortune 10. I, and everybody else on the team, left within a year of joining. The division where I was in hired me to grow both Data Science practice and the team. The company was enormous, so there were data science teams all across the company, all operating in silos.
The biggest problem can be summarized like this: If you want to run your business differently, let the new people you hire do something different. If you want to do the same old thing and just act you are doing something differently, just teach the old people to talk like the new people.
I've got plenty of anecdotes from the year, but here's a simple one. The division had an existing "Research and Analytics" group that used linear regression in Excel to develop models. They would develop models on awful (read: highly-multicollinear) data with 100 rows and get "R-squared of .98." Large long-term business decisions were made on these models.
Over the course of the year I fought this constantly. Why do you need more/better data when the other team is okay with the current data? Why are you struggling to get your R-squared to >.90 (output of any type of model had to be R-squared) when the other team can do it so easily? Why do you need to speak to so many people to understand the data better? What I realized over the course of the year was that nobody actually cared about the models unless they matched the "gut" of senior management. So it was easy for the other team to p-hack until they got a slidedeck people would like.
I think there are plenty of problems that lend themselves to good Data Science. These are ones where you can properly identify what a prediction or better understanding of data (e.g. variable importance, coefficient confidence intervals) means in terms of resources (e.g. people, money, time). You can also identify what it means to not have that data insight or model.
There are also plenty of business areas where the link between resource investment and long term success is less clear (e.g. Marketing, Sales Incentives) and more gut and smart work than science. A lot of people are trying to clear up that grey with data science, but I go back to what I said before: you need to let people think and try different things over time to see the value of those experiments.
Comically enough, the Research and Analysis team picked up all the lingo we used and became a huge success: work the old way, talk in the new way. Management decided their Data Science adventure was a failed experiment and we just hadn't lived up to the hype.
I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the company. Your comment is really ringing true, though I've found the company here to be generally great to work with in terms of their flexibility. Funny enough, it took me months of working with the IT department to get a VM with 8-cores and 64 GB of RAM, or the ability to spin up AWS instances (none of the analysts at the company had ever asked for that). I'm happy with it though, it's a fascinating challenge and I'll admit at times I feel a bit in over my head.
In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language.
As opposed to more long-term focus on what business problems can be solved by carefully cleaning data, building models, testing predictive power, and then deploying these models into production. Which obviously takes a lot more time, a lot more effort focusing on a single problem, and doesn't have nearly as many immediate deliverables that higher-ups can open their Outlook inbox and see. Naturally they then wonder why they are paying such a premium for someone who doesn't bring as many deliverables to the table as their 60-70k analysts. And then the data scientist wonders why he's spending most of his time sending Excel attachments. Definitely more of a problem in non-tech companies.
I work for a large consulting firm that does both of these activities (data engineer/analyst delivering rapid value + long-term data scientist / architects engineering fully integrated solutions connecting ML to business process) and I can say without a doubt we get much higher feedback from the the former than the latter.
Most businesses think small. I'm not even sure it's about risk/reward, either. I think people really don't have vision especially when it comes to technology and more specifically data.
IMO every Data Scientist needs to be able to quickly identify a win and deliver quick value, even in a longer term ML project. Iterate and get something out. Most people in large businesses cannot do any data ETL or build a dashboard, so if you can do that in the first few weeks of a project you've already got a deliverable/win.
Having some expertise with BI applications (e.g. Tableau) has been a major strength for showing value.
In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language.
100% Agree.
Some of the execs I reported into were obsessed with the idea of being a startup within a large organization, and part of this was the assumption that one of the things startups did was use data better than big companies. To them, there was some hidden value in their data that their Business Analysts did not have the skills to unlock. Data Scientists are perceived as the magic key to unlock this value, even when what the business is asking for doesn't make sense or isn't feasible in the short-term.
One of the small ways we delivered value was to write small ETL scripts that fed into Tableau dashboards. People loved the Dashboards because it cut down on so much Excel stuff that was done weekly/monthly/quarterly, etc. The company did have a large IT org with a BI team, but they were always busy and took months to do anything. So for us to do ETL+Tableau in a few days was seen as a miracle. Plus, Tableau has an Excel export :)
Of course, ETL+Tableau wasn't enough to retain the team.
Surely you could have used data to have demonstrated how poor the ‘non robust’ model was and won the glory of the guy who argues and proves points(and is disliked for it?). My experience.
What I realized over the course of the year was that data was just a vehicle to justify gut feelings, opinions, and reports from "trusted" external vendors. It was more politics and about gaining visibility than it ever was about the validity of a model.
That's very well put reasons. But another significant reason is very often a highly accurate model is not the right model that an organization wants. Hence when you are taking this modelling exercise to the next level, the whole effort of modelling fails because you have spent your time on something that is not required. Hence the accuracy-sophistication obsession of data scientists - irrespective of they are leaders or just ICs - make it tough to establish their credibility and broader organization vision.
And in many other cases, Media has created a myth of Data Scientists as Magicians, hence a lot of people expect a lot of unknowns from Data Scientists while a lot of people become Data Scientists always expecting to create unknowns and magics which is not going to yield any good better than a mere Exploratory Data Analysis based Insight from Microsoft Excel
probably some realization that 99% of their actual job is ORDER BY, GROUP BY, SUM...not a bunch of cool math scrawled on a whiteboard
I tried warning people off of this field given that they would be doing sql query monkey work and drawing simple line charts...most definitely not an upgrade from software development like it was sold as
They’re probably in the wrong company. Are you suggesting there are isn’t an explosion of jobs and applications requiring advance math and machine learning or statistical inference? If so you are off the mark.
You're right of course, but still there is a point: For 1 'intellectually' interesting data science job created there's probably 10 that have more to do with data massaging and all the relatively boring logistical stuff that come with data science.
Doesn't invalidate your point, but the majority of jobs under the very broad 'data science' label just aren't super interesting after all. Guess you just have to be careful to examine exactly what a specific 'data science' position entails.
I've found this to be very true. I've got friends who were hired as data scientists, paid a great salary, and are finding themselves doing basic reporting and SQL querying all day. No statistical modeling, and certainly no building anything.
Totally depends on what problem your trying to solve, what tools your employer have setup, and what data science you know how to do. The field is still not very mature, but as the tools get easier to use and companies understand how they can solve problems using machine learning and data science, and data pipelines are thought of at the beginning of projects, it will be a lot easier to apply more sophisticated techniques.
as the tools become more sophisticated, the more rote the work will be
if a cloud vendor deploys a tool that derives some canned results very quickly at the push of a button, business leaders will prefer those to magic deep queries that take a weekend to run
The author is way off base in saying software engineering is about assembly rather than discovery. That's only true of the lowest-skill non-innovating software shops, and even then, developers are constantly having to discover how to use the new tool of the month.
On an unrelated note I don't really buy into the idea of data science as a distinct thing from computer science.
> The author is way off base in saying software engineering is about assembly rather than discovery.
Depends. The interview process that is the fad these days selects for assembly workers. Organizations using that process are doing so to find people who aren't curious about broadening their knowledge and experience, but to find people highly competent at repeatedly doing their CS curriculum over and over.
> On an unrelated note I don't really buy into the idea of data science as a distinct thing from computer science
How is it not distinct? Data science is closer to a physical engineering discipline than an applied math one. The tools of the trade might be enhanced by application of CS, but the two are quite distinct.
> Organizations using that process are doing so to find people who aren't curious about broadening their knowledge and experience, but to find people highly competent at repeatedly doing their CS curriculum over and over.
And why are they doing this? Why don't they just buy an existing solution from another vendor, why build your own solution?
It was all hype. All of it. I enjoyed learning about data science, but in the end, there were no jobs I could actually apply for and realistically get. I also don't think they were providing the big wins for the company that would justify what they were getting paid. Again, all hype.
same here. i work great in teams, masters in computer science, do really well in kaggle competitions, I understand when to use the algorithms, how they work, etc.
but in the very few ds interviews I had, I was tanked as soon I got asked questions like: whats the formula for a T test. I know what the test is, and when its not appropriate to apply it, but I don't memorize those kinds of formulas. the field is just too big to do that
Were you making it past the initial stage/screen interviews where every question is a buzz-word test? Were you ever interviewed by people who you'd actually work with and know what their team needs? What geographical area do you reside roughly, if I may ask?
it was initial screenings, she was a data scientist (but really a statistician). I passed the first two screenings (code interview, database / sql interview). every question was about statistics, and probability.
I'm sorry but a standard t-test is like the most basic thing in statistics after means and variances. It's not like they want you prove the CLT or something. To me it's more like a statistics fizz-buzz.
To which, of course, the correct answer is "Unless you have good reason to believe the data is normally distributed, you should be using Mann-Whitney", then explain that :-)
Thats not really true though. The vast majority of t-tests are done using a sample average as a test statistic, and by the CLT, as the sample size goes up, the distribution of a sample average becomes approximately normal. So unless you have a really weak sample size, the t-test assumption holds even in the absence of normally distributed data.
For many companies, data science means statistics. Statistics are used for experiments and to analyze data. If you don't know what a t-test is, you cannot be hired as a statistician. However, you may be hired as a machine learning engineer.
Try looking for a company that does take-home assignments (these are mostly Kaggle-type problems). Then you don't need to learn formulas by rote, and the interview can focus on the assignment you did.
Experimental design (statistics) is important, but data science is a team sport. The people specialized in statistics may not be able to code up a deep net, and vice versa.
Computer scientists make great data scientists, especially if you know data infra. If I were to start a team from scratch, I'd focus on software engineers and physicists. Most are willing to pick up some ML to broaden their skills, and the ability to go end-to-end from conception to production is invaluable.
This is hyperbole. It's not all hype. A lot of it is hype, yes, but there are many examples of great value being derived from advanced analytics.
The real issue is setting expectations with those on the outside trying to get in - a 3-month boot camp with no prior programming or statistics knowledge is just not going to get you to the level of competence required for this type of career. All of these MOOCs for ML/AI/DS are bullshit if you haven't the foundation underneath, which, based off of those people I know who've taken those courses, doesn't exist for many MOOCers.
The fact of the matter is that most companies don't need "artificial intelligence"; they need intelligence - people with domain and statistical knowledge and enough programming skills to be dangerous. As cool as some AI/ML tech is, you could get so far as a data scientist with great knowledge of a few statistical approaches and above-average programming skills.
You're right in one sense, though: there are a lot of data science practices/applications that aren't worth the value they provide. There are others that are worth exponentially more than their operating costs. As with most things in statistics, whether data science is effective "depends."
Ask a databases person and they will tell you about ETLs and relational joins. Ask a statistician and you will hear about estimators and the bias-variance tradeoff. Ask a social scientist and they will tell you about experiments and causal inference. Ask a CS theoretician and they will mention streaming algorithms and probabilistic guarantees. Ask a machine learning person and you will probably hear a lot about prediction and stochastic gradient descent.
The approach that will breed the best practitioners is an interdisciplinary one; a great example is UBC's program: https://masterdatascience.science.ubc.ca/program/courses. And also future positions that separate the roles of data janitor, data analyst, machine learning engineer, and so on. There is a huge difference between what Facebook's Core Data Science team does and what most companies call "data science", which is sitting on a huge pile of poorly aggregated data and having little idea what to do with it.
Most importantly, data science has to be thoughtful and part of the engineering process, and can't be done after the fact with digital trace data. It's a lot easier to run experiments than it is to do statistical gymnastics with observational data.
The dissing of PhDs is unwarranted. Gross over-generalization. People being skeptical of ideas in a peer-review sort of way can be both good and bad. For example, the assumptions and claims of this article can do with a substantial amount of skepticism. Perhaps someone should express skepticism and view this article with the peer-review critical eye that he mentions.
I am reminded of climate-change skeptic articles that say “hey these scientists know nothing. See today it’s 10 below zero. Common sense! No global warming.”
Ideally, reviewers would just want your paper to be as good as it could be.
And yes, we should all appreciate not just the named author of the work, but all of the work that goes into it. This is true of all works, not just studies.
This is an article making baseless generalizations that all data science is failing in Silicon Valley, and that the true need of a company is to hire a "data strategist." Unsurprisingly, the article is written by a self-proclaimed data strategist.
This blog post would be more compelling if there were citations or even general context backing up the authors pretty wild claims ("the vast majority (we are talking 80-90%) want to leave their current job", "failure rate of data science teams is over 90% right now", "the top 5 data scientists I have ever worked with, only 1 had a PhD").
As a data scientist working for a relatively well known startup in the Bay Area, I don't dispute that there are problems in this industry. There are many under-/un-qualified managers running around (I don't know if that's a data science specific problem, or if it also applies to much of engineering as well). There are ICs and teams that have trouble producing industry-applicable work because they're too academically focused. There are also business-driven teams that have trouble producing quality/reproducible work because they're too business focused.
That said, the claim of 90% failure is ridiculous on its face. Having another MBA put together a powerpoint presentation about a company's data strategy isn't going to suddenly solve the actual issues in data science.
That was also my impression. It's kind of ironic that a Data Science/Strategist writing would be so heavily opinionated and have so little in term of data.
Ha, I like that the grammar of this distinguishes between "there is no strategy" and "the strategy is, actively, to produce bullshit" and lands on the second option, which jives with my experience :)
Felt the exact same way after reading this, thank you. The quotes on failure rate and 'vast majority' without any data to back it up, coming from someone who is a data strategist, was a little odd.
In my opinion, I feel like we are at peak "!@#@ data science! @#$@", and there will naturally be some deflation and correction in expectations just like any other occupation that comes out of the woodwork and is the 'sexiest job of the _____'. It's just the way it is. The article author seems to be carving himself out a niche to capitalize on this (which is fine).
Yeah. Academia doesn't care about interesting applications, they care about novel methods.
"I made a huge difference for the business by being thoughtful about feature selection and then applying a bog-standard regression method" isn't appealing to the academic mindset.
To be fair, people are chasing a force multiplier here. Yeah, that is a buzzy term. :( No I don't like that.
So, what I mean is that people are looking for something where they can apply one team's work and use it across many teams. The more the better. The idea is that you don't have to have many people being thoughtful producing methods, you just need many working hard applying them. If there is a better term for that, I'm game to use it.
> There are many under-/un-qualified managers running around (I don't know if that's a data science specific problem, or if it also applies to much of engineering as well).
This is a problem no matter what line of work you're in.
A working knowledge of some basic stats, access to some decent tooling, a good working knowledge of what the organization is trying to accomplish and the ability to persuade and lead based on findings will likely move the revenue needle much more for most companies than a pure "data science" role.
The trouble is of course we don't have a good name for that role, so it's all "data science".
Most managers fail in the "knowledge of basic stats" and "access to decent tools" areas. And if they have the knowledge, most larger organizations don't have a ton of managers sitting around writing python code during the day. Their day is full of meetings.
As someone near graduation with a PhD in mathematics with research focusing on machine learning / image processing and beginning the job search, this didn't leave me feeling very optimistic. Is this a problem of not setting proper expectations for a role on the company's part or not fully understanding the expected duties of the position before hired on the data scientist's part?
You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now consider "what you do" (e.g. ETL, domain interview, co-ordination etc.). Your success and happiness in this will depend a lot on whether you consider those things a waste of time taking you away from what you "really do", or an important part of how you do such activities in larger groups.
On the other hand, data science has a similar problem as software programming; failing to know how to find or produce good managers for it, companies tend to promote from the ranks of their best individual contributors. This can work well with enough support and mentorship (assuming the candidate wants to do it) but it can be a disaster without a plan and that support.
> You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either.
which explains my advisor's "enjoy it while you still can!" attitude whenever we talk of such things :)
A major point that's underappreciated in academia is quality of execution. The more complicated a model is, the less well you can execute it.
Academics are rewarded for building complicated things, but it often takes graduate PhDs entering industry a long time to learn about quality of execution. I think that's the fundamental tension between business and academia.
I wanted to say that I disagreed with you, but then I realized I don't know what you mean by "execution" here, though.
This statement "The more complicated a model is, the less well you can execute it" is fishy though. The typical problem in practice with complicated models is the data quality and quantity to support it, not the implementation or interpretation. I suppose here we also need to define "complexity" - I am using this to mean roughly number of parameters.
I took "execution" here to mean "scaling up". In a trivial case, a model may fail to scale up if it's computationally too slow or too sensitive to noisy data. Sometimes these shortcomings cannot be tested in a laboratory environment with limited data sets.
There's some irony in an article about data science using only anecdote and emotion to make it's argument. :P
PhD's are notoriously (and understandably) finicky about doing non-research work when the job is sold as research, that's why you have to be sure you really need them. That issue has been around since long before data science in SaaS companies was a thing, it's been around since before the internet was a thing.
Assuming that there really is an exodus problem, PhDs being unprepared to lead teams and do non-data-science work is one theory that seems reasonable, but it definitely isn't covering all possibilities.
Having been in a couple of companies hiring data scientists recently, I personally think the big problem is expectation for data science in general, more than who's been hired to do the job. The couple of companies I've worked with, anecdotally, seem to expect "data science" to uncover huge missing profits, expose gaping inefficiencies, and reveal new revenue streams the business people didn't see. And when there turns out to be only minor ways to streamline things, the team of data scientists need something else to do.
I don't blame the practitioners, I blame the business people who foolishly believed it was easy to shake billions out of their data if they applied enough wizardry.
for most, Big Data has been a sham, but it was very profitable for cloud vendors
Maybe these people want to leave because they realize they aren't adding value?
I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud)
I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even close. Usually it is a tremendous money fire. (Also, before the pitchforks come out, I'm sure there are places where the data science team is a profitable department, but it's not the norm, not by any stretch)
Part of the problem is the business doesn't know what questions to ask. Part of the problem is the technology itself. Spark streaming, Hadoop, and all the other tools really aren't very good (very good being defined by helping businesses answer burning questions in a reliable and timely manner)
The most valuable data insights I've seen come from purpose built analytics tools using simple storage backends (RDBMS, elasticsearch etc) where the person running the team is a domain expert, not a "data scientist".
The businesses usually think they have a lot of data because they look it in terms of number of years of data but in practice their datasets are only a couple hundred megabyte large. Yet they setup Spark, Hadoop, and all of that stuff to "extract insights" from it. Usually all they need is someone who knows python to write a script to parse their data and put it in Postgres
Dollars to donuts, this stuff always seems to be in that enterprise buzzword laden minefield of trendy make-work. There seems to be a tremendous amount of money and energy behind it for some reason, though.
I've seen some customers of ours being quoted six and seven figure prices by their internal data science teams to essentially slurp and transform a handful of SQL tables. It's the kind of thing that shouldn't cost six or seven hours to write the code for.
> I've seen some customers of ours being quoted six and seven figure prices by their internal data science teams to essentially slurp and transform a handful of SQL tables.
They don't wanna do it so they give a ridiculous estimate
I'm a firm believer that data science should be pulled out of IT and put into the business. However, storage is so cheap that you should never NOT collect data if you can. It's better to dump it into a data warehouse and wait for somebody who can use it to come along than it is to never collect it.
Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or Spark infrastructure.
I like data too but that approach has several hazards. The most obvious is liability: if you store it, you have to be responsible for protecting against misuse.
The second, however, is more subtle but often more damaging: people often assume that the data they have is the right data or complete so the drunkard's lampost problem is easy to fall into and people might not realize it as quickly because, hey, those numbers were based on so much data it took a day to run the query! Web analytics is notorious for that — people would make statements about performance, browser support, etc. from a bunch of log files and miss that this was skewed by bot traffic, confuse their server's response time as being a reasonable proxy for the user's perceived load-time, etc.
>I'm a firm believer that data science should be pulled out of IT and put into the business.
I say this from personal experience. The major risk here is that Data Scientists will constantly get pulled in as resources to support business fires and high visibility projects that should be handled by other people.
Otherwise what happens is this: "I know you are working on that Data Science project that is scoped for 4 weeks, but can you do this Business Analyst thing the insert exec name here asked for by this Friday?"
Data Scientists need to be shielded from that day to day analyst stuff, otherwise they'll never be able to do their jobs. One of the ways to do this is to put them in IT in a business facing consulting role. IT is typically (not always) is more focused on a proper solution, not day to day triage. So they can filter out projects that aren't appropriate.
This may end up repeating the mistakes of a decade or two ago: after a business leader gets two requests rejected, they are very unlikely to go back with a third request - they will simply outsource every future request. Soon the IT departments were being downsized because they couldn't get business leaders to use them instead of going outside.
Data should be moved out of IT and put into Data. Data headed up by a Chief Data Officer should be a first class citizen of every company. Not f’in IT, not Buisness. Data.
I think this is a key point - too often, data "scientists" are brought in, given lots of tools, paid lots of money, and then told to just count things. And build dashboards that count things.
The data scientists that I have seen that are successful and bring the most value and seem to have the most satisfaction are those that spend time actually doing analysis and provide deep insights into the problems they are looking in to. The answers are the easy part - asking the right question is the difficult part.
I really question this "90%" number for "failed" data science teams.
While I certainly can see overhype in the space leading to team failure (and I have indeed seen that), I'm just surprised that the 10% of teams I've been on were so successful by comparison.
But one of the nasty things I see happening in the space is that often times really great data science folks, in studying a space, come up with a way to invalidate a lot of the business models. If a major player adopted these, there is a chance of success, but it's also a major risk (as it basically unseats the rest of the business).
As such, I think there is a lot of pressure for good data scientists to end up in small teams leading what we'd call 'highly disruptive' businesses. We see this in the finance sector (where it's least likely to succeed) all the time.
Speak for yourself. I work as a data scientist at a large (non-tech) company. Our team of data scientists is directly responsible for massive amounts of profit that the company would not otherwise have. This article seems like a baseless generalization.
A lot of value from Data Science comes in large non-tech companies. It's not the super glamorous type of DS. It is about using data to drive optimizations, process improvements, manage supply chains, instrumenting alert mechanisms to monitor key mechanical and software components etc.
I've never seen a "data scientist" produce anything meaningful, at least at my company. They run some data through some Python framework and produce weird meaningless numbers that I'm supposed to be impressed by or to trust. No insight as to how that number came to be. "87.525%". Ok sweet. Could've come from /dev/urandom for all I know. At least 3 of them have left in the 2 years that I've been here.
It seems like 90% of these guys are buzzword wankers who jumped onto the "Big $$$" bandwagon. Same type of person who put "PHP Wizard" on their resume in 2005.
Good riddance.
I'm sure, of course, that quite a few of them are decent, intelligent people producing real value at real companies. But the ones I've dealt with? Nah.
In a company that's thousands of people large, across 5 continents, I have the power to make dozens of people, most of whom I've never communicated with outside of the occasional email, leave? I basically have superpowers.
Like I said, I'm sure "data science" as a science is great, and I'm sure there's plenty of "data scientists" that I would love to sit down and talk with and learn from. I just have never met any.
Data science and machine learning are different beasts. CS based Machine learning and AI seem useful in task automation. Its real value is in building IP.
DATA science is more like quant consulting.It is actually a subdiscipline in management science in my opinion.
The key role of a data scientist in non SV company is usually 6-fold.
First understand the business problem thoroughly. Get the people,process and value stack nailed.
Second and this is the most imp part. Figure out how to frame the business problem. Framing isnt as linear as it looks. But it helps save a ton of time and money if you front load time in framing problem. Substep here is to frame it as an "information" or data problem.
Third articulate a strategy to gather data or information. Inhouse data available? Web scrape data, buy data, build generative models. Youd be suprised how much ML work happens in this step
Fourth shape the information to a format that can be analysed.
Fifth, study the hell out of this dataset- the PHD angle comes here. Very few people are systematically trained to study data with rigor and be humble and honest about their findings.
Lastly connect the dots between insights and business problem at hand.too often this stops with bar graphs and some scatter plots. Thats just lazy. You need to really take ownership of the problem and educate business why the solution will help and be honest about ehat it will take to get it to work. The people, process,value stack in line 1 kicks in here in recommendation and action.
A bonus point. Put some kpi to track how well your suggestions are working.
Real data science is super super hard. Its like finding a real nugget of diamond in dust.
Data science is a way thinking. Its a business culture to be honest.
Pardon the cynicism, but all I see when I read this is MBA turd-word lobbing. It is meaningless drivel, bloviating self-promotion directed at people who don't understand anything about how their world works while thinking they are qualified to lead it because of X degree at Y institution, the admittance of whom was due to Z (powerful rich person) that is friends with his dad.
Others have noted the lack of data coming from a data strategist. That is the feature, not the flaw. Data isn't necessary in these people's worlds. They spout platitudes about data, laud it when it supports their world view, and ignore it when it doesn't. People that buy the world that this guy is selling are suckers.
As someone (a Director) running for the exit... lack of control over: taking on new projects, staffing, infrastructure, meeting schedules, deadlines. Complete lack of ability to say "no" to our biggest client. Add in conflicting priorities from leadership, month long delays in compensation adjustment, lack of clarity around valuation status, and having to focus primarily on new and maintenance ETL work.
I got frustrated enough to where I almost quit on the spot a few weeks ago. Frustrated enough to b on the verge of tears and punches because I felt I'd wasted the past two years of my life. I worked to get myself transferred from my leadership role to, ostensibly, a more thought leadership type role which should happen in next few weeks.
However, if I don't see significant changes in January, February, and March then I'm leaving. Honestly, I'll probably leave anyway because I think I can grow my skillset, work on better problems, and set my future up better somewhere else.
What I hate about this situation is that I used to be the biggest supporter this company had. But years of this difficulty is enough.
I feel ya; this is almost exactly what I've felt this year. It's just incompetent management and leadership, and it's all too common. But it's incredibly demoralizing to be on the front lines of it. When you can't control anything, and are reduced to reacting on ever more tactical issues, it burns you out hard and fast. Indecisiveness is killer; better to pick something and do it, than waffle around forever spinning the wheels.
101 comments
[ 0.25 ms ] story [ 180 ms ] threadA full-time data engineer should probably have been hired.
I wonder if this leads to an analogous discussion as http://wiki.c2.com/?ArchitectsDontCode.
You told me last year that one data scientist is gonna solve all my problems. Now you're telling me I need to babysit him with five data engineers, because he is not going to get his hands dirty with data??
Who would sign up if they told you up front that your main contributions would be shuffling email, filling out TPS reports, attending pointless meetings, and being roadblocked every time you tried to automate, eliminate, or improve that drudgery? You can smash your head against the wall trying to make things better, but all that ends up doing is pissing you off and giving you a headache.
The biggest problem can be summarized like this: If you want to run your business differently, let the new people you hire do something different. If you want to do the same old thing and just act you are doing something differently, just teach the old people to talk like the new people.
I've got plenty of anecdotes from the year, but here's a simple one. The division had an existing "Research and Analytics" group that used linear regression in Excel to develop models. They would develop models on awful (read: highly-multicollinear) data with 100 rows and get "R-squared of .98." Large long-term business decisions were made on these models.
Over the course of the year I fought this constantly. Why do you need more/better data when the other team is okay with the current data? Why are you struggling to get your R-squared to >.90 (output of any type of model had to be R-squared) when the other team can do it so easily? Why do you need to speak to so many people to understand the data better? What I realized over the course of the year was that nobody actually cared about the models unless they matched the "gut" of senior management. So it was easy for the other team to p-hack until they got a slidedeck people would like.
I think there are plenty of problems that lend themselves to good Data Science. These are ones where you can properly identify what a prediction or better understanding of data (e.g. variable importance, coefficient confidence intervals) means in terms of resources (e.g. people, money, time). You can also identify what it means to not have that data insight or model.
There are also plenty of business areas where the link between resource investment and long term success is less clear (e.g. Marketing, Sales Incentives) and more gut and smart work than science. A lot of people are trying to clear up that grey with data science, but I go back to what I said before: you need to let people think and try different things over time to see the value of those experiments.
Comically enough, the Research and Analysis team picked up all the lingo we used and became a huge success: work the old way, talk in the new way. Management decided their Data Science adventure was a failed experiment and we just hadn't lived up to the hype.
I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the company. Your comment is really ringing true, though I've found the company here to be generally great to work with in terms of their flexibility. Funny enough, it took me months of working with the IT department to get a VM with 8-cores and 64 GB of RAM, or the ability to spin up AWS instances (none of the analysts at the company had ever asked for that). I'm happy with it though, it's a fascinating challenge and I'll admit at times I feel a bit in over my head.
In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language.
As opposed to more long-term focus on what business problems can be solved by carefully cleaning data, building models, testing predictive power, and then deploying these models into production. Which obviously takes a lot more time, a lot more effort focusing on a single problem, and doesn't have nearly as many immediate deliverables that higher-ups can open their Outlook inbox and see. Naturally they then wonder why they are paying such a premium for someone who doesn't bring as many deliverables to the table as their 60-70k analysts. And then the data scientist wonders why he's spending most of his time sending Excel attachments. Definitely more of a problem in non-tech companies.
Most businesses think small. I'm not even sure it's about risk/reward, either. I think people really don't have vision especially when it comes to technology and more specifically data.
The next generation is coming, though.
Having some expertise with BI applications (e.g. Tableau) has been a major strength for showing value.
100% Agree.
Some of the execs I reported into were obsessed with the idea of being a startup within a large organization, and part of this was the assumption that one of the things startups did was use data better than big companies. To them, there was some hidden value in their data that their Business Analysts did not have the skills to unlock. Data Scientists are perceived as the magic key to unlock this value, even when what the business is asking for doesn't make sense or isn't feasible in the short-term.
One of the small ways we delivered value was to write small ETL scripts that fed into Tableau dashboards. People loved the Dashboards because it cut down on so much Excel stuff that was done weekly/monthly/quarterly, etc. The company did have a large IT org with a BI team, but they were always busy and took months to do anything. So for us to do ETL+Tableau in a few days was seen as a miracle. Plus, Tableau has an Excel export :)
Of course, ETL+Tableau wasn't enough to retain the team.
And in many other cases, Media has created a myth of Data Scientists as Magicians, hence a lot of people expect a lot of unknowns from Data Scientists while a lot of people become Data Scientists always expecting to create unknowns and magics which is not going to yield any good better than a mere Exploratory Data Analysis based Insight from Microsoft Excel
I tried warning people off of this field given that they would be doing sql query monkey work and drawing simple line charts...most definitely not an upgrade from software development like it was sold as
Doesn't invalidate your point, but the majority of jobs under the very broad 'data science' label just aren't super interesting after all. Guess you just have to be careful to examine exactly what a specific 'data science' position entails.
Any decent Comp. Sci. University will have enough math and statistics.
if a cloud vendor deploys a tool that derives some canned results very quickly at the push of a button, business leaders will prefer those to magic deep queries that take a weekend to run
On an unrelated note I don't really buy into the idea of data science as a distinct thing from computer science.
Depends. The interview process that is the fad these days selects for assembly workers. Organizations using that process are doing so to find people who aren't curious about broadening their knowledge and experience, but to find people highly competent at repeatedly doing their CS curriculum over and over.
> On an unrelated note I don't really buy into the idea of data science as a distinct thing from computer science
How is it not distinct? Data science is closer to a physical engineering discipline than an applied math one. The tools of the trade might be enhanced by application of CS, but the two are quite distinct.
And why are they doing this? Why don't they just buy an existing solution from another vendor, why build your own solution?
There's a definite place for these workers.
but in the very few ds interviews I had, I was tanked as soon I got asked questions like: whats the formula for a T test. I know what the test is, and when its not appropriate to apply it, but I don't memorize those kinds of formulas. the field is just too big to do that
Were you making it past the initial stage/screen interviews where every question is a buzz-word test? Were you ever interviewed by people who you'd actually work with and know what their team needs? What geographical area do you reside roughly, if I may ask?
This position was in NYC.
Experimental design (statistics) is important, but data science is a team sport. The people specialized in statistics may not be able to code up a deep net, and vice versa.
Computer scientists make great data scientists, especially if you know data infra. If I were to start a team from scratch, I'd focus on software engineers and physicists. Most are willing to pick up some ML to broaden their skills, and the ability to go end-to-end from conception to production is invaluable.
The real issue is setting expectations with those on the outside trying to get in - a 3-month boot camp with no prior programming or statistics knowledge is just not going to get you to the level of competence required for this type of career. All of these MOOCs for ML/AI/DS are bullshit if you haven't the foundation underneath, which, based off of those people I know who've taken those courses, doesn't exist for many MOOCers.
The fact of the matter is that most companies don't need "artificial intelligence"; they need intelligence - people with domain and statistical knowledge and enough programming skills to be dangerous. As cool as some AI/ML tech is, you could get so far as a data scientist with great knowledge of a few statistical approaches and above-average programming skills.
You're right in one sense, though: there are a lot of data science practices/applications that aren't worth the value they provide. There are others that are worth exponentially more than their operating costs. As with most things in statistics, whether data science is effective "depends."
Ask a databases person and they will tell you about ETLs and relational joins. Ask a statistician and you will hear about estimators and the bias-variance tradeoff. Ask a social scientist and they will tell you about experiments and causal inference. Ask a CS theoretician and they will mention streaming algorithms and probabilistic guarantees. Ask a machine learning person and you will probably hear a lot about prediction and stochastic gradient descent.
The approach that will breed the best practitioners is an interdisciplinary one; a great example is UBC's program: https://masterdatascience.science.ubc.ca/program/courses. And also future positions that separate the roles of data janitor, data analyst, machine learning engineer, and so on. There is a huge difference between what Facebook's Core Data Science team does and what most companies call "data science", which is sitting on a huge pile of poorly aggregated data and having little idea what to do with it.
Most importantly, data science has to be thoughtful and part of the engineering process, and can't be done after the fact with digital trace data. It's a lot easier to run experiments than it is to do statistical gymnastics with observational data.
I am reminded of climate-change skeptic articles that say “hey these scientists know nothing. See today it’s 10 below zero. Common sense! No global warming.”
And yes, we should all appreciate not just the named author of the work, but all of the work that goes into it. This is true of all works, not just studies.
That is a recipe for disaster - non-technical people leading technical people. Maybe we should hire an MBA as VP of Engineering too?
This blog post would be more compelling if there were citations or even general context backing up the authors pretty wild claims ("the vast majority (we are talking 80-90%) want to leave their current job", "failure rate of data science teams is over 90% right now", "the top 5 data scientists I have ever worked with, only 1 had a PhD").
As a data scientist working for a relatively well known startup in the Bay Area, I don't dispute that there are problems in this industry. There are many under-/un-qualified managers running around (I don't know if that's a data science specific problem, or if it also applies to much of engineering as well). There are ICs and teams that have trouble producing industry-applicable work because they're too academically focused. There are also business-driven teams that have trouble producing quality/reproducible work because they're too business focused.
That said, the claim of 90% failure is ridiculous on its face. Having another MBA put together a powerpoint presentation about a company's data strategy isn't going to suddenly solve the actual issues in data science.
In my opinion, I feel like we are at peak "!@#@ data science! @#$@", and there will naturally be some deflation and correction in expectations just like any other occupation that comes out of the woodwork and is the 'sexiest job of the _____'. It's just the way it is. The article author seems to be carving himself out a niche to capitalize on this (which is fine).
Ironically, I assumed he was talking about NYC. Apparently he's actually in Minneapolis
"I made a huge difference for the business by being thoughtful about feature selection and then applying a bog-standard regression method" isn't appealing to the academic mindset.
So, what I mean is that people are looking for something where they can apply one team's work and use it across many teams. The more the better. The idea is that you don't have to have many people being thoughtful producing methods, you just need many working hard applying them. If there is a better term for that, I'm game to use it.
This is a problem no matter what line of work you're in.
I considered flagging the article but decided against it after reading your comment, which I hope remains visible.
It is unsupported in the article, but on its face, isn't a 90% failure rate about right for bleeding edge tech efforts?
The trouble is of course we don't have a good name for that role, so it's all "data science".
You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now consider "what you do" (e.g. ETL, domain interview, co-ordination etc.). Your success and happiness in this will depend a lot on whether you consider those things a waste of time taking you away from what you "really do", or an important part of how you do such activities in larger groups.
On the other hand, data science has a similar problem as software programming; failing to know how to find or produce good managers for it, companies tend to promote from the ranks of their best individual contributors. This can work well with enough support and mentorship (assuming the candidate wants to do it) but it can be a disaster without a plan and that support.
which explains my advisor's "enjoy it while you still can!" attitude whenever we talk of such things :)
Academics are rewarded for building complicated things, but it often takes graduate PhDs entering industry a long time to learn about quality of execution. I think that's the fundamental tension between business and academia.
This statement "The more complicated a model is, the less well you can execute it" is fishy though. The typical problem in practice with complicated models is the data quality and quantity to support it, not the implementation or interpretation. I suppose here we also need to define "complexity" - I am using this to mean roughly number of parameters.
PhD's are notoriously (and understandably) finicky about doing non-research work when the job is sold as research, that's why you have to be sure you really need them. That issue has been around since long before data science in SaaS companies was a thing, it's been around since before the internet was a thing.
Assuming that there really is an exodus problem, PhDs being unprepared to lead teams and do non-data-science work is one theory that seems reasonable, but it definitely isn't covering all possibilities.
Having been in a couple of companies hiring data scientists recently, I personally think the big problem is expectation for data science in general, more than who's been hired to do the job. The couple of companies I've worked with, anecdotally, seem to expect "data science" to uncover huge missing profits, expose gaping inefficiencies, and reveal new revenue streams the business people didn't see. And when there turns out to be only minor ways to streamline things, the team of data scientists need something else to do.
for most, Big Data has been a sham, but it was very profitable for cloud vendors
I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud)
I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even close. Usually it is a tremendous money fire. (Also, before the pitchforks come out, I'm sure there are places where the data science team is a profitable department, but it's not the norm, not by any stretch)
Part of the problem is the business doesn't know what questions to ask. Part of the problem is the technology itself. Spark streaming, Hadoop, and all the other tools really aren't very good (very good being defined by helping businesses answer burning questions in a reliable and timely manner)
The most valuable data insights I've seen come from purpose built analytics tools using simple storage backends (RDBMS, elasticsearch etc) where the person running the team is a domain expert, not a "data scientist".
I've seen some customers of ours being quoted six and seven figure prices by their internal data science teams to essentially slurp and transform a handful of SQL tables. It's the kind of thing that shouldn't cost six or seven hours to write the code for.
They don't wanna do it so they give a ridiculous estimate
Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or Spark infrastructure.
Unless it involves customers and clients, in which case it's actually a sneaky liability... At least in terms of PR, if not legality.
The second, however, is more subtle but often more damaging: people often assume that the data they have is the right data or complete so the drunkard's lampost problem is easy to fall into and people might not realize it as quickly because, hey, those numbers were based on so much data it took a day to run the query! Web analytics is notorious for that — people would make statements about performance, browser support, etc. from a bunch of log files and miss that this was skewed by bot traffic, confuse their server's response time as being a reasonable proxy for the user's perceived load-time, etc.
I say this from personal experience. The major risk here is that Data Scientists will constantly get pulled in as resources to support business fires and high visibility projects that should be handled by other people.
Otherwise what happens is this: "I know you are working on that Data Science project that is scoped for 4 weeks, but can you do this Business Analyst thing the insert exec name here asked for by this Friday?"
Data Scientists need to be shielded from that day to day analyst stuff, otherwise they'll never be able to do their jobs. One of the ways to do this is to put them in IT in a business facing consulting role. IT is typically (not always) is more focused on a proper solution, not day to day triage. So they can filter out projects that aren't appropriate.
The data scientists that I have seen that are successful and bring the most value and seem to have the most satisfaction are those that spend time actually doing analysis and provide deep insights into the problems they are looking in to. The answers are the easy part - asking the right question is the difficult part.
While I certainly can see overhype in the space leading to team failure (and I have indeed seen that), I'm just surprised that the 10% of teams I've been on were so successful by comparison.
But one of the nasty things I see happening in the space is that often times really great data science folks, in studying a space, come up with a way to invalidate a lot of the business models. If a major player adopted these, there is a chance of success, but it's also a major risk (as it basically unseats the rest of the business).
As such, I think there is a lot of pressure for good data scientists to end up in small teams leading what we'd call 'highly disruptive' businesses. We see this in the finance sector (where it's least likely to succeed) all the time.
It seems like 90% of these guys are buzzword wankers who jumped onto the "Big $$$" bandwagon. Same type of person who put "PHP Wizard" on their resume in 2005.
Good riddance.
I'm sure, of course, that quite a few of them are decent, intelligent people producing real value at real companies. But the ones I've dealt with? Nah.
I think many of them are PhD students who are leaving academia, voluntarily or not, and data science is a logical first job.
Like I said, I'm sure "data science" as a science is great, and I'm sure there's plenty of "data scientists" that I would love to sit down and talk with and learn from. I just have never met any.
DATA science is more like quant consulting.It is actually a subdiscipline in management science in my opinion.
The key role of a data scientist in non SV company is usually 6-fold. First understand the business problem thoroughly. Get the people,process and value stack nailed.
Second and this is the most imp part. Figure out how to frame the business problem. Framing isnt as linear as it looks. But it helps save a ton of time and money if you front load time in framing problem. Substep here is to frame it as an "information" or data problem.
Third articulate a strategy to gather data or information. Inhouse data available? Web scrape data, buy data, build generative models. Youd be suprised how much ML work happens in this step
Fourth shape the information to a format that can be analysed.
Fifth, study the hell out of this dataset- the PHD angle comes here. Very few people are systematically trained to study data with rigor and be humble and honest about their findings.
Lastly connect the dots between insights and business problem at hand.too often this stops with bar graphs and some scatter plots. Thats just lazy. You need to really take ownership of the problem and educate business why the solution will help and be honest about ehat it will take to get it to work. The people, process,value stack in line 1 kicks in here in recommendation and action.
A bonus point. Put some kpi to track how well your suggestions are working.
Real data science is super super hard. Its like finding a real nugget of diamond in dust.
Data science is a way thinking. Its a business culture to be honest.
Others have noted the lack of data coming from a data strategist. That is the feature, not the flaw. Data isn't necessary in these people's worlds. They spout platitudes about data, laud it when it supports their world view, and ignore it when it doesn't. People that buy the world that this guy is selling are suckers.
Alot of times, it means "the guy who does crap in SPSS or needs Tableau".
I got frustrated enough to where I almost quit on the spot a few weeks ago. Frustrated enough to b on the verge of tears and punches because I felt I'd wasted the past two years of my life. I worked to get myself transferred from my leadership role to, ostensibly, a more thought leadership type role which should happen in next few weeks.
However, if I don't see significant changes in January, February, and March then I'm leaving. Honestly, I'll probably leave anyway because I think I can grow my skillset, work on better problems, and set my future up better somewhere else.
What I hate about this situation is that I used to be the biggest supporter this company had. But years of this difficulty is enough.