Look, this is evil. I don't want to hear any complaints about my calling a spade a spade. The Nazis did it: evil. Google does it, and it is evil, and they go out of their way to say WE ARE NOT EVIL and guess what: it is evil.
Why is it evil? You have a bunch of people in corporate america who are FORCING an information technology posture on the American people that the American people HATE because it is HORRIBLE and EVIL. Next, these EVIL people go around saying what they are doing is "data science" and start blaming mathematicians.
This is absolutely 100% evil. Mathematicians have zero interest in this. Pointing the finger at mathematicians and calling them "data scientists" is intolerable stupidity and the people who do it should expect to get their fingers cut off by mathematicians who don't like to be pointed at by IDIOTS.
•Can you walk away from a problem when the solution is “good enough,” are you able to switch between tasks or problems with relative ease? Are you OK with simple solutions to problems that could have more complicated solutions, but only with rapidly diminishing returns?
Translation: Can you be a good assembly line worker, devoid of pride or curiosity and just do the damned job? If so, great, we can use you! If not, please return to academia.
A lot of things in the industry are built via hacks where good enough is good enough. The goal is not to satiate curiosity but to generate business value. This does not equate to lack of pride, just a different goal.
Yeah, like the IoT. I'm sorry, but we've reached the point where that level of blithe pursuit of a "different goal" is becoming destructive on a broad scale.
A lot of successful academics I know (including mathematicians) operate on similar principles. Not everybody embraces the Gowers/Perelman/Zhang model, nor should they.
That is not the intended meaning at all. It can be a great point of pride and skill to find a solution that is as good as possible subject to complexity and time constraints that do not exist in academia.
I think in academia you have the same issues, maybe not to deliver a product but to deliver papers. You get use to the good enough and moving to the next thing, and you're constantly switching between projects, students, students projects... I haven't seen much diff in academia and work, except naming the same thing differently.
I'm sorry, what? Being a good engineer in the real world means building quality solutions within time and cost constraints. It's a craft. One can be a good engineer and have both pride and curiosity. You also need humility and the ability to put your customer's needs ahead of your ego.
I mean, by all means open up a little boutique data science consultancy that charges three orders of magnitude more for a solution that can't be updated by anyone without a ph.d and only beats an out of the box svm by .1%, where you put your pride and curiosity (your vanity) first. You might even find some suckers to patronize you, but man... I must be misunderstanding you because that is the most clueless and entitled thing I've read in days.
98% of the time data science customers actually just want some basic statistical insights into their data to make better informed decisions. If you aren't willing to help with that, and also can't find a research group to take you in (to work on their projects) and also can't bootstrap your own startup, then yes, stay in academia.
The money is good because people are willing to pay. If the world needed more PHDs, then the world would pay them more. Thats how markets work. Supply and demand.
This definition of engineer is so broad as to be meaningless. There is no "fake world" with "fake world" problems, pretty much everyone is trying to create real world solutions to real world problems. Similarly most jobs are focused on the end goal. A data scientist job usually involves a mix of science and engineering.
My definition of "real world" problems are ones that people are willing to pay for. If nobody is willing to give you money for whatever it is that you are doing, then it probably isn't solving someone's actual problem.
The problem with this approach is that people pay much more money for limited gains they can exploit than for general gains that benefit everyone equally. Advancing academic knowledge provides gains distributed over a very large number of people, but the current system often does not efficiently allocate resources to this problem.
Well, I went from academia (theoretical physics) to data science, and "good enough" solutions is exactly the thing I like the most.
That is - things that rather solve a concrete problem (given all constraints, including: dirty data, finite time, understanding the needs of clients) rather that have a pure and beautiful, but utterly useless for practical purposes, theorem to prove.
Haha, my buddy is in numerical physics.
His ideas of good enough is never touching my codebase.
He is smart though, and is starting to get why we have build systems and test coverage ;)
I've been encouraging my daughter, a statistics major, to pursue data science by including Python/R in her studies and then possibly heading back for an MBA. But not sure if an MBA would be a benefit. Thoughts from actual data scientists?
Personally I think a PhD would be better than an MBA if she really wants to do data science, but generally speaking I'm for as more edu as possible, and I'm sure she'll benefit from an MBA too.
The PhD will only be useful in a small (but growing) subset of data science jobs. Their are data scientists who develop new algorithms and techniques, and those who apply the. For application, the PhD is probably overkill and extensive experience with a bachelor's, or a master's is better. For theory the PhD can't be beat, of course. I say the opportunity costs are high, and should be carefully weighed, because I have my bachelor's and am making more than a data scientist friend of mine who has his PhD. The vast majority of what he learned in the PhD program he hasn't found useful in the real world (yet, anyways). So it's all about what problems you want to work on. Personally I thrive in the application arena of data science, while others thrive in the R&D data science arena. If you like one you probably won't like the other.
It may not necessarily be the best option for the OP's daughter, but it is possible to do a PhD that uses data science as a tool rather than the focus. There's a lot of interesting work going on in academia using data science in a variety of fields (like medicine) related to predictive analytics (predicting hospital readmission, predicting therapy outcomes), time series analysis (analyzing EKGs, and other vital sign data) etc. A masters would likely allow you to work on similar projects, but if the goal is a PhD, there are certainly application rather than theory focused research groups across the country
I didn't say necessarily a PhD in machine learning. I think, generally speaking, that a technical PhD vs an MBA could make for a better data scientist.
This said, I don't have an MBA, and most of the MBA people I know are not technical at all or try to stay away from the "technicalities". There's for sure a huge need of technically and business savvy people.
I'd recommend she become familiar with Python/R, go work in the data-science/tech industry for a few years, and then head back for an MBA, which might even be sponsored by her company.
IMO, an MBA proves most powerful when backed by real-world experience.
Thanks, that was my intuition as well. I'd actually suggested doing the Stats BS with a couple of intro programming courses, then a data science boot camp, then a couple years as a data scientist before doing the MBA.
The reason for the MBA was so she had a sense of the business value of the analyses she was performing, and, was more 'promotable' to executive positions.
I sat in on some MBA and masters level business analytics courses. My junior-level math stats course went deeper than the business analytics courses go. They had second semester masters students who didn't know how to interpret the coefficients for a linear regression.
MBA courses for data science will bore her to tears. If she adds the R and Python and takes a couple graduate level math stats courses before she graduates, she'll know more than almost all the graduates from those programs
I'm a data scientist with a stats background. I'm also a self-taught programmer (going on 15 yrs of programming in various languages). There are very few data science jobs that I am aware of where somebody will hand you a tidy data set and ask you to start creating models. For those jobs, it's great to go straight from stats/math to data science, as long as you learn a bit of R and some SAS along the way. Most data science jobs that I know of involve a lot of data engineering. That is to say, you need to understand where the data originates, how to get it through an API or via other scripts, how to design and implement a reasonable (often relational) model of it and put it into a database, how to clean it, how to query it into nice and clean output files, and then how to model it. You also need to know how to visualize it effectively and use it to tell a business story.
I would not be able to do my job at all if I didn't know Python, R, and JavaScript well (and know my way around various Linux flavors). The modeling is fun, but it comes at the end of a long pipeline requiring a lot of skills that are more engineering oriented.
Rarely do my tasks sound like "model this weekly and give me the result."
Often, they sound like, "I need you to pull together data from these 5 sources, model it, and produce a weekly report showing these derived KPIs. And I need to be able to access it in a web browser so that I can send links to colleagues. And it needs to be secure. And generating a report across an arbitrary date range needs to take less than a minute."
By the way, that is a request that I've gotten at three different jobs. To give you a sense of what this looks like: most recently, I wrote Python scripts to harvest and ETL the data into a Postgres database (running on Google Cloud) and a BigQuery table, then wrote a Flask app to accept the arbitrary report requests and query the database, run models, store computed results in a local SQLite database for fast future retrieval... finally producing dynamic Reveal.js slide decks available through the Flask app.
That's a long-winded way of saying that I strongly suggest that your daughter get some practical experience with the data engineering side of data science, preferably using Python for fetching, manipulating, storing, cleaning, and preparing data. It's the most flexible tool for the job and it easily the most important tool in my data science toolkit.
Replying to myself here just to throw out an idea for an easy project that is representative of the type of work a data scientist might be expected to do.
Register as a Twitter dev (free) and get the tokens, etc. needed to access the public API. Pick a topic of interest -- hockey, for example, or whatever floats your boat -- and write scripts to harvest all tweets coming off of the public API related to the topic. Design a relational schema for the tweets and push them into a SQLite database. I suggest SQLite because it's ubiquitous and has a low barrier to entry. Something like MongoDB also works well for dumping everything straight off of the API. You can then have another script pull out of MongoDB and push into SQLite, for example. Once you're collecting tweets, storing them, etc... try some unsupervised clustering; k-means, for example, or a decision tree. If you feel up to it, go through 1000 or so of the tweets manually and label them according to some target variable of interest to the project. Then, use that labeled data set to run supervised models. Maybe start with a binary target variable and run a simple logistic regression. Then, visualize the data. There are a lot of ways to go about this, but I suggest trying to use something JavaScript based, such as D3 or p5.js, since it allows you to create interactive web-based visuals. Create a public GitHub repo and push work to it as you progress through the project. When done, use GitHub pages to put a summary of the project online.
Twitter data is great because there are tons of variables. It's horrifying because it's like reading the refuse of language, littered with abbreviations, emoji, and other weird characters. However, it's the terrible part of it that makes it great for learning.
There are other similar public APIs that would accommodate similar projects. Having a few self-initiated projects similar to this under your belt will really help you when it comes time to apply to graduate school or to jobs. If nothing else, it will give you something to talk about in interviews.
As mentioned separately, I suggested a data science boot camp after getting the Stats BS, work a couple of years as a data scientist, then go back for the MBA. My thinking RE the MBA was to give a sense of the business value of the analyses she's performing, and, to make it easier to promote her to executive positions.
Maybe that's old school thinking, I know that the MBA in general gets a mixed reception these days, but we can look more closely at it after she's out of school.
To me, an MBA is more about business strategy. Knowing the relevance of data to the operations of a business is vital to being an effective data scientist. Knowing when to make the call to lay off 500 people is the job of somebody with an MBA. Data scientists prepare and present the evidence in the language of business. There is crossover, of course.
What many data scientists (myself included) find is that they often are excluded from meetings and conversations that provide the context for the analysis they are doing. I always tell my supervisors that it is very helpful for me to sit in on as many business strategy meetings as possible, just to listen, because it builds context around the work that I do and helps keep me properly focused.
On the flip side, many involved in business strategy do not understand the nuances of analysis performed to support their business questions. There can be many reasons for that, from not being directly involved in the analysis to being excluded from data science meetings. Effective business analysis and data science requires trust between the players and that trust is built through showing an ability to deliver focused, relevant, and accurate results that support decision making.
As somebody who likes to write code and dislikes sitting in tons of meetings, I tend to avoid climbing the career ladder to management positions. I'm happy in a Senior Data Scientist position. At my last job, I was being groomed for management and I never got to do any actual data science work. It was boring as hell. I made a lateral move to a different company so that I could be more hands on and work in an industry that is more fun. Now, I get to write code and build and run models every day. I also get to present the results and have a trusting relationship with the managers.
As your daughter finishes school and gets some job experience, don't be too quick to suggest routes leading to business strategy and management. While it's the "top of the career ladder" at many companies, so to speak, it's not for everybody.
I've worked with dozens of data scientists and only one had an MBA. He didn't think it helped him get or do his job, and considered it a poor investment.
MS's and PhD's in stats, applied math, physics, computational bio/chem were far more common.
Not actual data scientist but I have a classmate with MBA. We both were in CS in undergrad.
She turned out she didn't like managing people and politics so yeah...
If you want data science stat and comp sci is the way to go imo. I'm bias cause I'm doing stat for master now.
There are some stat classes in MBA and they do prediction and stuff but your daughter may not like it. I certainly don't, they do power point and excel and visualization. Their regression classes ignore checking if their model's assumptions are valid are not (QQplot, residual, etc...) it's very dumb down.
>. A big thing about the transition to tech is that you possibly start communicating with people who don’t really know what a vector space is. Be ready to have those conversations. Honestly ask yourself if you’re OK with having those conversations.
This is so snobbish. What here is even think about ?
It's not snobbish at all. The CS/developer version is "someone who doesn't know what static typing is," or, maybe even "someone who doesn't know what computers are and are not capable of."
This is the wrong comparison. Read the second footnote. An intern is a developer in training. The post is talking about mathematicians (who spent a decade or more learning a highly technical field) going to work in an environment where, largely, their peers, boss and consumers of their work do not know, have no interest in and and will never learn the most basic concepts of their field.
When you get a PhD, you mainly only take classes in your department for the whole time, whereas in undergrad you still have classes in other fields. This usually extends beyond coworkers into friend circles too. This isn't math specific, but more generally to grad school.
Second, programmers at companies almost never work in isolation from other programmers, and in most open office environments are close enough they could touch another one from their desk. On the other hand, a startup with five to ten engineers may be hiring you as their first data scientist, and bigger companies may be putting you on an embedded team, with the nearest data scientist a hallway or floor away. And this isn't a big deal in terms of teamwork, but most data scientists don't have a PhD in mathematics, so if you find higher level mathematical ideas and notation to be the most efficient way for you to think about a problem, your colleagues may not. That's also not a math specific thing though -- an economist, statistician, mathematician, computer scientist, physicist, etc. are all going to have slightly different ways they think about things internally.
His audience is academics - who up until this point have been mostly in and around other mathematicians. Its important to ask yourself, "are you OK with having to sell/explain your work to people who aren't versed in even the basics of what you do?"
Obviously that isn't all data science jobs, but in my experience most of my colleagues have to evangelize their work within their organization to some level.
With just about any kind of speciality, you spend time developing your specialized knowledge and you learn to speak the language surrounding it as part of a facility for concisely navigating (hopefully) precisely in (potentially) complex web of ideas.
When you interface with non-specialists, you can't rely on the shared specialized vocabulary. This means you have to do a lot of work to distill which insights are key and figure out how to communicate them (along with supporting ideas) to an audience whose highest insight resolution is going to be varying and who may each bring their own language to the table.
So on top of building your problem domain model, you're going to have to build a model of how the people you're working with can understand it. And you will have to do this over and over.
You can be snobby about that, and say that it's soooo hard being so much smarter in your specialized field than non-specialists are, but you don't have to be snobby to realize that this is a dimension of the work that you might not enjoy or may even not be cut out for.
And of course, you don't have to be a mathematician to have experienced this. It's certainly sufficient, but not necessary. If you've been a developer in a company that has non-dev coworkers and you've never hit this turbulent boundary, you've either been remarkably well insulated, or you're so remarkably natural at that kind of job that you definitely should be doing it. :)
I was a PhD student in mathematical logic, and I know a couple people who got PhDs in this area and became data scientists. One thing about this field (and other areas of math) is that almost everyone in it feels mediocre. It's like 10% of the people are 10x better than the other 90%, 1% are 10x better than the next 9%, .1% are 10x better than the next .9%, etc. The top two people in the field seem to be notably better than everyone else. Another thing is that the problems are incredibly arcane. They have no connection to practical concerns or even often other parts of math, and also you can't really explain what you are working on to someone who works in a different area of mathematical logic (unless maybe the person is extremely good). So you have a lot of people getting PhDs who are very, very smart by any normal measure but who would be doing mediocre and arcane work in mathematical logic. Part of the appeal of working in data science or working for the NSA or something is that it is somewhat down-to-earth and people will think you are really smart even if you are at the bottom of the top .01%. (And you can do really great work at this level -- I don't mean to say it is all about caring what other people think.) It is a little weird that the blog post is so negative in a way.
A lot of these "arcane" topics have long-term horizons. Agree with everything else but the whole point of research is to probe the space of possibilities (theoretical and practical) to find major ideas. Most of them fail but that's part of the exploratory process - it's expected.
It's expected, but as someone doing research it's super discouraging. I can certainly see the appeal of moving away of basic research with a very long term time horizon (fine for science, but frustrating for individuals) and towards a field with the potential of more immediate impact.
Your "90/10 all the way down" explanation is a really good one that I may borrow. I often tell people new to the field that because there's more in the overall field of data science that anyone can know, everyone feels like they're deficient from time to time since an aspect you don't understand but someone else does will naturally come up. It can be useful when that happens to keep in mind the subjects that you do know well and others don't to help stave off the impostor syndrome that can crop up.
I think you got the parent's meaning exactly backward. The 90/10 thing is about their phd field of study (mathematical logic), and they're saying that something almost opposite of that is true for data science.
With data science there is a lot more demand for skilled people who are not at the top of the game. As a result there are a number of really good career paths, which really isn't what you see in academic work.
Industry can cheerfully, usefully absorb thousands of solidly "average" (in this particular sense) mathematicians (or similar) in a way that academia just has no plan for. Even if they do not work on anything quite as technically interesting as they had previously in other ways the job may be more rewarding; And I don't mean simply financially.
That being said, most academics are not a good fit initially.
I'm from an engineering background, going in the direction of data scientist. Sometimes I find that my math skills could be stronger, and I try to read up on things when I encounter them, but still it sometimes feels like there is an infinite amount to learn. Maybe I could use some more systematic approach to it. Anyone else who has walked this path, and could come with some useful advice/resources?
Yes, I am also interested in going the self study route with MOOCs but there is way too much information and resources that it is hard to figure out how/where to start. It would definitely be helpful if you could provide recommendations on books.
I got a bachelor in CS and went industry for 6 years. I'm coming back for master in applied statistic for data science.
Just take statistic, especially Multivariate for big data (big data for statisticians is huge predictors not petabyte of observations).
Math people can do so much in data science. I think statistic is better for a non math person and it's much better suited for data. Since statistic is all about data.
On the linear algebra front, you get some understanding from your first course, but the more you internalize it by meeting the ideas in different contexts, the more useful it will be. A decent amount of higher math is turning things into almost-linear problems, and then trying to sort out the parts which don't quite fit...
There is certainly a lot out there. You don't necessarily need to forge you're own path from scratch though. For a general roadmap you can use what has been established as standard by a institution with significant experience in this area. They've put in the hard work over many years to figure out what's useful: https://math.mit.edu/academics/undergrad/major/course18c.php
Check out the syllabuses and self-study at your own pace
Slightly off topic: They still can't build proper software. Academics (including mathematicians) are notoriously bad at writhing production grade software. This leads to handovers of 'proof of concepts' to seasoned software developer team who than struggle with the (often complex) mathematics/science behind it. Imho universities should give a bit more attention on how to write quality software; a bit of test driven development and continuous integration is not that hard and would massively improve the quality of the software written by scientists.
As a developer, I assure you that even some developers can't write production quality software. Especially some enterprise developers. They either over engineer and never ship or under engineer and ship crap.
Then again, those are things that can easily be thought. The harder parts of programming such as choosing the right abstraction, or coming up with the right approach for solving a problem, or even formulating a problem, are more of mathematical nature.
This is precisely why I dread hiring academic-only profile. Note: I am myself from academia, but I was lucky enough to specialized in CS-related field (NLP). When I wanted to leave academia, Data Science was not a thing where I live and I had to start again at a junior dev position (i.e.: it was that, starving or staying in academia).
Now, I work with a lot of people way smarter than I am, who are mostly useless because they can hardly prototype their stuff in Python or run an SQL query. And they'd expect to only work on the best, cleaned and formatted dataset and only do high-end maths on those. Reality hits hard, we're losing money paying them and they're wasting their time not doing what they like. Add to that the frustration / jealousy that this creates.
In that regard, I like that famous definition for Data Scientist: "A programmer that know more about statistics than most programmers, or a statistician that knows more about programming than most statisticians".
And don't get me started on the general repulsion for understanding the basics of how a business runs from academia. Data Science is all about application.
I've always heard the definition of data scientist as "Somebody who knows less about programming that programmers and less about statistics than statisticians"
Reminds me of my first job at a world leading RnD organisation when I took the test code that the lead engineer on a fluids mixing project had written and added some sensible prompts.
His original Code was cli program who's sole prompt was ?
You had to enter integers , 1, 2 3 etc to select the next option - the possibilities for errors where immense - by this time we had scaled to 1:1 tests where the chemicals for a run could cots over 10K£ per run
> They still can't build proper software. Academics (including mathematicians) are notoriously bad at writhing production grade software. This leads to handovers of 'proof of concepts' to seasoned software developer team who than struggle with the (often complex) mathematics/science behind it.
So you want employees who can understand complex mathematics and science but are also good software engineers.
Those people exist, but you have to pay to get them.
(As an aside, lots of Ph.D.'s -- especially in CS -- build systems that are as good or better than a lot of industry code. All of the best and worst code I've read has come from Academia.)
My theory of bell curves. Academia accepts a much wider distribution so that it can get some few at the far right. Business hates the variation as it wreaks havoc on building your brand/growth etc and wants the average just to the right of profitable.
One side of this that bugs the crap out of me: I have a PhD in math and, after getting sick of teaching mediocre students with a palpable aversion to mathematics, I spent a couple of years applying for data science jobs and getting very few responses. The few interviews I got seemed to go well, but I had no offers and gave up on leaving academics. I have experience in software development (C++), I have a portfolio of statistics/data analysis projects in R/RStudio, but what I think is toxic is having spent so many years in academics; I finished my Phd in 2003. Being an over-40 academic feels like an insurmountable liability, even though I have many of the skills that people complain about being in short supply. And another thing: get off my lawn, you rowdy kids! :|
I'd suggest figuring out which niche of data science most aligns with your personal interests and start there. Hiring managers don't discriminate based on "too many years in academia", it's more what's the reason they should hire YOU over candidate y?
Diehard animal lover? I bet the Sierra Club would love a statistician who could help them quantify how many elephants are being poached in Africa per year.
Interested in sports? Strava gets millions of data points a day, I'm sure they're looking for help finding signals in the noise.
Can't get enough of financial markets? Finance jobs all over the place for quants.
If your profile is too generic they'll see no good reason to hire you, but if you're passionate about X it can really work in your favor.
The job search is depressing but never give up hope.
As an over 40yr old, yes it's probably too late to switch. After 30 it was a shock to me getting turned down for jobs when I aced the technical interviews.
You might be more expensive than some of the youngins too. Just a possibility. I'm sure there could have been something else but new kids on the block often take less than they can.
You can be turned down for a job for a lot of reasons, personality fit is probably the No. 1 reason people are passed on (no data here, just my own anecdotal experience). I wouldn't assume it's ageism.
But that brings me back to my original point, you have to find a place where your personality and interests are a match, not just RandomJob.com.
If you have a higher degree in math, there is always another job out there waiting for you.
There definitely does seem to be a bias against academia in industry. From my experience applying to jobs, most employers seem to value 1 year in industry more than 5 years in academia. The data science field is flooded with academic applicants right now, don't believe the rumors. But it can happen, I know of physics postdocs over 40 who transitioned to industry, and grad students do it fairly often.
There might be a cost issue. Employers will see the PhD as raising salary expectations without adding value. The only real way to make inroads is to get a job related to your PhD and then do coding work.
Typical interview for higher jobs aren't focused on tests or quizzes. If that's what your getting your not applying to the right places - and there might not be many places.
> There might be a cost issue. Employers will see the PhD as raising salary expectations without adding value.
It could be true in some fields (biotech and life sciences, pharma/chemistry, "real" engineering, ...) but in math/CS?
Most companies hire PhD grads at one job grade/level above entry-level, i.e. it's typically not worth much more than a ~2 years head start salary-wise. If a candidate has spent 4/5 years gaining experience in the specific skill set you're hiring for, that's a bargain.
Phds might be relatively rare and so advantaged. But maybe not enough to warrant a second Phd if the first one is unrelated to the specific positions. I would add that to the GP otherwise.
This post mostly describes me. I am a PhD student in applied mathematics and returned school to get my PhD precisely because I wanted to get a job outside of academics, most likely as a data scientist or scientific programmer. 2.5 years into my program I still struggle to see beyond the degree but am currently looking at non-academic internship opportunities as a way to start to untangle myself from academics. I also spend a significant amount of my time trying to teach my self good programming practices that I don't get from mathematics (how to write clean, adaptable code, choosing appropriate design patterns and writing generic code without over-abstracting, writing readable documentation and using version control, etc.) To those of you who have left academics do you have any other suggestions? How about for marketing myself as a programming mathematician when competing against CS and engineering students for internships/jobs?
I remember asking why i was rejected from Palantir and vaguely recall the phrase "too mathematical".
After years of trying I have been unable to make the transition from Mathematics into Data Science and have since shifted my attention to other things.
I interview probably two to three people a week for a DS job. It's a really difficult role to hire for. I'm basically looking for mathematicians with some knowledge of stats who also enjoy coding and follow best practices in coding (i.e. They didn't just pick it up and hack something that works but care enough to document and structure so others can follow it at a minimum) but we also need the MBA component as well, a large part of this industry is how do you take what a DS does and deploy it in a business and ensure the business derives value from it. But I think coming at it from a mathematical background puts you in a much stronger footing than the cs guys who try to learn the math later. I rarely find strong candidates without a formal mathematical background.
Interesting, thanks. It obviously depends a lot on the job but I would have thought that a good math background (which typically CS grads should have) would be enough for many applications. Especially since you mention the business problem aspect. I think that you can actually get quite far in DS (at least in deep learning) with rather "basic math". Unless you're innovating at the bleeding edge it's usually enough to reproduce existing ideas and applying them to interesting problems is the real key.
So my hiring would have focused on CS grads with a natural interest in math, ideally very rigorous/meticulous.
It makes me almost feel sad for the people that followed the "why bother doing a CS degree, you can learn to code without one" trend that probably peaked and is now maybe beginning to die a slow death as the market is flooded with people who can write code.
If they ever want to transition into DS they're going to have to skill up and do the equivalent of undergrad CS math curriculum (discrete, calculus, linear algebra, math stats, etc.), which let's be honest, you cannot pick this up in any meaningful way in a "few months" of after hours/weekend study like you can when learning to program or learning some new "framework"; either that or just remain another dime-a-dozen code-cutter. It's sad because if you just did the 4 years of computer science and all the math and the stats that go with it you'd be so close to pivoting right now if you wanted to.
> you cannot pick this up in any meaningful way in a "few months" of after hours/weekend study
You can't pick up coding like this either. See Peter Norvig's famous "Teach yourself programming in 10 years" article. The delta in the wisdom you obtain, between a few side projects over months and battle hardened experience with real products and code bases over years, is immense.
Exactly. This is one place where the "good school" heuristic is actually a good one -- those programs almost uniformly demand some mathematical maturity of their students.
I've known many CS undergrads who are better mathematicians than people with graduate Mathematics training.
Your comment makes me feel better about myself. I've got a BS in math, some programming chops in python/swift/objc/c/sql/etc, and I quit an MA in economics.
I think doing well on Kaggle competitions would already be a good enough indicator for smart companies so it might be a good idea to have student teams do those on the side. Might be a decent idea to inform students that these exist. I feel like this applies even more for other degrees like econ(ometrics) (since I feel like a good math grad should not have trouble getting any data science job).
There is positive correlation between applied math and programming. I see expertise in these areas coupled with a knack of getting a point across as a potential career opportunity. No wonder academics in some parts of Asia have, for generations, seen math as “evergreen.”
If you see programming as a tool for solving domain-specific problems, then domain experts who can exploit the tool obviously have an edge over programmers who rely on inputs from these experts when developing domain-specific applications.
As a mathematician who successfully made the leap, I think the biggest requirement is ability to swallow your pride and "lower" yourself to seriously study professional coding.
There's too much temptation to hack something in Python or whatever, google as you go, etc. What I did was sit down and practically memorize entire programming manuals. Many academics would refuse to do something so plebeian.
An earnest question I've heard from people trying to switch is: what's a good benchmark for testing where you stand in terms of coding skills? How do you calibrate and measure your production-quality coding skills in Scala, C, etc. when you have spent all your life in academia?
I feel Kaggle is sufficient for the exploratory part of data science. But Kaggle's relevance to testing and honing "production quality" code-writing skills is sometimes minimal.
In the larger context of programming and software engineering, scientific programming is fairly easy to code-up (though they are harder to conceptualize and understand mathematically). The coding part of non-CS/non-CE/non-some-parts-of-EE academia is pretty much mostly scientific programming. Yet, production quality code is seldom just scientific programming.
121 comments
[ 1792 ms ] story [ 3120 ms ] threadWhy is it evil? You have a bunch of people in corporate america who are FORCING an information technology posture on the American people that the American people HATE because it is HORRIBLE and EVIL. Next, these EVIL people go around saying what they are doing is "data science" and start blaming mathematicians.
This is absolutely 100% evil. Mathematicians have zero interest in this. Pointing the finger at mathematicians and calling them "data scientists" is intolerable stupidity and the people who do it should expect to get their fingers cut off by mathematicians who don't like to be pointed at by IDIOTS.
Translation: Can you be a good assembly line worker, devoid of pride or curiosity and just do the damned job? If so, great, we can use you! If not, please return to academia.
Lovely stuff.
I mean, by all means open up a little boutique data science consultancy that charges three orders of magnitude more for a solution that can't be updated by anyone without a ph.d and only beats an out of the box svm by .1%, where you put your pride and curiosity (your vanity) first. You might even find some suckers to patronize you, but man... I must be misunderstanding you because that is the most clueless and entitled thing I've read in days.
98% of the time data science customers actually just want some basic statistical insights into their data to make better informed decisions. If you aren't willing to help with that, and also can't find a research group to take you in (to work on their projects) and also can't bootstrap your own startup, then yes, stay in academia.
At best you're talking about a career change, at worst you're talking about a watered-down version of your intended career.
The money is good though, that's true.
Not so much. "Wants" as "Thinks it needs", but even then, the amount of student debt suggests that neither of those are entirely true either.
What matters is not how elegant your solution is, what matters is solving the problem. And doing it under cost and time constraints.
An engineer is first and foremost, a person focussed on the end goal, above all else.
You've never met a number theorist.
And you've never met a cryptographer ;-)
That is - things that rather solve a concrete problem (given all constraints, including: dirty data, finite time, understanding the needs of clients) rather that have a pure and beautiful, but utterly useless for practical purposes, theorem to prove.
But sure, tastes do vary.
This said, I don't have an MBA, and most of the MBA people I know are not technical at all or try to stay away from the "technicalities". There's for sure a huge need of technically and business savvy people.
IMO, an MBA proves most powerful when backed by real-world experience.
The reason for the MBA was so she had a sense of the business value of the analyses she was performing, and, was more 'promotable' to executive positions.
MBA courses for data science will bore her to tears. If she adds the R and Python and takes a couple graduate level math stats courses before she graduates, she'll know more than almost all the graduates from those programs
I would not be able to do my job at all if I didn't know Python, R, and JavaScript well (and know my way around various Linux flavors). The modeling is fun, but it comes at the end of a long pipeline requiring a lot of skills that are more engineering oriented.
Rarely do my tasks sound like "model this weekly and give me the result."
Often, they sound like, "I need you to pull together data from these 5 sources, model it, and produce a weekly report showing these derived KPIs. And I need to be able to access it in a web browser so that I can send links to colleagues. And it needs to be secure. And generating a report across an arbitrary date range needs to take less than a minute."
By the way, that is a request that I've gotten at three different jobs. To give you a sense of what this looks like: most recently, I wrote Python scripts to harvest and ETL the data into a Postgres database (running on Google Cloud) and a BigQuery table, then wrote a Flask app to accept the arbitrary report requests and query the database, run models, store computed results in a local SQLite database for fast future retrieval... finally producing dynamic Reveal.js slide decks available through the Flask app.
That's a long-winded way of saying that I strongly suggest that your daughter get some practical experience with the data engineering side of data science, preferably using Python for fetching, manipulating, storing, cleaning, and preparing data. It's the most flexible tool for the job and it easily the most important tool in my data science toolkit.
Register as a Twitter dev (free) and get the tokens, etc. needed to access the public API. Pick a topic of interest -- hockey, for example, or whatever floats your boat -- and write scripts to harvest all tweets coming off of the public API related to the topic. Design a relational schema for the tweets and push them into a SQLite database. I suggest SQLite because it's ubiquitous and has a low barrier to entry. Something like MongoDB also works well for dumping everything straight off of the API. You can then have another script pull out of MongoDB and push into SQLite, for example. Once you're collecting tweets, storing them, etc... try some unsupervised clustering; k-means, for example, or a decision tree. If you feel up to it, go through 1000 or so of the tweets manually and label them according to some target variable of interest to the project. Then, use that labeled data set to run supervised models. Maybe start with a binary target variable and run a simple logistic regression. Then, visualize the data. There are a lot of ways to go about this, but I suggest trying to use something JavaScript based, such as D3 or p5.js, since it allows you to create interactive web-based visuals. Create a public GitHub repo and push work to it as you progress through the project. When done, use GitHub pages to put a summary of the project online.
Twitter data is great because there are tons of variables. It's horrifying because it's like reading the refuse of language, littered with abbreviations, emoji, and other weird characters. However, it's the terrible part of it that makes it great for learning.
There are other similar public APIs that would accommodate similar projects. Having a few self-initiated projects similar to this under your belt will really help you when it comes time to apply to graduate school or to jobs. If nothing else, it will give you something to talk about in interviews.
As mentioned separately, I suggested a data science boot camp after getting the Stats BS, work a couple of years as a data scientist, then go back for the MBA. My thinking RE the MBA was to give a sense of the business value of the analyses she's performing, and, to make it easier to promote her to executive positions.
Maybe that's old school thinking, I know that the MBA in general gets a mixed reception these days, but we can look more closely at it after she's out of school.
What many data scientists (myself included) find is that they often are excluded from meetings and conversations that provide the context for the analysis they are doing. I always tell my supervisors that it is very helpful for me to sit in on as many business strategy meetings as possible, just to listen, because it builds context around the work that I do and helps keep me properly focused.
On the flip side, many involved in business strategy do not understand the nuances of analysis performed to support their business questions. There can be many reasons for that, from not being directly involved in the analysis to being excluded from data science meetings. Effective business analysis and data science requires trust between the players and that trust is built through showing an ability to deliver focused, relevant, and accurate results that support decision making.
As somebody who likes to write code and dislikes sitting in tons of meetings, I tend to avoid climbing the career ladder to management positions. I'm happy in a Senior Data Scientist position. At my last job, I was being groomed for management and I never got to do any actual data science work. It was boring as hell. I made a lateral move to a different company so that I could be more hands on and work in an industry that is more fun. Now, I get to write code and build and run models every day. I also get to present the results and have a trusting relationship with the managers.
As your daughter finishes school and gets some job experience, don't be too quick to suggest routes leading to business strategy and management. While it's the "top of the career ladder" at many companies, so to speak, it's not for everybody.
MS's and PhD's in stats, applied math, physics, computational bio/chem were far more common.
She turned out she didn't like managing people and politics so yeah...
If you want data science stat and comp sci is the way to go imo. I'm bias cause I'm doing stat for master now.
There are some stat classes in MBA and they do prediction and stuff but your daughter may not like it. I certainly don't, they do power point and excel and visualization. Their regression classes ignore checking if their model's assumptions are valid are not (QQplot, residual, etc...) it's very dumb down.
This is so snobbish. What here is even think about ?
The startups and Googles of this world are the exception when it comes to employer tech-savviness.
Second, programmers at companies almost never work in isolation from other programmers, and in most open office environments are close enough they could touch another one from their desk. On the other hand, a startup with five to ten engineers may be hiring you as their first data scientist, and bigger companies may be putting you on an embedded team, with the nearest data scientist a hallway or floor away. And this isn't a big deal in terms of teamwork, but most data scientists don't have a PhD in mathematics, so if you find higher level mathematical ideas and notation to be the most efficient way for you to think about a problem, your colleagues may not. That's also not a math specific thing though -- an economist, statistician, mathematician, computer scientist, physicist, etc. are all going to have slightly different ways they think about things internally.
His audience is academics - who up until this point have been mostly in and around other mathematicians. Its important to ask yourself, "are you OK with having to sell/explain your work to people who aren't versed in even the basics of what you do?"
Obviously that isn't all data science jobs, but in my experience most of my colleagues have to evangelize their work within their organization to some level.
When you interface with non-specialists, you can't rely on the shared specialized vocabulary. This means you have to do a lot of work to distill which insights are key and figure out how to communicate them (along with supporting ideas) to an audience whose highest insight resolution is going to be varying and who may each bring their own language to the table.
So on top of building your problem domain model, you're going to have to build a model of how the people you're working with can understand it. And you will have to do this over and over.
You can be snobby about that, and say that it's soooo hard being so much smarter in your specialized field than non-specialists are, but you don't have to be snobby to realize that this is a dimension of the work that you might not enjoy or may even not be cut out for.
And of course, you don't have to be a mathematician to have experienced this. It's certainly sufficient, but not necessary. If you've been a developer in a company that has non-dev coworkers and you've never hit this turbulent boundary, you've either been remarkably well insulated, or you're so remarkably natural at that kind of job that you definitely should be doing it. :)
- why: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html (on academia vs industry)
- how: http://p.migdal.pl/2016/03/15/data-science-intro-for-math-ph... (also got reprinted at KDnuggets)
Industry can cheerfully, usefully absorb thousands of solidly "average" (in this particular sense) mathematicians (or similar) in a way that academia just has no plan for. Even if they do not work on anything quite as technically interesting as they had previously in other ways the job may be more rewarding; And I don't mean simply financially.
That being said, most academics are not a good fit initially.
That's very different.
No one can keep up with the flood of JS frameworks, either.
Information fire hoses are very different from extreme differences in ability.
I'm from an engineering background, going in the direction of data scientist. Sometimes I find that my math skills could be stronger, and I try to read up on things when I encounter them, but still it sometimes feels like there is an infinite amount to learn. Maybe I could use some more systematic approach to it. Anyone else who has walked this path, and could come with some useful advice/resources?
I can give recommendations on books for the former but not the latter.
Just take statistic, especially Multivariate for big data (big data for statisticians is huge predictors not petabyte of observations).
Math people can do so much in data science. I think statistic is better for a non math person and it's much better suited for data. Since statistic is all about data.
On the linear algebra front, you get some understanding from your first course, but the more you internalize it by meeting the ideas in different contexts, the more useful it will be. A decent amount of higher math is turning things into almost-linear problems, and then trying to sort out the parts which don't quite fit...
Check out the syllabuses and self-study at your own pace
Now, I work with a lot of people way smarter than I am, who are mostly useless because they can hardly prototype their stuff in Python or run an SQL query. And they'd expect to only work on the best, cleaned and formatted dataset and only do high-end maths on those. Reality hits hard, we're losing money paying them and they're wasting their time not doing what they like. Add to that the frustration / jealousy that this creates.
In that regard, I like that famous definition for Data Scientist: "A programmer that know more about statistics than most programmers, or a statistician that knows more about programming than most statisticians".
And don't get me started on the general repulsion for understanding the basics of how a business runs from academia. Data Science is all about application.
His original Code was cli program who's sole prompt was ?
You had to enter integers , 1, 2 3 etc to select the next option - the possibilities for errors where immense - by this time we had scaled to 1:1 tests where the chemicals for a run could cots over 10K£ per run
So you want employees who can understand complex mathematics and science but are also good software engineers.
Those people exist, but you have to pay to get them.
(As an aside, lots of Ph.D.'s -- especially in CS -- build systems that are as good or better than a lot of industry code. All of the best and worst code I've read has come from Academia.)
I'd suggest figuring out which niche of data science most aligns with your personal interests and start there. Hiring managers don't discriminate based on "too many years in academia", it's more what's the reason they should hire YOU over candidate y?
Diehard animal lover? I bet the Sierra Club would love a statistician who could help them quantify how many elephants are being poached in Africa per year.
Interested in sports? Strava gets millions of data points a day, I'm sure they're looking for help finding signals in the noise.
Can't get enough of financial markets? Finance jobs all over the place for quants.
If your profile is too generic they'll see no good reason to hire you, but if you're passionate about X it can really work in your favor.
The job search is depressing but never give up hope.
But that brings me back to my original point, you have to find a place where your personality and interests are a match, not just RandomJob.com.
If you have a higher degree in math, there is always another job out there waiting for you.
Typical interview for higher jobs aren't focused on tests or quizzes. If that's what your getting your not applying to the right places - and there might not be many places.
It could be true in some fields (biotech and life sciences, pharma/chemistry, "real" engineering, ...) but in math/CS?
Most companies hire PhD grads at one job grade/level above entry-level, i.e. it's typically not worth much more than a ~2 years head start salary-wise. If a candidate has spent 4/5 years gaining experience in the specific skill set you're hiring for, that's a bargain.
After years of trying I have been unable to make the transition from Mathematics into Data Science and have since shifted my attention to other things.
If they ever want to transition into DS they're going to have to skill up and do the equivalent of undergrad CS math curriculum (discrete, calculus, linear algebra, math stats, etc.), which let's be honest, you cannot pick this up in any meaningful way in a "few months" of after hours/weekend study like you can when learning to program or learning some new "framework"; either that or just remain another dime-a-dozen code-cutter. It's sad because if you just did the 4 years of computer science and all the math and the stats that go with it you'd be so close to pivoting right now if you wanted to.
You can't pick up coding like this either. See Peter Norvig's famous "Teach yourself programming in 10 years" article. The delta in the wisdom you obtain, between a few side projects over months and battle hardened experience with real products and code bases over years, is immense.
I've known many CS undergrads who are better mathematicians than people with graduate Mathematics training.
Theory is nice. But Experience trumps all. Esp when you think it mostly boils down to PCA/DBSCAN, regression and scatter matrices.
A month of Python/R/D3 on some initial data like pubmed/twitter or any toy dataset would go a looong way.
Do you enjoy kicking the ball towards the gate and scoring? Then you might be good for Football.
There's too much temptation to hack something in Python or whatever, google as you go, etc. What I did was sit down and practically memorize entire programming manuals. Many academics would refuse to do something so plebeian.
I feel Kaggle is sufficient for the exploratory part of data science. But Kaggle's relevance to testing and honing "production quality" code-writing skills is sometimes minimal.
In the larger context of programming and software engineering, scientific programming is fairly easy to code-up (though they are harder to conceptualize and understand mathematically). The coding part of non-CS/non-CE/non-some-parts-of-EE academia is pretty much mostly scientific programming. Yet, production quality code is seldom just scientific programming.