Ask HN: How do I choose between software engineering and data science?
I'm applying for internships for summer 2019, and as the title suggests, am unsure of whether to apply for swe or data science positions. My background and skill set are mostly in statistics and machine learning, so I'm inclined to apply for data science, but many of the postings I've looked at list a PhD as a minimum requirement, and I am an undergrad. On the flip side, there are lots of swe postings looking for undergraduates, but they expect knowledge of data structures/algorithms and programming languages that I don't have (I'm not a CS major).
I think I'm a reasonably strong applicant, but am unsure how to navigate between the Scylla of having the wrong degree and Charybdis of having the wrong background. Any advice would be much appreciated!
36 comments
[ 5.1 ms ] story [ 87.2 ms ] threadIt sounds like data science is more in line with your experience and interests.
A math degree. You're confusing data engineering and data science. There are plenty of people who work on theory and do little to no programming.
Machine learning is a CS field. It emerged out of CS. Any claims to the contrary are hokey revisionism. As to what "data science" entails, that's become a super loaded buzzword, so I'm not even sure where to begin. And "data engineering," please don't even. Just fancy terms for statistics and discrete math.
Plus, I could say the same thing about ML. It's just graph theory, linear algebra and calculus with some statistics mixed in. Where's the absolutely necessary programming? There are plenty of opportunities to do ML theory with little programming, if any. There absolutely needs to be a distinction between theorists and engineers, because they aren't the same thing. Most of the programming is the grunt work you pass off to the engineers.
As you surmised, the latter is more focused on algorithms and data structures as the basis for solving problems. Your gut response is good. Go with your gut.
You're too young (I don't know your age, but young in the process) to be worried about which is perfect. Apply to both and whichever you get do it super well. Can't lose! Good luck!
Also be sure to explain in a customized cover letter why you would be a good match for each position you apply for.
What about not being a CS major prevents you from picking up Sedgewick or a programming language reference?
In my experience, it is probably easier to differentiate yourself and proof your worth by producing great products in software engineering.
Data science is overrun at the moment by everyone chasing the hype. So it is kind of hard to proof your worth by producing great data science, you just will not be heard among all the shouting of people, sub-orgs and consultants trying to sell their latest deep learning model for a data-set that would fit on a floppy disk (okay, that was exagerated, a zip drive).
Source, I have a math masters (statistics, ML).
Data Science is a weird field. A lot of the jobs descriptions have similar keywords, but there is just a huge amount of variance in what the job requires. There are definitely a large number of Data Science roles where solving a business problem requires you to write a good amount of code for integrating with other systems, data ET(maybe L), building UIs, etc that really is about making the core algorithm consumable by business owners.
When you interview ask what the day to day of somebody in that role is doing. You'll be able to figure out fairly quickly where they fall on this spectrum. Find the one that fits what you want.
IME, at smaller companies they don't have enough people to have 4 people (a Data Scientist, a Data Engineer, a SWE, and a Business Expert) just to get a data science project from conception to production. That's all done by one person with help from a business expert.
Don't get me wrong, I really liked my position and my team, and loved what I did everyday. Career-wise though, I would consider Software Engineering better, unless you plan on doing a Masters/PHD right after undergrad.
Data Science is a much younger field than Software Engineering. While there is a ton of room to grow, it also means there aren't good hiring practices in place. Companies are way more conservative about hiring Data Scientists than Software Engineers. There usually aren't the same kinds of "coding challenges" as for engineers. While that sounds like a good thing, it means that companies have to filter out candidates some other way. In most cases, (good) companies filter out candidates by looking only at applicants with a graduate degree or with >3 years of experience. This makes it a very tough field to break into without already having experience.
Money certainly isn’t everything, but I’m considering switching to software engineering (from data science) because I would like to reach financial independence more quickly than my current trajectory allows.
It's also worth noting that although every position I've applied for has asked for a PhD my one year masters has sufficed in every case.
Data Science is a useful skillset for everyone to have, but the majority of the work in any practical data science role is in getting the data in the right place, in the right format to do the data science. This makes it such that most small companies can't actually support having a full time data scientist who can't also write code.
You have a good background in stats and ML - use that with practical experience in SWE to make your skillset more useful and broadly applicable.
Where I work we prefer DS canidates that have some SE background and could comfortably deploy a model (even if it's just to heroku)
We pass on a lot of really smart DS applicants who havent had the SE xp, simply because many of them would take a lot longer to get up to speed
This is just one data point, and some DS jobs probably wont require the SE xp, but hope it helps & best of luck!
The important question is whether you're interested in what the company and specific team are doing. Example: I once interned at Google, on the Chrome team. I mean, it's GOOGLE, free food and wonderful smart people and again it is GOOGLE and I'm an UNDERGRAD! What I learned that summer was that I don't actually care about web browsers at all. And so my internship was kind of a bust just because I didn't care much about what we were doing. I had no drive to stay at the office late to keep grinding away at the problem.
What motivates you? What really interests you? Whether you're doing data science or software engineering, that will be far more important to you having a successful internship.
After it fails miserably, if you blame the model, then you should become a software engineering. If you blame the algorithm, then you should become a data scientist.
Yes, software pays more now, and data science (let alone data engineering) is still maturing and figuring itself out, especially at junior levels. Your current background will help with data science, but doing software for an internship won't hurt you in the future if you want to do data science.
Having a programming background helps with data work, some of which is programming directly and indirectly to talk to data engineers/software engineers, and vice versa, a math and data background is super helpful in attacking software problems in many areas.
If you feel like picking the wrong thing now at a young age will scar you forever, you're doing it wrong. In this whole industry, things change constantly, and you will have to reinvent yourself and learn with it. If you don't like it, you can always switch specialties, or even generalize a little more broadly.
Reference: I've been in IT for the better part of 20 years, much of that as a web development generalist, and now I'm doing data engineering. ~75% of the skills overlap.
I would also suggest to apply for a lot of them anyway, there's not enough skilled and experienced people to fill all positions right now, so some of them will recruit more junior people than they might be looking for at first.