Ask HN: I realise data science is not for me and feel depressed
The project here is interesting, but I realize that I do not like any of the "real" DS tasks, e.g. anything that involves actually building a model from the statistical standpoint. Now, I write a lot of SQL: the company has hundreds of billions of calling records, so you have to really think about the queries you write, and even if they are quite optimized I still sit here for a long time waiting for the response. And even that is more exciting than any of the statistical tasks I face
Little background: I started coding when I was 16, back then I built an iOS app using Swift to help me learn English vocabulary. After that, I've been coding as a hobby on the side during my studies. I liked mathematics and statistics as a freshman, and during my second year found out about Data Science. Given my background, I thought it would be a perfect fit for me. I built a few pet-projects using ML during my third year, and started applying. They impressed one of the employers (big bank), so these projects basically landed me my first internship, which led me to the current job
The thing I like the most is actually building stuff with my hands, and I think SWE is a better fit here. I guess this is the reason I never liked Kaggle. Also, most of my internship I spent building a simple web interface for the existing model (mostly back, like connecting to DB, transforming data, deploying to Linux, etc), which was 100% SWE task and I liked it. So there were hints along the way...
How would you recommend me to switch to SWE the fastest way possible?
I'm quite depressed now because I've put enormous effort into switching to DS: I applied to 80 job postings before I got my first internship, and I feel this is for nothing now. Also, my university and faculty considered to be really difficult, so studying DS on the side was brutal.
Besides, I live in Russia and want to relocate (for obv reasons), and feel like it's easier with SWE skills (especially given that I know English and in the process of receiving EU passport). This puts an enormous time pressure on me given how the things are playing out now...
I would appreciate any opinion/advice!
147 comments
[ 2.9 ms ] story [ 213 ms ] threadPositives for you:
- you're still at university and you already have work experience
- you haven't sunk years into a career you don't like
- you're employed and earning some money
- you're figuring out what's for you and what isn't
- you have some coding experience
It really seems like you should look for a SWE job, I don't think there's much more to it. What makes it difficult is everything that's going on in Russia and the fact that you want to move to a different country. But on every other dimension it seems like you're in a great spot.
For your next job just find a developer position.
Even if you wanted to get totally away from data most any modern system is going to need to interface with data systems at some point. Your skills would be super valuable.
But no reason to be depressed. You should be proud about the effort you put into getting this position. It's never lost. I can't speak about your particular situation in Russia, but as far as switching to SWE, it should be easy. Some people switch careers in their 40s. As a young graduate, it's very easy to change path.
First thing you should do is see if you can pick up an SWE job with your current skills. When I got my first software job (in 1996, writing test scripts in the aerospace industry) all I knew was a little bit of matlab, basic and pascal. See what's out there! There's more to software than FAANG
For most employers, just checking off the degree is more important than exactly what it is. This is especially true for software engineering. You will have many, many coworkers who have degrees in physics or bioinformatics or mathematics and work as SWEs. Most of their stories will be exactly the same as yours, although "...but there weren't any good paying jobs so..." will be in the running.
With the OP, he's 4 months away from finishing his degree. In the grand scheme of life, that's a "blink of an eye". There is no point not to, and (I don't agree with this at all) people judge you differently if you have a degree vs. not.
Employers basically hire staff to get stuff done. The rest is details about specialization, experience, skill-set, etc. All of which is rendered irrelevant if you don't get the jobs you are assigned done.
Re: a PhD -- ideally you should ask the question why you want one is before you start. Cursory research quickly reveals that academia as a career track is very challenging. You don't need a PhD to be a great SWE. A PhD is training to be a researcher and an expert in a specific field.
But I see anything above Bachelors as a personal preference (with a few exceptions). If you want to finish PhD, then go for it.
Thinking to my own experience, PhDs are pretty uncommon but of those I've worked with, some are brilliant, and some are terrible -- just like non-PhDs. I wouldn't consider a candidate with a PhD better than a non-PhD candidate, but then some people would love to hire someone with a PhD. I'd say on balance, it's a positive to have, but I would never recommend someone pursue a PhD as a route into a SWE role. A year of hobbyist programming experience is enough to get a SWE job.
You know what other degree I see a LOT of in SWE? Music. I've hired countless music majors.
I massively prefer people who flunked university and just got a job writing code because they could. University no longer trains people to think, instead it's about jumping through hoops and dealing with pointless administrative bullshit [0]. source: I have a Master's degree.
And yes, even better are the people who wrote code as a hobby, and then decided to switch career to become professional developers. Some of the best coders I know are in that category (though the best coder I ever hired taught herself to code while getting off heroin in a Glasgow slum).
[0] Ok, so yes if you're hiring for a large corporate I can see how this would appeal.
I think it's because of the salary level - like HR/management cannot get their heads around paying someone who doesn't have a degree the kind of salary that developers get these days.
I quit university after my 2nd semester co-op during undergrad (to work at my current firm), I have since been promoted four times and now work as a senior engineer (though I cannot technically use this title). Trying to move firms now after working for 5 years, It is incredibly difficult for other firms to accept that I've been working in the field successfully given that I do not have a completed degree. On one hand, I have skill now that I could have never earned in school - on another hand I may have gimped my ability to move deep into the upper half of the first six figures salary.
Completing a degree means you can commit and deliver, IMO.
Instead, leaving for writing code just because you can tells me that you took the short path, and that your missing basics (the real ones, which teaches you University) will knock at your door some day.
My 2 cents.
Not completing a degree doesn't mean you can't so this statement doesn't tell us all that much.
Then yes, I might be of course proven wrong, I just gave my 2 cents on what not finishing a degree tells me, including my bias.
Then you could be a false negative, in the sense that you might have not finished your degree but be able to commit 100% on other things. However, as we all know, spotting false negatives is very hard and who hires don't like these bets if not in very particular circumstances, as it is going to be their fault if the person is then not a false negative but a true one. With or without backup data.
I think is no coincidence that as far as I can see the OP basically got the "finish your degree" as first and foremost comment.
Don't take me wrong: I have been a false negative and an outlier my entire life, now I love it, but in the early days I really wished someone told me what I am saying now.
I'm not projecting my own experiences on others, I just take issue with your claim. Studying and working are very different beasts, I think it's very much possible for people to not finish degrees yet be really good employees that finish their tasks as agreed upon. The incentive structure is so different that the two can hardly be compared. Not finishing your degree, ignoring personal/financial issues, tells us nothing about their work performance.
> However, as we all know, spotting false negatives is very hard and who hires don't like these bets if not in very particular circumstances, as it is going to be their fault if the person is then not a false negative but a true one.
Yes that's hard but that's a very different issue from your original claim. It's a good reason for a recruiter to go for the person with the degree instead of the person without the degree, but that still doesn't mean that the person without the degree can't commit and finish their work.
> I think is no coincidence that as far as I can see the OP basically got the "finish your degree" as first and foremost comment.
He has 3-4 months left of a 5 year education, of course people will recommend him to finish his degree. There is very little opportunity cost downside.
My original claim was not an actual claim, it was just what not finishing something one starts tells me. A sensation. And by "not finishing" I do not mean trying for a bit and then quitting, that's of course OK.
What I meant is not finishing, after investing vigorous effort, because one gets "enchanted" with something else (e.g. "oh look I can code in javascript and build a website, who cares about finishing studying the basics, I don't need them anymore!").
By the term "finishing" I implicitly assumed to be near the finish line. If then this is not a symptom of poor commitment capability, I don't know what else it could be.
On another angle, the comment to which I responded originally basically said "University does not teach you anything valuable at all, just quit and go writing code" and here I strongly disagree and will always do.
What do you do when it becomes obvious that the thing you started is now irrelevant, or was poorly-conceived, even mistaken, when you started?
Genuinely curious. I've walked away from a lot of projects because I learned while doing them.
So does a lot of other valuable and still relevant lived experiences, and they come without the price tag.
But we all already know that.
However, if you start something, I think that it is a good attitude to finish it, provided that you don't face major practical/financial issues along the way.
I 100% agree with this. I've just got a bit (maybe a lot) of chagrin towards the sometimes strident mentality that college is the only meaningful way an individual can prove they possess such a capable characteristic, or that completing college is an indicator that such a person will retain such a characteristic throughout their careers/lives.
It's useful as a snapshot of an individual's educational accomplishments for sure, and more power to them for that accomplishment, but I will always have a bit of reticence about the application of "has degree" past that.
Flunking out of university is not the "short path". It's the hard path. Getting a degree is not difficult, or even hard work. It's the easy choice - the thing everyone expects you to do.
I don't want people who follow orders and do what they're told. I want people who are prepared to make hard decisions, take the less-travelled path, do what they feel is right and not just what everyone else tells them is right.
I want someone who can actually code, and not someone who has a certificate that says they can code.
My 2 cents.
I saw it many times: javascript kids or self-taught backend Python "engineers" falling extremely short when it came to scalability, algorithmic complexity, or just abstracting concepts.
Similarly, I saw several self-claimed "data scientists" not even knowing what a non gaussian error distribution means.
Not knowing this stuff jeopardise your work and the projects you are working in.
Then, if you are telling me that your interview process should spot these shortcomings at the same level a university degree can, then to me this sounds a bit unrealistic.
However, if you need someone that can write code to pass the unit-tests that some one else wrote without taking any architectural decision, than OK I can agree with you, but it does not sound much of an appealing career path IMO.
The problem is too many tech bros have been pissing on liberal arts for 30 years. And now people are looking around and saying 'we have too many tech bros'.
My college years were spent reading various viewpoints, bringing them together, and synthesizing them into a coherent narrative. This turns out to be very helpful when it comes to solving problems and analyzing security incidents. I've found it more useful than that semester of C+ the new hires took for their CompSci degree.
Are you able to elaborate on what you mean by these statements? What does being able to think mean and how would one be able to demonstrate that?
Do you think the only way to learn this is in the liberal arts?
> too many tech bros have been pissing on liberal arts for 30 years.
Your comment seems to be pissing on STEM. Am I completely misreading what you're saying? STEM and the vocational or mechanical arts have been pissed on by the liberal arts for a lot longer than the last 30 years.
Those words bust the biggest bias I realized I had/have - against addicts. Thank you very much!
"Never trust a junkie" is still very good advice.
Talent at coding doesn't automatically come with the personality or the temperament to be a good career software dev.
You can basically break a "software engineering" degree down to three components - writing code, "core theory", and "specialist knowledge".
You may already have "writing code" down well enough for entry level gigs; if not, side projects are a fine way to get you there. Look over job postings that interest you, see what languages they ask for, try writing some code in that language.
For "core theory", there are three basic classes you'd ordinarily take: algorithms/data structures, an introduction to compilers, and an introduction to operating systems.
If you've taken these courses already while in your current degree program, great, you're done. Don't worry about all of the extra specialist knowledge, you can pick it up later if you need or want it. You'll know as much as any other entry level generic software engineer.
Algorithms/data structures tells you why some code is fast and why some is slow, and the theory of fast code. Compilers teaches you how programs work under the hood, how the computer actually interprets and executes your programs. Operating systems teach you how the computer as a whole works what happens when you write to a file or a socket or a screen.
You don't need to be able to write a compiler or am operating system from scratch to be a good generic software engineer, but having a rough overview of how the entire system works is one of the things that elevates you past being "just" a self-taught coder. It helps you understand what your code is doing as well as how to write it.
If you haven't been exposed to the "core theory", you're probably best off just trying to pick up books or (free) online courses on the subject. Unless you have just fantastic financial resources you shouldn't try to change your degree or get extra formal schooling in the subject. As long as you can write code, there will be people happy to hire you as a SWE, so you don't even have to wait to master the core theory to start applying to jobs.
Start with algorithms/data structures. It's the most useful and a good opportunity to get more practice writing code in, especially in new languages.
It's the best course I've ever done tbh, and is actually interesting. I never felt like I was forcing my way through it.
Certainly someone can recommend a good resource here?
The Wizard Book maybe?
Good advice.
Listen to this.
Focus on finishing the degree, and if possible, get as good grades as possible.
That will open a lot of doors.
____
To this comment:
I have switched from Physics to Deep Learning. This is more because of "getting to know myself better" than money.
I grew up being good at math (scorewise when little, and later, actually getting and understanding) and science. And I had a lot of biographies and a-little-harder-than pop-sci kind of science articles at home. I thought that I would become a Physicist and become a Professor.
College was a big disappointed. The beaurocracy aside, the college system relies on memorizing numerical problems, proofs, and regurgitating them during test.
And also, the field of Physics is highly saturated. It feels like all the stuff has been done. Need a field where I can contribute, rather than just watch passively.
In college, for the first time, I got to work in a real Electronics lab in my Physics major class. Absolutely loved the experience.
Reading theory by bigshots in books, and applying them with my own hands! Building stuff! That's when I first found out I loved building things and applying things on my own hand rather as much as, or more than I loved the ivory tower of pure theory.
Coupled with my frustration, I decided I will become a programmer because that is where I got to build things and not just regurgitate text.
My metor-friends (older) told me that Data Science was hot. Better get into that. Although they meant analysis type work, I soon transitioned to Deep Learning. Vision particularly.
I am on brink of publishing two research papers independently, and worked in two jobs where I developed Deep Learning systems.
I am very happy that I gave up Physics.
I may not even stay here. This transition has taught me to be less stringent on life's choices.
I have also learned that I don't particularly like Physics or Math or Deep Learning. I like esoteric things that not many can do, pay well, give intellectual kicks, opportunity to work with hands, and are math-y.
Looking at someone's GPA honestly wouldn't help me much at all. It's not like I know how rigorous <insert university here>'s math or economics departments are.
Unless you went to a college that's a household name in your geographical region (for example, Harvard in the United States), I'd leave off the GPA from the resume.
Try to learn as much as you can, but don't fret about getting the best grades. Very few people will honestly care about your grades; they care that you graduated and whether you have the technical and interpersonal skills to fit on the team. If a person won't hire you because your don't have a 4.0, trust me that's not someone you want to work for in the long run.
I've been working in IT and software development since I left high school; it was never a barrier that I went to school to be an aircraft mechanic and don't have a tech-related degree.
And honestly, finding an entry-level SWE job with a degree in Mathematics + a background in data science is probably easier than finding an actual data science position.
Any time you have a situation where people are filtering large number of candidates impersonally, from hiring to immigration to dating apps, some of the time you'll be blocked for not having a college degree.
Sure, you can find plenty of start-ups or other smaller shops to apply to; maybe you'd even prefer to work for people who can see past a degree or lack thereof. If your life falls out that you can't get a degree or just have better options on the table than starting or finishing a degree-granting program, it's not the end of the world, you have plenty of options and opportunity. But it's still probably worthwhile to keep all the options you can if it's relatively little extra cost.
[0] If you're epsilon away from completion, if you've already paid for your final semester, if you're not in actual danger from staying in the program, if you don't need to get a job yesterday because you've run out of runway for living expenses, etc etc.
There is a bunch of pure SWE work in the serving part and still requires some math background that, thanks for your education, you seem to have. This should allow you to get exposure to a lot of pure SWE-backend tasks which could allow for a full transfer into SWEland later on.
However, I am not sure how common these figures are in the industry. I work for a FAANG company as an ML Eng like I described, but I aware that not all companies have such people.
Finally, I wish you the best of luck in getting an EU passport!
Maybe telecom isn't for you. Maybe you'd find happiness in doing data science for an NGO that is saving lives in Africa, or an early startup that needs their second "data and data infrastructure" person to get the company bootstrapped. Or maybe not- that's just one thing you think about.
At the same time, nothing is wasted. You've learned stuff. Hopefully, you will continue to learn stuff your whole career, your whole life. If it happens that the first major skill you learned wasn't the one most critical to your career, that's okay.
Graduate with your degree. It serves as proof that you can learn stuff and not have your life fall apart for a few years.
First: Finish your degree - a valuable and important signal to employers is that you finish things, not start and stop whenever you have a change in emotion. It's ok to feel discouraged sometimes, but it's never ok to quit what you're supposed to finish.
Second: Your data science experience is very valuable already, and so once you start learning SWE, you have a good mix of skills that many of your peers with only data science or SWE won't have.
Third: I still like Ruby on Rails for web app development, and there is still a ton of opportunities out there for employment on it. So think about learning that. To incorporate data science into web apps for production, there are a lot of options, and Rails will keep you productive with moving forward and learning more in that field.
Good luck, and congrats on your upcoming graduation!
Once you've graduated don't let the fact that your work experience is Data Science based put you off for applying for SWE positions. I've worked with a lot of software engineers whose education is in a different field, so switching over is certainly possible.
For the Company or another..
Finish your studies, and then find an employer. Do a few online courses if you feel you need to brush up on your skills.
Your degree is most valuable by existing, not being in any specific field.
There are of course position that can be quite strict from qualifications point of view, but don't let that stop you. There are many, many others.
I recently completed the transfer from BA to BI and then to a BI that does DE job.
Finish your degree, obviously... you can pivot to other jobs if you like, but once you find a real DS role it may improve.
I have a very similar experience to yours (didn't like the analytics side of Data Science) and my advice would be to look into ML engineering roles. ML engineering is heavily SWE-oriented, sort of an intersection between Data Science and Software Engineering. I switched to ML engineer and I totally love it, you get to learn tons of cool SWE stuff (Docker, kubernetes, linux, cloud, ci/cd) and you're still close to newest ML research, which I love :)
Most of the work is basics statistics, some SQL and poking at a ML library until you get the output you wanted.
Unsurprisingly, the day to day work can be very underwhelming.
95% of your value as a "data scientist" is figuring out how to squeeze diamonds from tons and tons of steaming piles of sh*t, and that kind of work is basically just filtering, collating, collecting, organizing from disparate data sets with different fields, data types, formats, some of them with varying degrees of accuracy. it's not fun, but very few other people are doing it, and the result is valuable to a lot of people, so it pays well.
I feel like data science gets called out disproportionately for something completely normal.
In my experience, smaller companies might be more for you. The job positions are often more versatile and varied. I imagine you are more likely to find a job there where you can combine SWE with your DS knowledge and still like it. In such companys your tasks typically require you to solve real business problems all the way through, for example: "We have some sales data. Can you write us a tool that helps us decide where to focus our marketing efforts?"
But firstly, as others have pointed out, since you already invested a lot of time into DS, finish your degree as fast as possible, and then take the opportunity to build a portfolio (even if it is pure DS) and a professional network to help you find a better job. I do not think that it should be too difficult to transition into SWE with a DS degree; and after a few years of professional activity, no one is interested in what you have originally studied anyway.
2) look at data engineering
Either on specific jobs or systems (map/reduce, streaming, databases etc).
Three, you have been given a gift. You know what you don't want and some things about what you do. Nothing you did was a waste. Focus less on the specific goals and be in the moment. It is great that you have industry experience, even before graduation. That is where you will do/did a whole bunch of learning. Also, don't let this specific job color your whole view of DS.
Peace
Have you considered ML Engineering? This involves integrating models into applications so that might satisfy your urge to create rather than analyze & model?
The only way you can screw up is by staying in a field long after you’ve realized it’s not for you. It’s crazy to keep doing something you don’t want to do, for your entire working career. Whether it’s from the sunk-cost fallacy or feelings of shame or just complacency, it’s a sad thing.
It takes courage to be honest with yourself and make a change.
(I’m in marketing now but got a brutally difficult degree in naval architecture. Can you believe it?)