Ask HN: How do you escape CRUD jobs?
You know the drill. When you left college, you wanted to work for the big guys - Intel, Google, Apple, NASA - doing things which can change the world. You wanted to do stuff like create a compiler or figure out an algorithm.
Unfortunately, reality hit back hard and you got stuck at a typical company doing CRUD operations. Make reports here, manage documents here, upgrade calendar software here. It pays pretty good but it's not the dream you wanted. Irony then rings and the deeper you move in your career, the more CRUD experience you get and the less "let's design the next generation algorithm" experience you have. Thus causing those companies to pigeonhole your resume to pass you on for someone else.
How do you escape from this trap?
92 comments
[ 3.8 ms ] story [ 110 ms ] threadYou don't get a job in compilers by complaining. You do it by getting commit access to gcc.
Hop to it.
Alternately, do a startup in the space you want to be in and let VCs pay for the privilege of you educating yourself. Either way.
The biggest mistake new college grads make in their first job is that the only thing they can do is what their manager told them to do. Sure you have to do that stuff too but if you're smart, figure out a way to get that stuff done and get other fun stuff done. That stuff you get to pick for yourself so it will be, by definition, not CRUD.
Reading theory and books won't get you far as you will realize that the real-world problems bring a lot of perspective that is not documented well enough in most books.
So, how do find a challenging problem to solve? Most people will suggest working on side projects, but some really hard problems (e.g. distributed computing) requires resources that make it impossible to pursue as side projects. Also, having people much smarter than you around you helps with the learning. So, the alternative in this case is to join a start-up or a big co just so that you get a chance to explore difficult problems.
Also, what do you mean by crud? I assume... http://en.wikipedia.org/wiki/Create,_read,_update_and_delete
It wasn't until I decided to migrate a personal project of mine from php to rails, and had to set it all up on a VPS with Phusion Passenger, all by myself, that I made that order of magnitude jump in skill level from novice to intermediate dev. Having that side project has also let me try out so many different things I could never be able to try out at work without know if it would work or not.
Those countless hours you spend off the clock screwing around with something just because you want to is simply invaluable to your progress as a developer, this is why to this day, I keep an eye out for developers that find time to hack on side projects.
On the other hand, in more developed countries, the demand is quite big for C++/Desktop/3D/Compilers etc "good stuff".
Web + large scale JavaScript stuff also falls in the latter case, although I don't consider it as "good" ;)
I don't think this relationship really holds. It's really more like: in every non-US country the overwhelming demand is for CRUD stuff. The demand for non-CRUD software is almost exclusively an American phenomenon.
This is, of course, somewhat of a generalization, but I do believe the trend holds true.
I grew up in Canada, where the software scene is very heavily CRUD. The main CS employers in town are such exciting names as SAP, IBM, TD Bank, Royal Bank, Bank of Montreal, and Scotia Bank. I spy a pattern. The demand for "higher level" programmers is tiny (there's a Mozilla outpost, and a small smattering of startups few people even know about).
From talking with friends internationally, it seems like this is the case in the UK and France also - all highly developed countries.
The only place where I've seen any substantial opportunities for C++, 3D, compilers, and other non-CRUD code is in the US.
There is a lot of C++ work in 'developing countries' (dependent on the country as well)
No 3D in France? Have you heard of Catia? There is C++ as well
YMMV, even though I like working with low level programming, the high level, semi-crud work can be nice and with less BS
Now you may debate the status of South Africa as either developing/developed, but I work in South Africa and since 98 I have worked on C/Cpp, VB, Python and mixtures of DB backends. Country does not determine the tool.
The solution determines the toolset.
Could you use threading to speed up bulk processing of documents?
Can you use some new package or gem to improve your day to day development flow? (for example, i just found a rails gem that helps speed up start up time for rails tasks, totally using that at work)
How about your Version control, are you using the latest techniques or type of software?
What about learning more about dev ops, how to manage the server and do little things here and there without bothering the server guy?
What about search, are you writing contorted sql queries to hit the databases for your search, or are you using elasticsearch, solr, sphinx or lucene which are better suited for those things?
Is your logging app writing to file logs, or (worse your RDBMS database) or could you use mongodb to speed that up?
Are you analyzing data you collect in an insightful way or is it time to read up on some analytics tutorials, to come up with smarter ways to inspect your data?
how fast is your software? Could it be faster? How can you make it faster?
etc etc etc
In the end it comes down to your curiosity to learn and try new things. That will help you keep abreast of new ideas and help you spot opportunities to apply them sensibly. It should get you from where you are until the point where you feel that opportunities like these are exhausted, at which point, its time to move on.
Never let yourself stop growing as a developer.
As @trustfundbaby says, you can apply new techs sensibly and that's what you should do, without the extremes.
And guess what? You could actually learn something from someone else!
Great portion of developers aren't "bored", they are bad, so they do everything as they go, without optimizing, without planning, etc. When they try using different things outside of their limited knowledge base, they screw up even harder. Creating the unnecessary complexity you talk about.
I find refactoring/optimization to be a necessity and even tho my boss probably won't ever know I optimized X, I know I did it, and I will keep doing it. It's how you grow as a developer, specially in CRUD companies that do not care.
And keep in mind, most developers don't want to get out of the CRUD. And they are very happy with a good paying CRUD job.
Keep your pulse on new technologies, and write your own to expand your expertise. But don't use the wrong tool for the job out of boredom.
The reason a lot of work is CRUD work is because most web applications and business applications are just data/information management systems targeted toward a certain use. My unscientific guess would be that 95% of at least minimally complex web and business applications boil down to being CRUD with a UI and a lot of marketing spin. This means that if you work in this segment you are going to be mostly working in CRUD. There are many other segments of computing where your work will be incredibly different.
Examples:
- Computer graphics/gaming
- Embedded systems
- Networking
Caveat - after working in a new segment you most likely will find that they have their own 95% core competency skill set (like CRUD is for web/business apps)
- Robotics (via working on open source educational robotics stuff)
- Virtual Worlds (via making hardware to interact with them)
- Health Devices (via reverse engineering health devices)
- Neuroscience (via reverse engineering eegs)
- Haptics (via reverse engineering haptics controllers. Noticing a trend here?)
- Digital/Interactive Art (via writing Max/Puredata/OpenFrameworks/Processing plugins for aforementioned projects)
- Teledildonics (via [censored])
(My github is at http://github.com/qdot, where you can witness the ADHD firsthand)
And the list grows every time I start working with a new library or device. Most of my dayjobs tagentially relate to something I've worked on in my spare time, though I try to keep from taking a job that'd completely subsume any one interest, as that tends to be a permanent mood killer (I'm looking at you, popular virtual world company that crushed my VW dreams).
So yeah, just find what you're interested in, do something in it, try to be part of a community around it, and let things work naturally.
Then comes the hard part: Tying all of this back together via /you/. You can make neat stuff all day long but if you can't tie it together with your own personality, then you're just a person who aimlessly cranks out neat stuff, which is something that will worry employers. Presenting a complete and coherent package of all the stuff you're interested in and why you're interested in it, with proof that you can do (more importantly: finish) it and show how that can benefit others is really the key. But damn, is it ever hard to get there.
PS stay the hell out of teledildonics. That is MY TURF.
Your company probably isn't interested in switching to cool language x or doing that big rewrite that you'd hoped to lead. You'll have to identify the skills you need for your ideal job and work on them yourself.
A great first step is a to automate repetitive parts of your job using languages and tools that you want to learn. Don't ask if you can do this, just do it discreetly at first. It's a good stepping stone between the familiarity of your job and side projects at home.
Even our massive financial platform is a glorified CRUD box with a mere three bits of cool algorithm stuff which have been the same since 1995.
Most of it is pasting bits of crap onto more bits of crap.
Even google apps is pretty much just a big CRUD system.
The art and science is making all these operations scale. Well that's what it is for me. 50 operations a second - no problems. We hit 10000 queries/second. Things work differently then which is where you have to turn back to deep computer science knowledge.
I've developed a deep knowledge of mathematics, statistics and algorithms trying to keep CRUD systems alive. we write our own cache, store and messaging layers as well as a logging system that can keep up with the audit requirements of the above.
At the end of the day, its quite interesting if you don't shrug it off.
Embrace what you have.
Edit: I am also the guy who gets all the traditionally shitty jobs. The hard things to debug, distributed bugs, timing and locking issues, race conditions, reliability problems, the 3 day debugging sessions, the Microsoft patch breakage dance and the 'its 2am and everything just caught fire' jobs. I LIVE for this. I am literally the last hope every time. I never lose. Being trusted with this and as the last line of defence is an honour and why I rather like what I do as well.
It's always challenging if you go looking for challenges.
Jobs should be more than a paycheck. You exchange your time/effort for growing/learning. Both parties are aware of it and should act accordingly. If one party is not living up to the bargain it's time to find a new partner.
Don't blame the CRUD for all the bullshit that is often associated with it.
(Reminder: CRUD stands for Create Retrieve Update Delete, and is the underlying philosophy for the database applications that run the world we live in, the 90% of the iceberg that most hackers never see or even think about.)
I've been doing CRUD for 33 years and it's been an incredible ride. I credit my CRUD work for putting me in the right hand 5% of that bell curve, ahead of the rest of my fellow hackers who have built so little of substance.
Sure, I've built apps that move people and things to the right place, and also have done a lot of "reports here, manage documents here, upgrade calender software here", but what job doesn't have some crap work go to along with the gravy? I've never seen any good job without it's share of maintenance, refactoring, testing, process, and meetings with drones.
I've also build software that generates other software, invented new frameworks, and devised algorithms that dramatically improved the way we make and build and move the stuff that's probably in your cubicle right now. I've taken cool academic theory from my pure mathematical background to build software that has blown away the posers everywhere around me. And you know what? You do that just once, and every champion in the enterprise runs to you with their giant budgets to solve their CRUD problem du jour with your "genius".
Believe me, you don't have to work at "Intel, Google, Apple, or NASA" to "change the world". You're just as likely to be a cog in the wheel at those places as anywhere else.
You can change the world from where ever you're at, and in fact, if you're working on a CRUD app, I think you're actually in a better position to do it. You just have to ask a better question.
My suggestion: "How to you escape the shackles and bullshit of your CRUD job and use all those great resources in your head to turn that job on its side and change the world from right where you're at?"
Find the answer to that question. It may be easier than you think.
Listen to this man. I was a full-time software engineer at Google for 5+ years, and I didn't change shit.
The only realistic shot for hackers to change the world is through their own startup.
Also, I think the phrase "change the world" is overrated. Sometimes it's not less important to keep the world running, which also requires constant work and a lot of human genius.
Imagine the software landscape as it would be right now with no free software whatsoever. Then imagine the hundreds of brilliant people required to devise and lead each of those projects...
He should and does get the credit for having the insight and chops to have started it all, and is the single-most important person for Linux, but today's Linux is the result of the work of thousands of programmers.
Actually, I think CRUD itself is a good thing. Why? Because it's a description of a simple interface. You really get the elegance of a CRUD API in the corporate world. Normally, you have a bunch of stupid features and requirements that attached themselves like barnacles.
In truth, building CRUD apps places a person well above the average in the typical corporate stack. At least you're building new stuff and have a chance to shine. Unlike on a maintenance job (which is what most people get) you will get visibility if yo do good work. Most corporate denizens don't even get that. They're lucky to write 200 lines of new code per month.
My suggestion: "How to you escape the shackles and bullshit of your CRUD job and use all those great resources in your head to turn that job on its side and change the world from right where you're at?"
The problem here is that most people need an income, every month, or they start to have serious financial problems. Knowing where overperformance leads-- getting attacked by envious/insecure idiots and ending up unemployed-- I wouldn't recommend it unless he has financial security.
In my experience, I have seen the opposite a number of times -- people who overperform have been rewarded with more responsibility and more-interesting jobs. This includes situations in which people automated away their job responsibilities.
It's more subtle than that. People don't get fired directly for overperformance, so much as they make enemies who later sabotage them. It's hard to do anything important and not piss someone off.
I have an overperformance story from Google that's legendary, although I didn't actually get fired.
Overperformance doesn't inexorably lead to termination, and it's certainly not immediate. It is, however, more likely to lead to termination than the opposite. (On the other hand, underperformance is more toxic to your career in the long term.)
"Performance" is a middle-class myth for AFCs (Average Frustrated Chumps). You get fired if you fail politically. Being at either extreme, performance-wise, increases the likelihood that this happens.
Looking at my work history, it cost me 3-4 jobs at companies that subsequently failed. The best example is one where the new and essential product needed to avoid an m*n database transaction explosion, where m is the number of clients. My warnings about this---and it was very easy to explain and the math is rather simple after all---kept me off that project and sidelined to a dead end. The project ... went even worse than expected, was never reliable after the 2nd client logged in (the project manager had never even done a multiple client system before), until a new crew was hired to rewrite it, but it was a little late by then....
When Mr. Church say "You get fired if you fail politically." he's spot on. That was obviously a factor these jobs and cost me another one or two. All of these were small companies, except for Lucent during it's year of free fall from 106,000 employees to a targeted 35,000 or so; they ended up getting bought by Alcetal.
Here's a constructive comment (I hope): one thing you have to watch out for is managers who are failed programmers. Even worse is if they had no input into your hiring.
This sounds like what you're saying--
Law 1: never outshine the master. https://www.youtube.com/watch?v=W6ldbjR2trg
Is any of that stuff open-source? What you say seems pretty cool, but I find it pretty uninspiring when the hard work of good hackers isn't free to be used and contributed to by others.
Fuck that.
The answer to your question is simple, and has very little to do with "changing the world" from your current job: you need to reach for what you want. If you're not happy with your current work, find new work that's better. Nobody is just going to hand you a plum job because you're kicking ass on CRUD apps. The world doesn't work that way. And it's highly unlikely that you're going to move into ML or compiler design by writing a classifier or a DSL for your current CRUD app-writing gig.
Want to be doing machine learning? Apply to positions that involve machine learning. Those could be at your current company, but probably not. You might have to take a pay cut. You might have to move. You might have to go outside of your comfort zone. But you need to work for it.
Can't get a job doing what you want because you're not qualified? Okay, fine. Go back to school. Find a junior-level job in a field that's orthogonal to what you really want. Maybe get a low-paying tech position in an academic lab that does research into ML or compilers or computer graphics or something else. Take courses at the local university in statistics, math and theory. Get an MS or PhD. People on HN will tell you that college is a waste of time, but they're wrong. They're mostly trapped doing CRUD work, like you. Education -- and certification -- matters. So, get some, if you're not qualified. Work your ass off, and learn theory. Most of your peers won't do the work.
The point is that you "escape the trap" by working really hard in a focused way. And it's a bit of a hill-climbing problem, because you don't really know your end goal, and you don't really know what the path looks like to get there, but you nearly always know a direction that's "up" from where you are right now. So, step aggressively in an upward direction. The way you avoid getting trapped is by not standing still.
This is about the vertical within our horizontal discipline. Working at a marketing company will be very different than defense. Some people want to work with visually appealing things, some people will only be happy if their software helps people. Don't blame the CRUD (cleaver play on words though). If you are truly a developer and find the horizontal you love you will be glad to work on it's CRUD.
Now, the good news is that most software companies are running at about 5% efficiency. Most of the code they have exists to serve dipshit requirements that aren't necessary and make no sense. It's junk complexity that no one would miss if it were gone. Why is this relevant? Because if you're putting 8 hours per day into order-following grunt work, you're an idiot. Do that, and of course you'll have no time to work on interesting stuff. The way you get a better job is to spend at least half your day learning the skills that you'll need to get where you want to go.
Treat it as an optimization problem. You want not to get fired. That's one constraint, but not a tight one because a lot of people keep their jobs for years while doing very little. Your objective: you want a better job in a year. Are you going to get a better job within the same company, doing more interesting work? Or are you going to leave for an external promotion as soon as it's viable? No one is going to know if you put side-project accomplishments on your resume, so you should.
Most people only "look for work" when they're in shitty situations. The rest of the time, they either (a) coast, or (b) put their all into their assigned work because they "really believe in the company". Both of these extremes are ridiculous. Always keep an eye open for something better, and start your search process when things are going well, not when they've gone to shit, politics have turned against you-- note, 95% of firings are about politics, not "performance"; that's an attempt to use middle-class guilt to prevent a fired employee from seeking legal recourse-- and you've lost your confidence.
You need to be selfish, because no one will look out for your advancement. Treat well the people who treat you well. Find ways to read machine learning books at work. (Don't open a book at your desk; that's political suicide. Get an e-reader or PDF.) One thing: don't write any code that you care about owning using company resources or "working time". If you want to turn a side project into something salable, get up at 5:00 or dedicate your weekends. It's not worth the risk.
So yes, the disgusting truth is that 90% of the work you'll be assigned will actually hurt your career. This is why, in most jobs, you should be putting just enough into your assigned work to get by, and using the surplus for your own advancement.
By the way, most of the work going on in the corporate "big guys" is junk as well. Yes, there are people at Google and Amazon who get to work on cutting-edge machine learning algorithms, but most people at those companies don't, you'll almost certainly not get such work in your first year, and the politics you have to navigate to get onto those projects will disgust you and, if you succeed, make you a worse person.
One solution would be only to work at open-allocation companies like Valve, but in 2012 there aren't that many of those yet. There will be more in the future, as they outshine the closed-allocation dinosaurs and starve them of talent, but that will probably take 10-20 years.
For the mean time, you have to learn to fight. One of the problems with most career advice sites is that they give AFC advice instead of teaching Game. The people who get to work on the good projects are those who figure out how to control the division of labor. They found a way to make people trust them "prematurely" and gained enough influence over the division of labor to have a niche. It's not a trivial thing to do. You'll have to go far out of your comfort zone, learn some social skills you might have been weak in, and it will take years. Right now, though, I don't see any other option.
Would you please share your professional experiences in industries other than software? Thank you.
Software politics are especially nasty, because the industry tries hard to believe that it's a meritocracy, but work where there's potential for excellence is so rare that it's allocated as a political favor. Most software engineers are assigned work that hurts their career, and have to figure out how to navigate this. The winning strategy seems to be to sneak away time and resources to learn what you really want, while creating the appearance of full dedication to the assigned work. It feels dirty and it takes a while before people are comfortable with it because, hey, everyone does it.
The best way to understand closed-allocation software politics is to watch a film about an unrelated industry: Glengarry Glenn Ross. The characters are sleazy salesmen in the 1980s, but it's really about Work and, specifically, the emasculating state of affairs that exists when high-quality work (the Glengarry leads) is inaccessible or allocated politically, leaving the rest to fail (with low quality work) in a game that looks like a meritocracy. When the film opens, you're led to believe that they're losers. They're being yelled at by their boss's friend (played by Alec Baldwin) in a performance that's truly epic. What becomes clear toward the end is that they are talented (if scummy) salesman, who have been set up to fail, because the leads given to them actually are useless. (One is a nutty old couple with no financial resources; they "just like talking to salesmen.") That's like software. The industry turns people into mediocrities and failures, but that's because the no-vision morons who call the shots at the tops of these companies are only giving them low-quality work, and they stop learning.
They're all just as packed with a shit-ton of politics as each other.
The worst thing I did was making pizzas while at college. That was asslicking and backstabbing central.
It's absolutely right that no one will look out for your career other than you, because it's no one else's job to help you advance. It's completely politics - not that it has to be cut-throat, but you have to own your own work destiny.
Of course, not wanting to advance is fine. Many of the techies-turned-managers I've worked for hated the advancement because it removed them from the actual tech. A few even made "lateral" moves to get back to the tech. Likewise, many of the engineers I've worked with are perfectly happy writing code, and just want a merit increase annually.
Someone once told me "dress for the job you want, not the one you have." It's basically that. If you want a better job, do your current job better. It will get noticed, and that's how you advance.
Advancement doesn't require becoming a manager. (If it does, in your company, then change companies yesterday.) It means working on more interesting, higher-impact projects.
Most engineers that I know only become managers because there is no other way to control the division of labor and get quality work. (Of course, in reality, managers don't get to do the decent work either because they spend all day in meetings.)
What you really need is enough control over the division of labor to continually have high-quality projects to work on. One solution would be only to work for open-allocation companies.
This has not been my experience. If it were true, I'd be the lead architect. I think politics are more important. It's not simply good enough to do your job well - you have to make sure everyone sees you doing your job well. In fact, you don't even need to do your job well to get ahead, you just have to step out a few critical times and make sure people see you doing a good thing. Hoping that you get noticed is, in my experience, the worst advice you can give.
Dilbert-style ass-licking and treachery is one way to play the game; being indispensable because you solve unsolvable problems is another. It is, by definition, the hacker's way.
It's tricky indeed because it requires a healthy dose of insubordination. A healthy company is a company which can be operated by real, i.e. mostly average, people. Big companies are optimized not to depend on individual genius, although they sometimes pathetically pretend the opposite. If you want to impose your better but non-standard solutions, you'll have to build them stealthy, them shove them down the middle management's throat when they painted themselves in a corner with the standard ones. Your way _might_ solve the problem, and it _will_ get you disliked by anal-retentive and insecure managers; move away from them, or make sure that you become indispensable faster than you become irritating to them. If you pull it off, they'll help you move away from them anyway.
Those problems are 'near' insolvable often because clueless people were in charge making decisions they shouldn't have been allowed to make. By rescuing orgs like that, you run the real risk of being relegated to perpetual clean up guy, and you bolster the decisions.
Rough example: System X was built so badly because of bad directives, that it's taking 3 hours for a report, and you need 10 reports per day, but you can only do 8 (8x3=24 hours). Every current employee and 2 outside consultants all say "this can't be fixed", not because it can not under any circumstance be fixed, but the cost/benefit doesn't even come close - it's a crappy bandage at best.
You, being "indispensable super dev" work overtime for 3 weeks to 'fix' things, and reports are now 15 minutes (like they should have been). Great - you just 'solved the unsolvable'. Whoop. You've perpetuated the bad decision making process, and it will be months or years before there can be real change in the org now.
Being 'indispensable' also usually means you're tied to crap projects and will never get moved out of that department/division in an upward capacity - you'll have to quit that company to get any real advancement.
You're right, it's a serious risk and you have to address it. Again, being a good, "straight A student" who does what he's told when and where he's told will harm you. You have to know to be bad at what you don't want to do, and at doing stuff a way you consider broken. There's a delicate balance to find between being recognized as valuable, remaining manageable, and not being threaded on. You need to be bad enough that people will try to avoid giving you that sort of shlep in the future, but it must not come off as insulting, and it must not be mistaken for incompetence.
The key point to keep in mind that dumb submission might save you a lot of flak, but will get you neither consideration from anyone, nor better work conditions. Know when to break the rules, and how much breaking you can get away with. You can't hack software in a company if you can't hack the company itself.
That's not what I'm advocating. You can get ahead without being sycophantic and treacherous, but you also need to know that other people will be, and you need to do whatever it takes to get enough credibility to defend yourself.
Acquire credibility. This doesn't mean "kiss ass" or "screw the other guy". That shit blows up as often as it works. It means that your job, when you start a new job, is to figure out what makes some people credible and others not, and either (a) to do what is required to be in the first set, or (b) to leave. I prefer option (b) over being slimy and sycophantic.
being indispensable because you solve unsolvable problems is another.
No one is indispensable. No one. Never forget that. I've seen companies fire their best people, losing millions of dollars, over inane political bullshit. Ego trumps money, which is just one way of keeping score to most people. Some people are more painful to part with than others, but there is no such thing as an irreplaceable employee.
The rest of what you are saying I agree with.
Indeed; I was just outlining a third option, between dumb submission and preparedness for job hopping.
> No one is indispensable. No one.
Replace "being indispensable" with "being perceived by your hierarchy as indispensable". Which, as you outline, can be two very different things, usually to the detriment of the delusional company.
More likely, in most non-technical companies, indispensable IT people are good people to replace at the earliest opportunity because they are "non-core" and could become problematic later. Non-technical companies want good, replaceable cogs using technology that are supported by many consulting companies.
They can't see your code that you developed at work, so a Github with some quality code would be good -- but the key is to remember that there is some employer out there who is better for you, and that that employer needs you as much as you need them.
Try my webapp, fiveyearitch.com , which we've optimized exactly for employed software engineers who want to scratch your itch.
I think it's important to leverage your current job, not escape it. You have a powerful starting position; you just need to hop over to a better job.
I've been geeking out about software architecture and engineering lately to make TDD not suck, so I built Obvious to scratch my own itch. It's the most fun I've had building software in years. I don't know if anyone else is going to care, but I've enjoyed it immensely. http://obvious.retromocha.com/ if you're curious.