are you also opposed to higher education/research in anything other than hard science, or do you think people in those fields should just stick to pencil and paper?
Well I guess it depends on what we define as art. I don’t think putting a piece of cheese on a table in the middle of a room is art or thought provoking beyond “this ‘artist’ is fucking shit”. Or running and jumping while drawing a line on the wall. That’s just insulting to actual artists who have talent that we can appreciate.
There was a Dilbert once (no longer available, for some reason), where Phil, the Prince of Insufficient Light, curses Dilbert to select one of two choices:
1) You will be paid well, but your code will be destroyed every night.
2) You will be poorly paid, but your code will be useful and appreciated.
Dilbert says "Hey! They're both better than what I have now! Wally, you should come and get some of this!"
Lots of reasons. Many folks are afraid of the unknown, so it's the devil you know, kind of thing.
That's why most folks aren't entrepreneurs. There's a ton of risk, involved in that kind of stuff, and most folks are risk-averse.
In my case, my job was sort of soulless, but only in parts. Many folks here, would say that it was a horror show, but I also worked for a company that gave me business cards that would open doors, I worked with some of the finest engineers and scientists in the world, enjoyed considerable trust, and had great stability.
Nothing comes for free. There's pluses and minuses to everything. I find it doesn't do me a whole lot of good to judge others on the choices they made, and, in the aggregate, there's not much in my life that I regret, and that's worth more than money.
There's plenty of low effort garbage on the internet that I would consider a waste way before this piece. Just the reactions of some other commenters making them rethink their career trajectory shows it has value.
Yes, I know many people in software/tech who have been 20+ years with the same company. Hopping every 2 years is not common.
That said, it is a red flag for me if most people in the team have been there for very long: it means that as a new comer I will always be bottom of the pile and have no career growth opportunity.
That’s a common story on HN but not at all a reflection of real life. Be careful letting the internet convince you of something without doing your research. There are lots of us with deep tenure at a single employer. It would not surprise me to find it’s still the majority of professionals.
I've seen it. Some people are weird and just want a stable job that pays them alright so they can focus on their family and other things they care about.
A few times in my career I've seen people with a decade-plus experience leave companies they would have stayed at for another two decades if it were within their control. It's sad. They struggle a bit because they often realise that they've over fit their skillsets to that single company.
One guy I knew was hired as a "web developer" in ~2005 because of his HTML/CSS knowledge. He continued to do basically just HTML/CSS until he lost his job in 2020. Lovely guy, but he saw his job as just a job and couldn't give a damn about learning new technologies. I often wonder what he's doing now... I doubt just knowing HTML/CSS is a hireable skillset anymore.
> Some people are weird and just want a stable job that pays them alright so they can focus on their family and other things they care about.
What's weird about that? It's how things work in every other sector, why would our sector be any different?
I wouldn't want to be stuck in the same position for decades, but that's different than being at the same company for decades. If I found a company that treated me right, paid me enough not to worry about money, and allowed me to progress internally (at a reasonable place), I wouldn't complain.
I had about 15 jobs over 40 years, not including 9 years at the two companies I started. But some were short term contracts, and several (in a row, sadly) ended in my employer going out of business, mass layoffs, etc. I always found doing something new was revitalizing. Also found that sometimes something new was awful. Being comfortable changing what technologies you are expected to use helps, taking advantage of new tech being new, and not many people knowing it, is good too. At my last job before retiring I stayed for 5.5 years, and could still be working there today, except it was enough of a career.
If I don't have easily-transferrable technical skills, how much additional value can I offer to a new potential employer in any engineering (non-management) role above junior or maybe middle-ish? I obviously recognize that there are other areas of skills, and that learning new tech gets easier with experience, but isn't there a limit to that?
Most coding is pretty mechanical... and most juniors can pick that up easily. In spite of the mysticism surrounding software development, higher level languages are actually intended to be easy to pick up. If you have good problem solving skills, those are harder to replace with a couple of juniors (or ChatGPT).
If coding is mechanical, then the code will often turn out to be overheady, uninspired suboptimal solutions á la everything is a class, not well extensible and maintainable, or the task at hand truly not fit for the skill of a good developer.
Coding is a craft. There is an art aspect to it and there is a lot of room for inspiration and ideas to improve things.
I don't disagree, but in my opinion code organization (which is what most of these things are) is mostly subjective, in other words, beauty is in the eye of the beholder. Good organization certainly helps with maintainability and extensibility, but it's not a prerequisite, and it won't on its own solve tricky or vague problems.
Question: Wouldn't a couple of juniors be more expensive? I don't think I ever work at a company where junior developer was paid less than half of a senior developer.
Exactly this. Guy had a comfortable stable job for years. Probably knew codebase and technology enough to have lots of time for family/friends/hobbies. More than this story scares me how HN reacted
I can only assume it's because HN is full of young, inexperienced people whose main wish is to stay on the cutting edge of technology rather than sinking into a stable profession
You may be right, but I don't think experience is what makes people prioritize stability over anything else. I'd be so depressed at a real job like the one described, it would be irrelevant that it was stable and gave me extra time. Some people are ambitious, some just like variety.
Wait you mean life isn't a video game where my position in a company completely defines me as a person, and if I don't destroy all my colleagues (read: competitors)with cruelty and cunning schemes I still have value?
Only if you don't have other aspirations in life. My day job is solely to fund my personal projects that I actually enjoy. Not everyone is out to be insanely rich.
If you fairly profit from your direct efforts in a lifelong career as a productive programmer, you will merely become rich, not insanely so. You appear to have constructed a strawman.
No, perhaps I was too subtle. My point is I don't care about the fact I might be making someone else rich. Life isn't about coming out ahead of everyone else you interact with. Some people simultaneously understand your point, and also don't care about it. Only within your own subjective opinion do you think they must.
It’s not, but from the outside the Bay Area’s cost of living looks impressively bad. A typical house in my neighborhood in the Northeast goes for less than 1/4 of that.
I met a thru-hiker on the Appalachian Trail that carried a few pouches of cat food. You need to carry emergency rations and they need to have decent nutritional content. But how do you stop yourself from eating it when mildly hungry? His theory was, it should be so gross that you wouldn't possibly eat it except in a true emergency. He made in from GA to CT without dipping into the forbidden slop.
It's 2023 and today companies will quite happily lay you off the moment you aren't profitable to them.
This fictional person made it all the way to retirement and good for him. But the guy who followed this strategy for 15 years and now is job hunting because the company needed to cut 18% of its staff? He's got a huge gap in useful skills and decades left before he can retire.
This is the real reason I put moderate effort into staying current: I've interviewed too many guys with 15 years in the field who can't write actual code at all because they only have written a few dozen lines ever. They are a sad lot and it's embarrassing to even interview them.
If I could be guaranteed I'd have the same job forever with the same tech then who cares, but I've never met anyone for whom that was true. The people that don't learn either retire early or are forced to switch careers.
Post hoc ergo propter hoc. They aren’t incompetent because they worked the same job for 15 years. And they probably weren’t even laid off because they were deemed incompetent.
You just got unlucky enough to interview them instead of someone who got laid off after 15 years and were amazing at their job and could pick up a new tech they’d never heard of and immediately be off to the races.
For ten years, I worked on codebases written in C++ without using any third party libraries (no, not even STL -- we needed faster, thread safe versions of strings and maps which were not available at the time; Boost? What's that?), its own custom messaging middleware built directly on sockets, its on distributed process management system built with unix syscalls (obviously). There was not a single line of HTML, Java etc. anywhere. It was a comfortable, high-paid life and I solved cutting edge problems (HFT, distributed, concurrent computing). In 2013, I forced myself to quit, start with a small company that paid me half the salary so that I could learn the web tech stack from the ground up. It was all to avoid this fate. I'd like to think I've succeeded.
What about it would you like to avoid? I don't understand the fears that people have in this thread. To me this sounds like the Platonic ideal of a programming job.
EDIT: I'll reply to my replier here because /u/Dang rate limited me (again), though I must commend him for letting me have one more comment than usual before he pushes the button ;)
> I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.
I would contend that such a person, unless very reckless with finances or extremely underpaid, should have enough money saved up to make what is mentioned there very unlikely. That said, I know some high-income earners struggle to not blow their entire bank account each month
I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.
I should think that years of specialized C++ work in hft environments would have come with impressively high pay… in which case one could retire early and self study anything for fun, not hope to find a job for financial reasons.
Ideally, yes! Unfortunately, there is such a thing as being young and not knowing your own worth / not knowing how to negotiate / not having a manager who looks out for your career growth.
In that case, if web development is your interest, then you have many possible paths.
The primary choices are full stack, backend, or frontend.
Obviously full stack is most flexible because you can theoretically do it all. But keeping up with so many technologies simultaneously is really tiring.
Take a look at Phoenix framework. It is a fantastic way to get started, and it will grow with you into just about any future project need. Plus Elixir is an excellent language built on top of the incredibly powerful Erlang ecosystem.
The web stack might give you more transferable skills but how did you calculate the trade off vs the 50% pay cut? Was the salary not actually very high and at a low ceiling compared to web work - i.e. you expect your salary to soon exceed what you made working in HFT? Or was there a serious risk of layoffs happening far in advance of your retirement? If the goal is to support yourself after retirement, are you saying in certain circumstances halving your salary 10 years into a tech career ultimately optimises your entire-career-earnings?
It wasn't such a purely material calculation. Less stress, more time to spend with family and friends, learning more things and doing something that you know actually adds value to the world -- those things matter.
That's mostly the guys fault and not the jobs fault.
Even in a bad code base you can start making things better, apply modern techniques and learn new things.
Also most of the skill you gain there is transferrable. E.g. just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS. It would just take a couple of weeks to get use to it.
Making things better and applying modern techniques don’t map to a feature story. In the environments being discussed you’ll have to wait for a cadre of architects and program managers to convince the C levels to commit to a multi year, millions of dollar effort to have any hope of improving a code base.
> Even in a bad code base you can start making things better, apply modern techniques and learn new things
All th bad codebase I've seen were a result of fossils who are still with the company gatekeeping anyone from fixing their code.
A few years ago, I had one architect stubbornly refuse to use source control, rejected all PRs that didn't use his homegrown buggy C++ libs for strings, refused to let anyone do the http server bits unless they used his C++ server and so on.
If there isn't a gatekeeper, you wouldn't have the bad codebase in the first place.
I've got to disagree. not every contributor is making the code base better. I'm in a situation where we are such a small team that there really is no gatekeeper, and we only require one approve per pr...and I've come back from vacation to some monstrosities that I have to look at with no time to fix.
After a pairing session with a a fellow developer, where I took the extra effort to refactor something along the way, probably an addition 15 minutes of effort...she quipped- no wonder it takes you so long to finish your stories sometimes.
I mean, that's not why- I'm ADHD, so my lack of productivity sometimes is invariably due to a day where I forgot my meds or didn't get a full nights rest.
> "just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS"
I disagree; working on a legacy PHP project isn't just opportunity cost (failing to sharpen your knife), it's actively harmful (leaving the knife in salt water, accelerating the rust).
It is a transferable skill. But not one many people want to pay for except in very niche areas these days. So if you are out of a job expect to grind for awhile trying to find someone to hire for C++. Unless you have network connections then you might be ok. Being good at C++ only goes so far, you have to have skills in techs people are willing to pay for. Not many companies are willing to wait while you skill up in their stack unless you are willing to work jr but they will not hire you anyway because 'you will jump ship'. Its not great.
that part is a complete fiction. People who have a lot of experience in the embedded world or performant but esoteric codebases are highly thought after. Even if the literal job isn't transferable, the skills for sure are.
My mother was a COBOL developer and didn't work for 20 years because she stayed home. She recently returned to her old bank she worked for because they are dragging people out of retirement as they can't find anyone else. She earns more than I do now.
People are crazy to give a job like that up for generic web dev work.
Most of my early programming work was with C and C++ (after I left academia, where it was mostly Fortran). I am, simply put, a better programmer for it, objectively so when compared to my peers who've never touched it, at basically every place I've worked at since (including a FAANG-level company, currently, where I'm mostly writing in go and python). At least, that's the feedback I get from them.
I think it's fair to say this skill difference is largely from that work in the C family, where I learned a number of different paradigms for design and development in that world, with all its footguns and low-level "gotchas". It has always been easier for me to parse and understand others' code, identify subtle bugs, to use debugging tools, and to identify some optimizations by examining the code that others might spend days using profilers to find.
Some of that would come with engineering experience in general, regardless of languages used, but not all of it, but I picked these skills up faster and earlier in my career, thanks to that early work. Skill with C++ is definitely transferrable, in at least some cases.
High frequency algorithmic trading. Often ran on colocated hardware within the exchange / trading system data centers. Started with stuff like Sun SPARC, Solaris and Oracle and we slowly made our way to Suse, Intel and... surprise, our own database optimised for fast writes.
Unfortunately, yes. It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.
The whole business model is misunderstood by your average internet commentator.
When you have a market, you need someone to be there to provide liquidity. Imagine if you're a farmer and you show up to the market with your wheat, but all the bakers have gone home that day. Or the baker shows up and there's no farmer. The market maker stands around all day offering to buy and sell so that you don't have to wait for the guy you're really trading with. Of course this middle man wants to get paid for it, but your cost as an average Joe is next to nothing. This is trading in time.
Now imagine you want to cook a meal and your ideal meat is beef, but actually you're ok with pork, so long as the pork is cheap enough to make it worth it. How much cheaper should it be? Well your fellow who knows all the pork and beef people will be able to gauge where the balancing spread is, given the amount of interest. In fact he will from time to time do the trade when the spread is out of line. This is trading in space.
So why all the fancy tech? After all market makers used to stand around in a pit in a colored jacket, and they didn't have degrees. My first boss in the market was one of these guys.
Well, things have gotten very tech heavy because as soon as prices are out of line, there is money to be made. Or rather, lost. As a market maker, you are constantly out there with your prices, offering to buy or sell at a very small spread. If some news happens that affects prices in a big way, you can be sure that you will buy when it's going down and sell when it's going up. In order to both have tight prices and avoid this "adverse selection", you really want to be able to react as quickly as you can when your system decides that something's up.
I explained that. It doesn't matter whether it's a computer doing it or a guy in a jacket, but once one guy gets a computer all the market makers need it.
Retail consumers aren't exactly disadvantaged by the gradual narrowing of spreads, so what are you trying to fix? The latency arms-race is exclusively professionals fighting each other.
Getting back on topic, being a software developer for an HFT shop can be a lot of fun: serious technical challenges and a rapid feedback loop in a role where you may be the revenue stream rather than just another cost center. But posters highlighting NIH syndrome are spot on: you risk pigeonholing yourself.
Markets were possible. But what about the current offering of ETFs and similar products? Despite all the fancy talk, many HFT firms are just market makers for the products that enable average joes to invest well and cheaply.
You don't have to be that old to remember how terrible the experience used to be, and how easily even sophisticated individuals were ripped off at every point of the process.
You could easily solve all of this without HFTs - you might introduce a few seconds of latency but virtually no real investors care about that level of latency and it would save them billions per year.
It absolutely would if you remove the cut HFTs take out of the market. I'm sure you'll make some argument about liquidity and spreads but, in the aggregate, the money from higher spreads goes to market participants so the market as a whole still saves by eliminating HFTs, even if the spreads go up a bit.
This is like saying if we eliminated millimeters from rulers, and only used centimeters, everything would get closer together.
HFTs do not take a cut out of the market, their cut comes 100% from other market makers. Those are the only folks who would benefit from eliminating HFTs, and unless you’re one of them, who cares?
I don’t care who buys when I am liquidating a position. That’s the whole point of a market. Therefore I don’t care about HFTs.
Generally, it's not about front-running normal people. Instead its racing other HFTs. Either a race to capture arbitrage opportunities, or to update your orders on one market based on trades made in another market.
The aim of the game is to collect a small spread on a lot of transactions.
The scary thing for an HFT like that is being 'run over' by a big party selling or buying a lot over a day.
If a pension fund is dumping 10٪ of their holdings of shell, it's hard to collect a spread because you need to find buyers for every stock you buy from them. Meanwhile the price is dropping as they sell off.
Hence you occasionally see 'front running' with price improvement. Instead of buying from the market at 1.02 a HFT gets to sell to you at 1.01, yielding you a better price. This is a good deal for the HFT, because a normal person isn't going to run the HFT over. And it's a good deal for the normal person because they get a better price.
No, it's a zero sum game. HFT firms fight to be the first to pick up a penny, but they can't force anyone to drop more pennies. They don't cause prices to go up more than they would have otherwise, or extract more value from the poor and middle class.
So you left a job that paid 2x and exposed you to hard problems that require reliable solutions for a job inside the horror clusterfuck that's the modern web where nothing really "works" and there's no real engineering to speak of besides monkeys throwing crap in a shared pile and shovelling it out the door ASAP.
Most people I know, inc. myself, would call this a "bruh moment".
I can't fathom willingly entering the web mines if you've already sunk in the hours to master high perf native code, OSdev, dist systems etc. People usually enter web to avoid having to learn such things.
To discard that knowledge and instead compete with an endless stream of fresh grads, boot camp grinders and LLMs for less money? A madness if I've seen one.
Web tech is more immediately rewarding. With C++ (and especially with modern web dev agile processes), you can spend a week building enough framework and infrastructure crap to stand up the barest inkling of a demo of the solution you plan to implement, only to have the whole thing pivot out from under you and become wasted work.
More complicated languages and frameworks that manage memory and do UI and everything for you let you get to a prototype much faster, so you can see whether it'll work or whether you want to toss out the idea much sooner.
It's hyperbolic but it's a common sentiment that I tend to agree with. The web dev world changes too fast and too often, many stacks involve absurd complexity for dubious performance and maintainability and other metrics.
I've been doing web dev for 20+ years. Which part is wrong?
It's not hard to find feature support or implementation differences between browsers. But these days, that's not really the problem. I can go read the ES language specification, and generally find user agents/node/deno/bun have compliant implementations.
The problem is brittle "ecosystem" stuff. How are the create-react-apps going? How is module bundling going amidst the UMD/CJS/ESM war?
Even if the decision he made was a bad one, you don't know anything about why it made sense for him at the time.
People aren't algorithms running in complete isolation and silence. They live in nuanced circumstances you can't possibly understand by his short history in a comment.
I hope you can find understanding and peace in the future.
There is a massive advantage in being able to see the web tech stack for what it is and wielding it without over-engineering or fanaticism, while your competitors are spending 2-3x on monstrosities that essentially does the same thing, but slower.
I was originally concerned about your parent comment regarding your new foray into web dev (that C++ path sounds amazing), but this comment has put my soul at ease.
If you walk into web dev with aggressive principles and a willingness to be controversial, you can get a lot of shit done and it will feel great. Watching others struggle under the various JS framework circuses has almost an opulent/aristocratic vibe in 2023.
And now you are jumping from one "stack" to another, none the wiser, your knowledge hopelessly outdated just a year in. The principles you learned in HFT? Those apply all the same.
I know it's not all about the money, but over the course of your entire career, do you think that move was financially positive or negative?
I'm very roughly estimating that you could probably have worked in the HFT job for around two-thirds the time you're going to have spent working on generic web tech, and still been better off.
Absolutely 100% positive. I was largely an individual contributor all that time, and only did some unstructured, untrained, unguided people & project management. Perhaps because it was such a niche, engineer-dominated environment. Several years after leaving that company, I quickly learned to effectively manage people and projects, architect systems using more mainstream/open source tech and eventually reached exec level (but that's a different story).
That's really some fields I'd love to get into -- less need to talk to people and more need to talk to the machine. What a dream! Alas I only scratched the surface of C++.
Ah, that's the missing punchline: you had ambitions to be something else other than an engineer, and you (quite correctly) recognised that you were never going to grow your wider skills at such a place.
For someone who had wanted to stay an IC, I don't think the move would have been quite as wise. Being in a niche, engineer-dominated environment, and being paid extremely well for it, is nirvana for many lifelong ICs. It certainly feels very different from the experience of Arthur from the linked article.
I don’t think anyone working with low-level C++ for HFT can face similar fate. Most of skills gained at such job seem very transferable. The article seems to be more about jobs where 95% of work consists in implementing CRUD in accordance with very specific business logic of the company.
The only reason to change such job I can imagine is to sit out your non-compete package. But I am not in this industry so maybe I’m missing something.
Yeah, I find it amazing someone would transfer to standard web CRUD development from that. Sure you can have enough of all the custom stuff, but there are plenty of companies that will hire you for your current salary for non-web work. (you can probably guess how little I like web work)
Same! The poster's pivot reads like horror to me. Like they lost their mind and joined a cult where they worship Memnon with sermons of JavaScript.
> It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.
Ah, nevermind. Turns out they were worshipping Memnon with C++ and renounced that cult with web tech! :p
It's not that you won't have the skills, it's somehow demonstrating them.
I also worked at an HFT C++ shop that used no STL. Every data structure has a weird name and API that you get used to. The whole architecture has its own idioms, and hasn't needed much upgrading in terms of language features.
Imagine you learn the grammar of English. You learn how to pluralise nouns, when to put an s at the end of a verb, how to to use commas, and so on. You then go to work at a place where they have a different word for coffee, drink, and order. Along with every other word. You end up learning where everything is and how to talk about it, and you're productive.
How do you apply for another job? Chances are you will be asked how to order a coffee and drink it.
This kind of issue cuts both ways - how do you hire new people and get them productive quickly on a code base where they will almost certainly understand nothing?
Getting rid of weird data structures that now have STL analogs then is a productive exercise - existing devs get to learn more STL and new devs have more familiar ground. You get to delete code too.
I did the same, I thought C++ was heading for obsolescence. Now a master of Java, Javascript and can write great full stack apps. Meanwhile HFT C++ jobs are paying double, I'm thinking of going back, even if the systems are boring.
been working on java jobs and easy code bases since I graduated 10 years ago, nothing had been really challenging and I'm not complaining. Would be interested in a job that pays more even do you think working on C++ jobs is more challenging or just boring?
Depends on the job of course. Probably more technical problems and tuning. Often code is low level which is interesting if its your thing but not if you like business problems.
Those stacks can be full of 'not invented here'. They are unique and have their own quirks. Quirks that you more than likely can not google for any sort of guidance as to what is going on. I wrote one of those stacks a few years ago. No std libs at all. Why? must fit in 32k and most of that space is needed to hold data. That means throwing a lot of those helper bits out or spend weeks figuring out how to bend the compiler to your will and not include everything. When it is easier just to roll your on CS version of whatever. But it sucks because you probably have odd edge cases.
Yeh it's a rough but rewarding life; I spend most of my hours on problems that aren't on Google, or any LLM, so I'm kinda used to it now. (Stockholm)
Would love to know of more spaces which require this level of programming - usually I feel kinda underworked at most shops.
Though, true enlightenment is combining the urge to NIH with some, light, sensible borrowing from those smarter than us :) very much a key skill unto itself.
The space I was in was IoT type items. Usually hub controllers. The thing is that is getting less and less as you can get a fairly robust thing now with a couple hundred meg of flash and ram for a very reasonable price. So those 'must fit in 32k' items are few and far between these days. With that sort of space you may as well use the proper libs that are usually nicely documented and all the quirks have been seen by hundreds of other poor souls such as yourself.
Embedded programming is this 99% of the time. Abstracting hardware away is a silly notion. Docker? Cloud? Hard to do when your code only runs
at all on one specific custom board. The protocol you wrote to talk to that one chip isn't working? Time to break out the oscilloscope and look at the actual data going to the pins.
Join us, the pay is worse and the work is harder, but also the prestige is lower.
I sometimes really miss embedded programming; it felt like real engineering in a way that higher-level software development often doesn't. Alas, the shop I worked for could not keep me busy enough, the projects began to feel repetitive, and I got really bored; it'd be hard to go back.
I have a pretty similar story. I spent 7 years working as a subcontractor for a big telco with CORBA, pthreads and the ACE reactor library. First week, I halted a whole contact center service by adding a sleep call to the codebase (it was on UAT environment). Fast forward several years and I could effortlessly debug cores with thousands of threads, reason about semaphores, reentrancy and even tell the difference between the POA and the BOA Orbix adaptors. I began to become the mythical guy who knew what everything did not because I was technically good, it was because I understood the business and how it mapped to the different architecture elements.
Then I felt like I could spend my whole career there and decided there was a whole world out there in the web, began to learn PHP in 2004 until the mythical "blog in 5 minutes" Rails demo appeared. I switched jobs in 2007 and I have never looked back.
Interesting. I've used semaphores with pthreads (only once -- after that I realised plain old mutexes will do). Like your sleep call, I once brought an entire service to a grinding halt by adding a simple `fsync` call (to fix a recovery failure because once a sequenced message we thought we had written to disk before a crash had actually got lost in the buffer). Understanding things at that level makes any new tech fairly easy to grok.
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
C++ is such a damn chore. This is partly why, everyone always assumes they can do it better, they don't trust any other code, so you end up with monoliths of NIMBY code and esoteric build setups. Code reviews are incredibly painful for the same sort of C++ reasons, everything can effect everything, the whole idea of encapsulation tends to fall apart when memory, time, ordering behavioral effects get relied on after some amount of time usage. Every change is a footgun to be found in production under just the right circumstances.
Hard to be productive when you can't trust the language, the tools, or other developers to be correct.
I hope it worked for you, but I don't see any big difference. Web technology will become dated faster than the old C++ codebase. If you learn some web tech stack today I expect that it will be obsolete in 5 to 10 years. What are you gonna do next? Move to another small company for half the salary to learn everything again and compete with 20 year olds?
There is something to be said for stability. Since our protagonist retired early, the pay must not have been bad. The glass is half full, or so I tell myself.
I know people who enjoy street performance and dumpster diving. Bringing your juggling balls (or guitar) to a street corner and trying to make money gives some pressure that makes it fun - and that you don't need the money means if people don't like you enough to pay it doesn't matter. Likewise people throw away a lot of treasures that you can fix up and sell/give away.
If he needs to be a street performer and dumpster diver to make ends meet that isn't good. However people do those things as a hobby all the time.
I worked with a guy who made easy into the 200k range in the early 2000s two separate pension plans. Total dumpster diver. He loved the thrill of finding stuff. Wore complete rags at work. Drove a car from the early 70s. Ate simple homemade leftovers ever day for lunch. He loved the idea of racking up a high score.
True, but people usually don't start with this on retirement. And the end of the story is about him planning to eat cat food(?). Not sure how to take this, as it also talks about Soylent and Premium Mix, which are not the cheapest.
That is one extreme. The other one is developers hopping from new project to new project without ever even seeing the product going live or getting user feedback/bug reports from the live product.
If you serve plain static HTML from S3 or Cloudflare Pages (or even a vanilla nginx on a vps) and put a decent CDN in front of it (and don't load 3p resources), it will astound people how fast your page loads. I regularly get this feedback on my own webpage.
It's not rocket science - it's how we made webpages in 2001. Browsers are really good at that.
Life is too short for working with crap code. Unfortunately there is more crap code out there, than there are opportunities of working on beautiful code. At least in most jobs there will at minimum be compromise and part of the job will be dealing with ugly code.
> He probably made fat bank which turned out to be golden handcuffs.
This is a big assumption, not everyone in this field is a high earner - plenty of people make okay-ish salaries without being anywhere near highly paid.
That's what Dan did at my old job. Retired at 50. We would go on per diem travel and he would bring food from the hotel. We called them 'Dan-wiches'. He was smart and frugal. Now living his best life.
That's not how it works.
Those who've been there 20 years and who would have difficulties finding another job are those who get low salary increases. On the other hand they've been there for so long without looking elsewhere that they probably have no idea that their pay is low.
I think you are correct. It seems like all of the other prices align with the weekly rate as 5 days. It also makes the daily rate seem more reasonable, at a yearly rate of $240,000. In a better interpretation of this, I could have maybe also assumed that this is just a fun way of stating something like "hiring to work on doubling users costs this much, but there isn't actually a guarantee that the userbase will be doubled."
I've spent 10 years as the lone dev at a small medical facility, writing database software and dealing with lab hardware etc.
I guess I've touched 100% of the codebase because I wrote it all, but I always wonder if my skillset is so niche that I'll always be at small companies.
Here's the deal though, work isn't everything and I'm at a job that recognises that.
You wrote it all? By yourself? That sounds ideal to me. The horrible part of this to me is that he inherited a mess and left without having done much to it, evidently being unable to do so for whatever reason. The headline says he "mastered" it, so clearly other forces were at work than his skill. That's the depressing part to me, pulling my hair out to try and understand someone else's antediluvian code base which was built under duress.
I think devs imagine someone else out there is having a better time at their computer than they are. I definitely do.
It also said he only wrote hundreds of lines in 35 years. So maybe one line a month. Either he's a wizard that spins gold from straw or "mastered" is being used ironically. Given that he could be replaced by two juniors I'll assume the latter.
595 comments
[ 4.8 ms ] story [ 139 ms ] threaddo you believe that something has to be purely "productive" to be deserving of space on the internet?
honestly, I don't think that's a huge leap given your previous comments—am I wrong?
1) You will be paid well, but your code will be destroyed every night.
2) You will be poorly paid, but your code will be useful and appreciated.
Dilbert says "Hey! They're both better than what I have now! Wally, you should come and get some of this!"
That's why most folks aren't entrepreneurs. There's a ton of risk, involved in that kind of stuff, and most folks are risk-averse.
In my case, my job was sort of soulless, but only in parts. Many folks here, would say that it was a horror show, but I also worked for a company that gave me business cards that would open doors, I worked with some of the finest engineers and scientists in the world, enjoyed considerable trust, and had great stability.
Nothing comes for free. There's pluses and minuses to everything. I find it doesn't do me a whole lot of good to judge others on the choices they made, and, in the aggregate, there's not much in my life that I regret, and that's worth more than money.
That said, it is a red flag for me if most people in the team have been there for very long: it means that as a new comer I will always be bottom of the pile and have no career growth opportunity.
A few times in my career I've seen people with a decade-plus experience leave companies they would have stayed at for another two decades if it were within their control. It's sad. They struggle a bit because they often realise that they've over fit their skillsets to that single company.
One guy I knew was hired as a "web developer" in ~2005 because of his HTML/CSS knowledge. He continued to do basically just HTML/CSS until he lost his job in 2020. Lovely guy, but he saw his job as just a job and couldn't give a damn about learning new technologies. I often wonder what he's doing now... I doubt just knowing HTML/CSS is a hireable skillset anymore.
Context is always important.
What's weird about that? It's how things work in every other sector, why would our sector be any different?
I wouldn't want to be stuck in the same position for decades, but that's different than being at the same company for decades. If I found a company that treated me right, paid me enough not to worry about money, and allowed me to progress internally (at a reasonable place), I wouldn't complain.
Coding is a craft. There is an art aspect to it and there is a lot of room for inspiration and ideas to improve things.
True but often enough great code is not needed just working code will do just fine.
There are better things to aspire to than predictability whilst making someone else rich.
Making someone else rich is the objection.
Having no job or income and having to worry about how to afford food or rent is surely much worse than "just" having a stable job.
Or how about a super stressful job, like the Japanese salarymen who are literally killing themselves?
https://www.zillow.com/homedetails/8057-W-Farm-Road-100-Will...
Projections, my dear Watson.
It's 2023 and today companies will quite happily lay you off the moment you aren't profitable to them.
This fictional person made it all the way to retirement and good for him. But the guy who followed this strategy for 15 years and now is job hunting because the company needed to cut 18% of its staff? He's got a huge gap in useful skills and decades left before he can retire.
That guy is in real trouble.
If I could be guaranteed I'd have the same job forever with the same tech then who cares, but I've never met anyone for whom that was true. The people that don't learn either retire early or are forced to switch careers.
You just got unlucky enough to interview them instead of someone who got laid off after 15 years and were amazing at their job and could pick up a new tech they’d never heard of and immediately be off to the races.
Arthur was happy.
My favourite passage from Camus' little known book Software Development is Existentialism
same fate
> I'd like to think I've succeeded.
possibly not
EDIT: I'll reply to my replier here because /u/Dang rate limited me (again), though I must commend him for letting me have one more comment than usual before he pushes the button ;)
> I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.
I would contend that such a person, unless very reckless with finances or extremely underpaid, should have enough money saved up to make what is mentioned there very unlikely. That said, I know some high-income earners struggle to not blow their entire bank account each month
In that case, if web development is your interest, then you have many possible paths.
The primary choices are full stack, backend, or frontend.
Obviously full stack is most flexible because you can theoretically do it all. But keeping up with so many technologies simultaneously is really tiring.
Take a look at Phoenix framework. It is a fantastic way to get started, and it will grow with you into just about any future project need. Plus Elixir is an excellent language built on top of the incredibly powerful Erlang ecosystem.
Even in a bad code base you can start making things better, apply modern techniques and learn new things.
Also most of the skill you gain there is transferrable. E.g. just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS. It would just take a couple of weeks to get use to it.
All th bad codebase I've seen were a result of fossils who are still with the company gatekeeping anyone from fixing their code.
A few years ago, I had one architect stubbornly refuse to use source control, rejected all PRs that didn't use his homegrown buggy C++ libs for strings, refused to let anyone do the http server bits unless they used his C++ server and so on.
If there isn't a gatekeeper, you wouldn't have the bad codebase in the first place.
I disagree; working on a legacy PHP project isn't just opportunity cost (failing to sharpen your knife), it's actively harmful (leaving the knife in salt water, accelerating the rust).
Start saving now.
I'm sure he could learn STL (standard C++ library) in a fraction of time it would take someone else to become proficient in C++ in general.
My mother was a COBOL developer and didn't work for 20 years because she stayed home. She recently returned to her old bank she worked for because they are dragging people out of retirement as they can't find anyone else. She earns more than I do now.
People are crazy to give a job like that up for generic web dev work.
I think it's fair to say this skill difference is largely from that work in the C family, where I learned a number of different paradigms for design and development in that world, with all its footguns and low-level "gotchas". It has always been easier for me to parse and understand others' code, identify subtle bugs, to use debugging tools, and to identify some optimizations by examining the code that others might spend days using profilers to find.
Some of that would come with engineering experience in general, regardless of languages used, but not all of it, but I picked these skills up faster and earlier in my career, thanks to that early work. Skill with C++ is definitely transferrable, in at least some cases.
Because that's what a lot of people think online. Maybe it's more nuanced. I'm curious how someone from that carreer sees it?
It is to help the super-rich who own hedge funds to frontrun the rich upper-middle-class people with significant disposable income.
When you have a market, you need someone to be there to provide liquidity. Imagine if you're a farmer and you show up to the market with your wheat, but all the bakers have gone home that day. Or the baker shows up and there's no farmer. The market maker stands around all day offering to buy and sell so that you don't have to wait for the guy you're really trading with. Of course this middle man wants to get paid for it, but your cost as an average Joe is next to nothing. This is trading in time.
Now imagine you want to cook a meal and your ideal meat is beef, but actually you're ok with pork, so long as the pork is cheap enough to make it worth it. How much cheaper should it be? Well your fellow who knows all the pork and beef people will be able to gauge where the balancing spread is, given the amount of interest. In fact he will from time to time do the trade when the spread is out of line. This is trading in space.
So why all the fancy tech? After all market makers used to stand around in a pit in a colored jacket, and they didn't have degrees. My first boss in the market was one of these guys.
Well, things have gotten very tech heavy because as soon as prices are out of line, there is money to be made. Or rather, lost. As a market maker, you are constantly out there with your prices, offering to buy or sell at a very small spread. If some news happens that affects prices in a big way, you can be sure that you will buy when it's going down and sell when it's going up. In order to both have tight prices and avoid this "adverse selection", you really want to be able to react as quickly as you can when your system decides that something's up.
Getting back on topic, being a software developer for an HFT shop can be a lot of fun: serious technical challenges and a rapid feedback loop in a role where you may be the revenue stream rather than just another cost center. But posters highlighting NIH syndrome are spot on: you risk pigeonholing yourself.
You don't have to be that old to remember how terrible the experience used to be, and how easily even sophisticated individuals were ripped off at every point of the process.
I guess I'm not old enough. What was the system like? How did sophisticated traders get ripped off?
> and it would save them billions per year.
This is not true.
HFTs do not take a cut out of the market, their cut comes 100% from other market makers. Those are the only folks who would benefit from eliminating HFTs, and unless you’re one of them, who cares?
I don’t care who buys when I am liquidating a position. That’s the whole point of a market. Therefore I don’t care about HFTs.
The aim of the game is to collect a small spread on a lot of transactions.
The scary thing for an HFT like that is being 'run over' by a big party selling or buying a lot over a day. If a pension fund is dumping 10٪ of their holdings of shell, it's hard to collect a spread because you need to find buyers for every stock you buy from them. Meanwhile the price is dropping as they sell off.
Hence you occasionally see 'front running' with price improvement. Instead of buying from the market at 1.02 a HFT gets to sell to you at 1.01, yielding you a better price. This is a good deal for the HFT, because a normal person isn't going to run the HFT over. And it's a good deal for the normal person because they get a better price.
Good for you I guess?
I can't fathom willingly entering the web mines if you've already sunk in the hours to master high perf native code, OSdev, dist systems etc. People usually enter web to avoid having to learn such things.
To discard that knowledge and instead compete with an endless stream of fresh grads, boot camp grinders and LLMs for less money? A madness if I've seen one.
Web tech is more immediately rewarding. With C++ (and especially with modern web dev agile processes), you can spend a week building enough framework and infrastructure crap to stand up the barest inkling of a demo of the solution you plan to implement, only to have the whole thing pivot out from under you and become wasted work.
More complicated languages and frameworks that manage memory and do UI and everything for you let you get to a prototype much faster, so you can see whether it'll work or whether you want to toss out the idea much sooner.
It's not hard to find feature support or implementation differences between browsers. But these days, that's not really the problem. I can go read the ES language specification, and generally find user agents/node/deno/bun have compliant implementations.
The problem is brittle "ecosystem" stuff. How are the create-react-apps going? How is module bundling going amidst the UMD/CJS/ESM war?
If you want interesting, go to HPC large scale simulation.
Even if the decision he made was a bad one, you don't know anything about why it made sense for him at the time.
People aren't algorithms running in complete isolation and silence. They live in nuanced circumstances you can't possibly understand by his short history in a comment.
I hope you can find understanding and peace in the future.
I'm not sure if that's intentional humour or not.
At that level of abstraction it’s all just strings
Exactly... I worry about the masses learning of this and gaining literacy over the landed gentry.
PHP/CGI basically had it solved. I emulate PHP in .NET6 today using verbatim and interpolated string operations.
If you walk into web dev with aggressive principles and a willingness to be controversial, you can get a lot of shit done and it will feel great. Watching others struggle under the various JS framework circuses has almost an opulent/aristocratic vibe in 2023.
I'm very roughly estimating that you could probably have worked in the HFT job for around two-thirds the time you're going to have spent working on generic web tech, and still been better off.
For someone who had wanted to stay an IC, I don't think the move would have been quite as wise. Being in a niche, engineer-dominated environment, and being paid extremely well for it, is nirvana for many lifelong ICs. It certainly feels very different from the experience of Arthur from the linked article.
The only reason to change such job I can imagine is to sit out your non-compete package. But I am not in this industry so maybe I’m missing something.
> It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.
Ah, nevermind. Turns out they were worshipping Memnon with C++ and renounced that cult with web tech! :p
I also worked at an HFT C++ shop that used no STL. Every data structure has a weird name and API that you get used to. The whole architecture has its own idioms, and hasn't needed much upgrading in terms of language features.
Imagine you learn the grammar of English. You learn how to pluralise nouns, when to put an s at the end of a verb, how to to use commas, and so on. You then go to work at a place where they have a different word for coffee, drink, and order. Along with every other word. You end up learning where everything is and how to talk about it, and you're productive.
How do you apply for another job? Chances are you will be asked how to order a coffee and drink it.
Very real issue btw, I still have friends there.
Getting rid of weird data structures that now have STL analogs then is a productive exercise - existing devs get to learn more STL and new devs have more familiar ground. You get to delete code too.
There's at least one in the bay area though.
Would love to know of more spaces which require this level of programming - usually I feel kinda underworked at most shops.
Though, true enlightenment is combining the urge to NIH with some, light, sensible borrowing from those smarter than us :) very much a key skill unto itself.
Join us, the pay is worse and the work is harder, but also the prestige is lower.
I tip my hat to you!
Then I felt like I could spend my whole career there and decided there was a whole world out there in the web, began to learn PHP in 2004 until the mythical "blog in 5 minutes" Rails demo appeared. I switched jobs in 2007 and I have never looked back.
If you have access to key systems, you can definitely reverse-Conway to understand how an organization really works.
https://en.wikipedia.org/wiki/Conway's_law
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
— Melvin E. Conway
Ouch. Should have stayed in legacy C++.
Hard to be productive when you can't trust the language, the tools, or other developers to be correct.
As God intended it!
If he needs to be a street performer and dumpster diver to make ends meet that isn't good. However people do those things as a hobby all the time.
He's doing well
> Westbrook once touched legacy code without ending the company.
probably your communication skills were too good for management to passt up the opportunity of promoting you.
It's not rocket science - it's how we made webpages in 2001. Browsers are really good at that.
^^^This!
Arthur was lucky to retire at 58. He probably made fat bank which turned out to be golden handcuffs. If he ever got downsized, he had been f*d.
This is a big assumption, not everyone in this field is a high earner - plenty of people make okay-ish salaries without being anywhere near highly paid.
[1] https://taylor.town/hire-me
> 3m $60k double your daily active users
I've spent 10 years as the lone dev at a small medical facility, writing database software and dealing with lab hardware etc.
I guess I've touched 100% of the codebase because I wrote it all, but I always wonder if my skillset is so niche that I'll always be at small companies.
Here's the deal though, work isn't everything and I'm at a job that recognises that.
I think devs imagine someone else out there is having a better time at their computer than they are. I definitely do.