Ask HN: Do you feel the quality of SWE has gone down?
It feels like a lot engineers now days don't seem to have a good cs background. They don't seem to understand things like cache, paging, virtual memory,cpu pipelines, algorithms or other things pretty important to CS.
I know we have a lot of bootcamps and people are joining because it pays decently, but is this necessarily a good thing for the industry?
When we have the next industry crash (.com crash) will these people stick around?
96 comments
[ 3.3 ms ] story [ 163 ms ] threadI feel like the people who jumped into boot camps to make a quick buck will move on to the next hot thing
I'd like to think that the people who are in the industry because they actually enjoy the challenges and problem solving will stick around
What I think matters: are they going to be effective at their job at your company? If they don’t know these basics I wouldn’t want them as an employee. Go back to school kid
The Java app at my current company does 23k database queries uncached for a directory listing with 100 files in a folder. There are two devs at work there for years, nobody bothered to look up the rather well documented api docs that just use a single query for that task from the upstream software we use.
Personally I'm frustrated and I'm looking to learning more CS to get a job that values quality over quantity.
I feel that the Open Source ecosystem has made building software a lot easier.
Also, the sheer number of software engineers have gone up. 20 years ago those who chose to become software engineers where often very passionate about it. Today, it’s like any other trade, you get a larger variety of people.
My last point is that organisations have also matured and know more about what to expect from software engineers. When I started programming professionally I had a lot more time to finish any given feature, today everything needs to get out the door faster.
Overall, I think we are building better software today.
Sure there are some things that aren't emphasised as much but most of them aren't super useful. JavaScript, css, and mobile development are way more useful today than algorithms, cpu pipelines etc...
Most of the last round of unicorns you could build without any of the items you mentioned, but you couldn't even get started without an understanding of modern web or mobile development.
This is coming from someone who is by no means an expert on any of those things but took a bunch of courses on them in college and have only used them a handful of times in the last decade of professional software development to eek out very small performance improvements.
Doing constant progress (learning) both - modern and everyday useful things meanwhile not giving up at things like security, protocols, lower level stuff, hardware requires a lot of discipline.
“How one can be a car mechanic without knowing how to blacksmith and forge car parts manually, from metal ore”.
It's very lazy thinking to resort to a "kids these days" mentality of modern software development.
What has also happened is that tools have improved at roughly the same rate, and the tasks to which we apply software development have also grown exponentially. So there are jobs that exist today that did not exist 15 years ago, and they just don't need these skills.
I don't know about anyone else, but must of my time is spent wrangling enormous scripts in tools like CloudFormation to deploy relatively small amounts of code.
What's happened with JavaScript on the frontend is nuts though. I can't follow what those guys are up to anymore and can only assume they have a better understanding of caching, algorithms, etc with the explosion of code there compared with the days before even jQuery.
There are still plenty of people that actually do know the theory and actually do work on low-level stuff, but at the same time the reality is that the olden days of engineering aren't coming back; they are what is currently often referred to as the '10x engineer' type of work. It doesn't scale, it doesn't work well with others and it doesn't return on investment all that well.
Unless you need someone who works on hardware, kernels, compilers, runtimes or severely constrained constructions (query planners, memory managers, transaction engines etc.) it really doesn't matter as much as it used to.
I don't know if SWE is getting better or worse, but I do think that most SWE operates on a much higher level of abstraction than it used to, and I think that's good thing.
For example it could look like “Not enough revenue to pay all the devs”. But you only need a big team because it’s basically spaghetti code and there’s a lot of fires to put out.
I’ve seen this.
It’s also really hard to convince managers even technical ones who are not in the code that there is complexity to deal with. “But it’s just a [something that prima facie sounds easy] should be easy!”
A perfect example of this is looping over some range of pixels, and many will just go ahead and do for (x) for (y) setPixel(x, y, f(x, y)). There's no logic error here but it's still terrible when dealing with images in the usual memory order idx = y * width + x.
It seems like many people have no idea how even basic abstractions (such as the aforementioned pixel indexing example) work, and have no performance expectations because they don't code in any systems languages.
People who are aware of such issues and still can structure large codebases well seem to be getting more rare to me, at least. In the 90s there were so many incredible demoscene programmers, and now... hmm...
(A related thing I wonder about is, where is the von Neumann or Newton of our times? There are more people around than ever, nutrition and medicine and poverty is globally better than ever, ...)
Honestly, who has time these days?
There are many healthy activities to take up rather than play with deprecated/obscure tech. Not to be judgemental of course, I draw my line at home repairs.
There's a lot of things you can learn from writing demos that is still incredibly valuable. I wouldn't expect 99.9% of developers to know those things, but if I saw it on a resume? That'd totally jump off the charts to me and I'd definitely want to interview that person.
Don't shame expertise just because you choose to spend your time differently.
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
https://news.ycombinator.com/newsguidelines.html
I made no assumptions other than what he explicitly said, which was insulting and displayed ignorance of a community he clearly does not understand. My original comment was also only "People who care about their craft?", which was simply saying that people get into demo coding because they care about doing something worthwhile (getting the most out of their machines in an artistically satisfying and interesting way).
There's a common social anti-pattern in most tech forums of hand-waving away any complex domain knowledge that the person doesn't personally use as "useless". I have no regrets about stating the contrary.
I am amused that I am the one getting a warning about this though. I'm not the one that made a drive-by ambiguous comment that could easily be read as singling out a fun community as being a waste of time. I just pointed out that that's what he just did.
HN has been super impressed and favorable to the feats of demoscene programming for over a decade now, so hopefully there's a lot more of that than this.
This presentation (PDF) explains why: https://www.aristeia.com/TalkNotes/ACCU2011_CPUCaches.pdf
These parallel programming course notes have an example of using z-order curves which is slightly better yet: http://ppc.cs.aalto.fi/ch2/v7/
And the best solution requires something like Halide https://halide-lang.org/ to find the best traversal order.
And these things do make a difference. L1 cache is faster than L2 cache which is faster than L3 which is faster than memory which is faster than hard disk etc. You want your code to keep things within those limits to make things faster.
Or for example if you start using virtual memory and paging to disk you might want to switch algorithms. For example you might want to use merge sort instead of quick sort if you don't have a lot of ram and you have to go to disk. However if you have 128GB of memory and mostly randomized data you want to use quicksort.
This is kind of trivial example, but I think this stuff is somewhat important.
Most people who go to a bootcamp (I did not, but have hired a developer who did) do not end up working in a role that requires understanding of the 5 topics above you mentioned, with the exception of cache and maybe algorithms. It's just not what most bootcamps are targeted for.
I have seen a lot of terrible code from both sides, and don't believe the quality of the code to be a function of the developer's level of formal education.
I mean, you're using "gate keeping" as a pejorative, but, gate keeping is super important in any profession. Doctors and lawyers have a lot of "gate keeping" too, but would you really want to go to a doctor without a medical degree or a lawyer that's been disbarred? You might say what engineers do isn't as important, but then, if you're running a software business that employees 50 people, that business shutting down because their engineers can't cut it is a fairly impactful thing to a lot of people.
There's nothing wrong with having novices at work. We need novices and apprentices, they're the lifeblood of our industry. The problem is that our novices don't know they're novices, and now we have novices teaching novices and telling them that a lot of important stuff doesn't matter. Or you have novices hiring novices, and now you have bloated engineering organizations that take a ton of time and manpower to do things that should be simple, and the entire industry gets a black eye for it.
If you're saying you wouldn't hire a developer who didn't know those concepts in your domain, OK.. I don't know what your domain is. Software is a huge field as I'm sure you know and there are many domains in which the requirements include understanding these CS concepts.
If you're saying you wouldn't hire a developer who didn't understand CPU pipelines in ANY software engineering context, I strongly believe you would be missing out on some highly skilled & capable people.
If a driving school doesn't teach parallel parking, pointing out that deficiency is not gatekeeping.
> It's just not what most bootcamps are targeted for.
If you're making $35,000 a year in the service industry, making $70,000 translating Photoshop files into HTML is life changing. If making that transition is your only goal, great.
But I think many people enter bootcamps with more ambitious goals. They'd like to move up the career ladder, take on more responsibilities, tackle more difficult problems, and receive commensurate compensation.
People I've talked to who come from non-CS backgrounds said they hit a wall years into their careers, having to play catch-up on the job. In my experience, CS didn't help me as a junior engineer; by the time I was senior, those concepts were invaluable.
When people with CS-gaps hit a wall in their career, we have a two options: we can give them more responsibilities anyway, which sets them up to fail, or we can identify gaps early and help them. That's the opposite of gatekeeping.
I haven't and wouldn't suggest that people be moved to a position with responsibilities over their head If you want to move up and you need to learn more, of course you need to find a way to learn the required concepts. I did this and continue to do it.
My point about bootcamps is most people who finish them aren't out there getting jobs that require knowledge of CPU pipelines. If that IS happening, someone is really bad at hiring.
2. No seriously though - what you've written touches on CS and completely ignores designing programs. You can be great at CS and still write crappy programs.
3. Though it's unlikely to have an industry-specific crash, recessions are inevitable and then people (competent or not) lose their jobs.
4. I guess my whole point is - even if these thing were true, so what? Each individual person has the choice to go into whichever field they like. Each person is also free to spend as much time and effort as they want on improving their CS and software-engineering skills. This knowledge isn't somehow exclusive to degree holders - there are so many free resources on CS and SWE. And as I mentioned, the degree doesn't guarantee that you'll be able to create good software.
It's like asking if the quality of mechanical engineers has gone down because mechanics don't know x, y, and z.
Even so, the number of mediocre/terrible software engineers and web developers doesn't seem to have changed much overall. We all know of old school desktop software that was incredibly poorly written. We all know examples of games which were incredibly poorly written. There have been badly coded, badly designed websites from people with little experience in the field since the web first became a thing.
Plus people joined because it paid decently back then too. The dotcom boom brought in a lot of people who only cared about the wage slip at the end of each month. The early days of gaming had tons of people jumping on the hype train for money, whether it was the home computer scene in the 80s in the UK or the early console gaming one when Atari was still a big player. Or perhaps for every generation since.
As for whether they'll stick around if it crashes? It depends. It may very well not crash at all. And while a certain percentage will leave if it does, others will stick around and learn more instead.
So no, I don't think the quality has gone down. There have always been people from informal backgrounds, a knowledge of CS hasn't ever been necessary or mainstream overall, and mediocre to terrible programmers and developers have been a thing since the field began.
But perhaps with the demand for engineers really high, and the economy at full employment, and the big tech companies growing and also explicitly optimizing hiring practices for things other than "best SWE skills", more of the not very competent folks are getting into the places you don't expect to find them
> It feels like a lot engineers now days don't seem to have a good cs background. They don't see to understand things like cache, paging, virtual memory, cpu pipelines, algorithms or other things pretty fundamental to CS.
Pipelines and caches and paging and virtual memory are stupidly complicated in modern processors. If you claim to understand these things and you don't either work at the company or have an NDA with the company so that you can implement drivers, you're probably full of shit.
What I can't stand are the "highly-ranked" schools that introduce students to a very basic and abstract (and outdated) notion of these topics, and students enter the workforce overconfident that they have understood the topic. You haven't understood the topic, and having some rough notion of the topic can often times be worse than if you didn't know anything at all.
tl;dr: Modern processors are proprietary IP and you should be skeptical of anyone who claims to deeply understand it but doesn't work for the company making it. You do not need to understand how one works to be a great software engineer.
https://www.agner.org/optimize/microarchitecture.pdf
1. Intrinsic motivation to be productive and solve problems
2. Ability to communicate well, especially via specifications and Slack.
Anything else can be learned in a few months max.
Yes but nearly every software developer job is web stuff these days and in the web world, they don't need it. (Cue all the HN posters saying "college is a waste of money" and "it's just a piece of paper".) All they need to be able to do is glue together libraries and frameworks created by people who do happen to have good CS backgrounds. You really have to go out of your way to find jobs that actually require knowing "cache, paging, virtual memory, cpu pipelines, algorithms", etc.
> When we have the next industry crash (.com crash) will these people stick around?
It won't crash, because everybody needs a web site these days, but it will become commoditized because the bar to entry keeps getting lower and lower. I've seen some comments already on HN saying that software pay is gradually becoming bimodal.
Personally, I think even that estimate is too high; I'd be surprised if the true number wasn't somewhere closer to 5%.
Go search on job boards and fine all those mystical jobs.
They are the tiny minority.
You claim there's MORE than web dev jobs. Go to a job site, type in developer, and count the web dev jobs. Now count the non-web dev jobs.
Worse still, you're repeating this and replying to me even after ThrowawayR2 gave you a hard number to go off (a mere 13%).
Not only do you have web businesses, but also almost all enterprise apps are now web apps. I'd guess that those are the vast majority of jobs in the market, but again you probably won't believe that.
They're all web dev jobs because it's easy to deploy. 20 years ago, I used to work in places that had to roll out desktop apps to everyone. It was a nightmare, as well as a security nightmare as they had to have the db open to the whole network.
As for algorithms: for most of the stuff what people really need is to have a basic grasp of complexity of the common containers and time accesses to various parts of the system but beyond that–just learn on the go.