this is the kind of thing you say during an interview, get rejected and then wonder if you just experienced ageism instead of any introspection about your own rant
A charitable interpretation of the article is of course a balance is what actually matters. It is good, on some level, that abstractions free up developers or whoever to focus on the business instead of just tiny little details. Their example, however, proves you need escape hatches sometimes, and you need to have a mind for the details at times, and right now the balance is far in the "abstract everything" direction which has its own issues.
I think you might've missed the point a little bit. It doesn't read to me like "abstractions bad", it's saying if you're going to use an abstraction you should learn how it works (and that we've built so many that nobody bothers to learn them, which is true).
I know Javascript, React and Express like the back of my hand. I can solve absolutely any problem I've run into with that stack, I can't even remember the last time I was stuck on something. But I've ended up on like 5 projects in a row with stuff like Typescript, NextJS and NestJS because they are red hot on the fad scale right now. Despite that, I still don't feel like I know much about them because I set that bar reasonably high, and out of all the people forcing me to use tech I don't want to use, none of them even come close to reaching that bar.
I'm sick of asking fairly basic questions about frameworks and libraries I'm not as familiar with, and getting back "oh, I dunno, have you tried...?" 15 times in a row before solving the problem. That's not an efficient way to work, no matter what 90% of the industry seems to think.
In the example at the end, people don’t necessarily need to know what files and folders are so its not bad that they dont know, it’s better for a new developer that they have a blank slate to learn about trees of storage paths, as the skeumorph is outdated. just like the save icon being a floppy disk was so outdated that instead of replacing it, people realized we dont actually need save icons anymore and everything should just always save
No, this is the kind of thing you say during an interview, get rejected, and then realise you were probably not going to like the work environment anyway.
That’s cope, none of the flags in an interview have anything to do with anything, especially after taking enough jobs to you realize that you’ll probably get called in last minute to interview additional people with no preparation whatsoever on the interview process so you just google a couple things to ask and wing it yourself
I feel like if I was alive and working 60 years ago I would have been saying the same thing about mechanics, construction, and analog electronics. But really it doesn't matter for most jobs. Knowing how things work underneath is needed if you are forming your own business and products from the ground up, it is only slightly useful if you are working for somebody else, and as time goes on what monetary value there is left in that knowledge will evaporate to nothing for 98% of workers.
Yet, someone has to know this stuff, or be capable of learning how it works and as a consequence be able to work on those arcane layers. Otherwise we stand no chance of finding new, more general abstractions to construct a stack of fewer abstractions in total. We probably won't get to fewer abstractions, but to avoid making the problem get worse even faster, we need some counterweight.
Sure most people don't need to understand this stuff.
But the problem today is that very few people, in tech not just the general public, understand the low level stuff, especially among new people entering the field.
Furthermore software abstractions tend to be far more leaky than abstractions in the physical world you mention. And when they leak you need someone who understands what's going on under the hood.
Actually I don't think this knowledge is most important for people forming their own businesses at all - they have many other problems to sort out. But I think it absolutely is valuable working for someone else. These people are the "goto guys" and, ultimately, a "technical insurance policy" for the companies they work for.
A company producing tech products without at least a few people on staff who can dig into and fix almost anything is in a difficult position ultimately.
There are some things you need to know though. For example most devs today "know" Docker, but very few realize that it opens the exposed ports on the host firewall. Exposing a port for another service to access internally is intuitively safe, until you look at the box and realize that same port was opened to the internet. Convenient thought the docker devs, but touching something so important without feedback is reckless. That's an abstraction that absolutely shouldn't be there, too many devs don't RTFM.
I support the notion of this post. In the last 6 years I’ve mostly been busy removing layers of abstraction in order to uncover the set of tools which is a good balance for me.
In example, replaced clojurescript with Javascript and then eventually Typescript.
Replaced Clojure with Java.
Replaced docker with VMs.
Avoided ansible in favor of simple bash scripts.
Avoid all kinds of firewalls in favor of understanding and using iptables directly.
Etc..
When it makes sense, additional layers of abstraction can always be added on top, if the foundation is solid.
You can build infra with cloud-init to achieve something similar. You start with base image (e.g. debian cloud-init image), you craft cloud-init script (like Dockerfile) and you end up with VM disk file which you can run.
It won't be better than docker, for sure, but if you're going to use VMs and stay sane, you'll need to reinvent it.
I don't know how others feel, but to me docker is definitely less complex internally than a VM. For one, I'm running a single kernel/set of drivers rather than two.
But operationally yeah I agree, the stricter separation could make it easier to use a VM to get stuff done.
Just to clarify a bit. A VM is already a container.
When you run docker inside a VM it’s another level of abstraction.
Yes, one can run docker containers on bare metal, however the isolation is poor and so is the security guarantees.
In terms of excess abstraction, with VMs or bare metal you just need to learn the essentials that you need to know anyways like for example linux networking and security. With docker there is additionally container networking.
Think about this from the perspective of the post. For us since we do use React CLJS and it’s array of libs did became an excess level of abstraction. A wrapper around the JS engine and JS ecosystem - which didn’t bring enough added value for us to justify its use.
Typescript didn’t attract me at first but eventually it does bring a lot of added value on top of js and it’s 90% JS anyways.
We build hardcore SPAs and the amazing tooling and libs for ts pay off.
Abstractions are like tech debt. When you consume an abstraction, you also get all the klocs (thousands of lines of code) that it represents, they have a price. When you have multiple sources (as in suppliers) of that abstraction this greatly lessons the risk. As does have an abstraction which is total.
The problem with the abstraction tower is not that it is a tower, but that it is square, when you can't tell where you are relative to the abstractions above or the abstractions below, then your abstraction gradient is too small and you are just generating mush. We have too much mush.
Way too much mush. And when questioning the mush, some people get bent out of shape because being critical of the temporary constructs that have collapsed “brings people down”.
I somehow agree with this. As a web developer who started on a framework first approach (Vue + Django), I was having one hell of a time trying to figure things out because of my lack of fundamental knowledge. I think abstraction is okay but you have to understand that just because you can make abstractions doesnt mean you should.
A young colleague of mine who has also started "framework first" with Vue + Django was recently confused about what a "serializer" was. They have "written some" within Django REST framework, but he was confused about their purpose.
I had to explain the problem with a single wire signalling bits in a series, recipient having to de-serialize them into some data structure. Then I had to explain that TCP emulates such a single wire using small packets.
I think that they have understood, but it was a funny feeling explaining this to someone who routinely deserializes form and JSON data, then serializes them into SQL queries, then deserializes query results in order to serialize them into templates or JSON.
I think there is one interesting angle to this problem.
I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge.
As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to the problems and logic behind all those new solutions. Understand how we come to have JavaScript and the browser mess we are in and so many other curious features of todays digital world.
I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc.
The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path.
For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option.
We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals.
There will always be hyper-nerds and just super smart people who will study all of the low level stuff. Granted, they will be vastly outnumbered by high level programmers.
I am continuously hiring new people, mostly senior devs and tech leads. Maybe one in a hundred has any understanding of what virtual memory is. Or why one process can't access memory of another process and, if you really want it, how to set it up.
Two decades ago that was common knowledge.
It went away just as the knowledge of how networking works. Currently, if they try to open a connection and it does not work they are pretty much at a complete loss to what happened, where and how to fix it. Aside maybe from a simple problem like DNS resolution error (and not even that -- most devs don't understand how DNS propagation works, for example).
So if those problems happen, they are mostly deferring to me as a guy who can solve every problem. I see no clear path to achieving a goal of having someone else to learn to do the same.
The thing is as someone who twenty plus years ago was able to write the webserver at socket level and so on, if you handed me a keyboard today I have forgotten all the context, the skills - they have pages out of my brain and now have redis and javascript and hadoop.
But you know those things exist and how things relate to each other and affect each other, or could affect each other, and you don't need to already have a mental text book in ram to solve a problem. I don't know you and yet I know this.
One little-appreciated fact is that many of the most accomplished and knowledgable engineers were tinkerers from an early age. If you start your computing journey at the age of 18 with a compsci degree, you might already be a decade behind some of your peers.
It’s not too different from pro tennis players, who typically start playing before the age of 8.
For those at the top of the game, computing is a calling as much as a job. They will do substantial learning on their own initiative.
You might be a decade behind, but there's still diminishing returns kicking in hard even just a few years in. That's disregarding inefficient learning and what else which may close the gap further.
Your latter point is far more important to the matter. Those who treat it as a passion more so than a job, are more likely to be the trendsetters. Growing up and being free from responsibilities makes it easier for that thing to become your passion.
And let's also not forget, a few decades ago, computers weren't exactly a cheap thing for parents with little understanding to let their kids tinker with at will. Being born in a family with enough wealth to get a computer, enough wealth / understanding to let a kid tinker with it, was an immense boon. A long with anything that type of family tends to have going for it alongside wealth. It's not that far-fetched an idea that it's the other things, rather than the early age interest of the kid itself, that got them into such a position later in life.
How often does a front-end developer need to know about /proc/pid/mem?
I think it's important to keep things in perspective here - when I took my first programming job the first week was setup. On my team, our onboarding has you branch, review and merge on your first day. Those abstractions allow for people to focus on the areas they're working on, and iterate there. A game developer doesn't need to understand the complexities around js type conversions, any more than a js developer needs to understands rendering thread latency in a browser.
On every team I've ever worked on, that headspace from the team would have been much better served by understanding the product and project, IMO.
Great question - is their app deployed to a container? In that case, they should know about it, because buried deep under layers of abstraction are cgroups. Answering questions like “why did Node try to use 32 cores” is much easier when you understand this.
Unless they're working in that space they won't know it. And that's fine. There's enough to keep in our memory and tools and resources for us to look up information if we need it.
>It went away just as the knowledge of how networking works. Currently, if they try to open a connection and it does not work they are pretty much at a complete loss to what happened, where and how to fix it
also, devs seem to not understand that a computer network is basically a massive distributed system (on the routing level anyways), and that things inside this system can and do fail nearly constantly.
things like latency, endpoint connectivity and available bandwith is not a fixed given!.
So many devs in my experience seem to write code which assumes the network is always available with the same characteristics as the point in time in which they wrote the software.
Not to even mention devs are even entire companies building HA mechanisms which rely on ethernet connectivity to work, making it very hard if not impossible to stretch them properly across network segments/locations without stretching ethernet. (which brings its own set of problems).
I feel similar, I was a Linux Sysadmin in the early days of Gentoo, studied C and Network programming, configured hardware and RAID arrays and did some courses like "From NAND to Tetris". Over the years I did a LOT of programming.
When "the cloud" came out, I could easily understand the magic behind it all, so it made a lot of sense to me and it still does.
I feel "computer literate", however, even a lot of solid but younger engineers I work with struggle with lower level concepts and just don't have the knowledge and experience with things like networking protocols to be able to solve hard problems.
There will be an increasingly important history component in comp sci., software engineering, infosec, and associated fields. I think this is a place where the IT field has done a bad job and, as a result, we've had unbelievable amounts of "wheel reinvention".
I'd love to hear from somebody with experience in scientific or other disciplines and IT who could contrast their fields with ours. I only know that I have had to go out of my way to pick up a historical component to my IT education, whereas it seems like a historical foundation is part of traditional math, science, and engineering training.
Vernor Vinge was right, though-- software (and hardware) archaeologist will be a real job.
I've heard that advanced math has a similar problem, that some subjects are quite deep that only a handful of people really understand the topic and passing the knowledge down to younger mathematicians is not a given.
I guess in IT it's the other way around, where you can start with advanced abstractions and frameworks without bothering what kind of rabbit hole is underneath (at least while it's going smooth), it gets deeper as you go closer to the hardware. While in maths you need to understand the basics to even know what's going on with more advanced areas. And this already applies to the fundamentals too, where in school it's expected you understand the stuff from previous years. At least that's my impression as a non-mathematician.
And then there's the mix between the two in languages that rely heavily on powerful type systems.
No, you can do the same in math. It just means ignoring the proofs and just using the theorems as they are. However sometimes the application and proof requires similar ways of thinking, and you kind of miss out by skipping the proof. But the same can be occasionally true for software engineering.
Are there good historical examples of this happening, where foundational knowledge is completely lost?
I know I have read about it in books as a fiction, but I assume it must have already happened here before.
Maybe its an oxymoronic question to ask, seeing as if its lost we might not even know its lost, but more in the vein of "We put the lime in the mortar because this is what we have always done", unaware of the actual properties of lime when interacting with concrete.
>Are there good historical examples of this happening, where foundational knowledge is completely lost?
Usually following the fall of empires. Ibn Khaldun (1332-1406) wrote:
"Perhaps they have written exhaustively on this topic, and their work did not reach us. There are many sciences. There have been numerous sages among the nations of mankind. The knowledge that has not come down to us is larger than the knowledge that has. Where are the sciences of the Persians that ‘Umar ordered to be wiped out at the time of the conquest? Where are the sciences of the Chaladaeans, the Syrians and the Babylonians, and the scholarly products and results that were theirs? Where are the sciences of the Copts, their predecessors? The sciences of only one nation, the Greeks, have come down to us, because they were translated through Al-Ma’mun’s efforts. He was successful in this direction because he had many translators at his disposal and spent much money in this connection."
> Are there good historical examples of this happening, where foundational knowledge is completely lost?
Maybe not quite in the context you were asking, but philosopher Alasdair MacIntyre argues as the premise in his book After Ethics that this is what happened to the philosophy of ethics after antiquity.
Copypasting from its Wikipedia article [0]:
> [After Ethics] begins with an allegory suggestive of the premise of the science-fiction novel A Canticle for Leibowitz: a world where all sciences have been dismantled quickly and almost entirely. MacIntyre asks what the sciences would look like if they were re-assembled from the remnants of scientific knowledge that survived the catastrophe.
> He claims that the new sciences, though superficially similar to the old, would in fact be devoid of real scientific content, because the key suppositions and attitudes would not be present. "The hypothesis which I wish to advance", he continues, "is that in the actual world which we inhabit the language of morality is in the same state of grave disorder as the language of natural science in the imaginary world which I described." Specifically, MacIntyre applies this hypothesis to advance the notion that the moral structures that emerged from the Enlightenment were philosophically doomed from the start because they were formed using the aforementioned incoherent language of morality.
It is all things. Encapsulate all things. Think fractals, hopf and always remember the turbo encabulator.
The original machine had a base-plate of prefabulated aluminite, surmounted by a malleable logarithmic casing in such a way that the two main spurving bearings were in a direct line with the pentametric fan. The latter consisted simply of six hydrocoptic marzlevanes, so fitted to the ambifacient lunar waneshaft that side fumbling was effectively prevented. The main winding was of the normal lotus-o-delta type placed in panendermic semi-bovoid slots in the stator, every seventh conductor being connected by a non-reversible tremie pipe to the differential girdlespring on the "up" end of the grammeters.
— John Hellins Quick, "The turbo-encabulator in industry", Students' Quarterly Journal, Vol. 15, Iss. 58, p. 22 (December 1944)
no I'm talking about a manned mission to the moons surface including all the cameras and "live" coverage as well as the lander. apparently that tech is lost.
With the tech and politics of the late 1960s it would have been far easier to actually land on the moon than fake it with the vast reams of evidence we have.
I think that's essentially what the medieval dark ages were before the renaissance revived historic cultural and intellectual achievements and began building upon them again.
Since you’re citing fiction, have you read the foundation series? The original inspiration for that book was The Decline and Fall of the Roman Empire, which is a history book (arguably the first modern history book).
Anyways, that just to say that people in post-Roman Empire Europe lived out the reality of lost knowledge. They lived with the remains of incredible art, architecture like the coliseum, public works like the aqueduct, etc. But they wouldn’t have known how to reproduce those works. How to make concrete, etc. was knowledge that was lost to them.
We lose knowledge all the time. Every person that dies carries a lifetime of experiences and information.
The thing is, we tend to lose the knowledge that is deemed useless. Anything considered useful is well spread. The problem is when our opinion doesn't match the reality, we have something around a generation to prove it, or the chance is gone.
Mathematics is full of examples of things that were discovered again and again, until somebody found a use for them and they entered our textbooks.
Analogue electronics design has been almost entirely replaced by digital micro-controllers to the point that finding old-school electrical engineers that can design non-digital circuits is becoming a challenge.
I believe parent is referring to design at the PCBA level, where this is mostly true outside of military/government designers who lag.
The reality in that sphere is there is little reason to limit oneself to the constraint of expertise in analog circuit designs when one can achieve the same functional outcome digitally, and use those skills more broadly.
Only where component sourcing is artificially limited, or risks of digital operation sufficiently large, and where the job market will support it, does it make sense to proceed in growing in analog circuit design expertise.
Meanwhile these designs are nearly all being functionally lapped by those in general industry, which participates in all the accelerating gains of digital technologies.
> There will be an increasingly important history component in comp sci., software engineering, infosec, and associated fields.
Much as I'd like that, I disagree.
The momentum has always been against that. We're members of a cult of the new, which does not value or even acknowledge the old. Which is why we keep reinventing the same solutions but with different tech stacks.
This drives developer value down which drives developer salaries down, so I don't see it changing.
FWIW, my history of math class as an undergrad was a math elective that almost nobody else took. Other math students didn't want to learn history and other history students didn't want math. And the curriculum stopped at the 18th century, before things got really interesting.
> We're members of a cult of the new, which does not value or even acknowledge the old
This just isn't true. We have Heartbleed because we refuse to move off how we wrote software 30 years ago[0]. HTTP was invented in the 1980s[1]. REST in 2000[2]. TCP/IP in the 1980s[3]. Ethernet the same[4].
I'm pretty sure we acknowledge and value all of those things. Our field, depending on how you define it, is only about 70 years old. Those things are pretty old in those terms.
>I think this is a place where the IT field has done a bad job and, as a result, we've had unbelievable amounts of "wheel reinvention".
Some of this just seems to be due to the maddening popularity contest of technologies and frameworks within companies. There is a frantic rush to make sure you're doing everything that your competitors are doing, but not much introspection regarding whether things will actually help your company. You're just keeping ahead for the sake of keeping ahead.
I couldn’t agree more and it’s something I didn’t understand at all until my final year at university. I was talking with one of my professors and complained about having to learn and use SML and Eiffel for some projects when in the real world, I’d be using something like C (this was 1994). He asked me if I was there for an education or training.
I think today it’s probably more true than ever. Schools are under pressure to turn out grads who will be productive on day one and I think it’s shortsighted.
The problem was that there was a shortage of developers, and that most universities still only had CompSci degrees, not Software Engineering degrees, coupled with those degrees were 3-4 years long... So enter the code bootcamp, where you're basically given a hammer, and taught that every problem can be solved with it..
Now those bootcamps aren't all like this, and there were and are good ones out there..
...but from what I've seen (I want to stress that this has happened rarely compared to the rest) a graduate of one of faster bootcamps, that learned a single language, like JavaScript, and try and solve every problem with whatever answer they found someone else upvoting.. without any amounts of thinking if it's the right solution for this specific problem.
The problem is as your professor said. Do you want an education or training... And also as you said, that you can't turn out effective software engineers with a quick or short amount of education.
"and that most universities still only had CompSci degrees, not Software Engineering degrees"
Is a Software Engineering degree really a thing? Perhaps outside the US? Can you point to a notable university that has this major? A quick search gives me the impression this is an online correspondence program type of thing.
At UCSD in the late 90s we had a Software Engineering class but that was it. Otherwise internships were supposed to prepare you for your careeer.
UT Austin has an undergrad Electrical and Computer Engineering program, which you can specialize into Software Engineering and Systems. That’s about as close as you’ll get, I think, and since it’s ECE you’ll also have a fair amount of hardware experience - not a bad thing IMO.
They do have a full SWE track as a Master’s program, which I did, and quite enjoyed.
My state university (UMass) education's CS degree had us working directly with CPUs and logic gates on breadboards (made an ALU!) We even designed our own x86-like CPU ISA to be run via simulation with our own progams. We built and programed robots from discrete components and microcontrollers written in assembly. Made our own rudimentary OS, including threading with mutexes and semaphores. Built our own pine-like terminal email client from scratch. Created design docs for our own games and executed on them in sprint teams throughout the semesters.
I cannot stress how important my education was to my current understanding and success as a software engineer in the field. A poor education is a huge detriment. When I joined the workforce I found many colleagues without one ill prepared for some of the basics, like debugging kills, using tools outside their domain expertise, etc.
That said I don't think a formal education is REQUIRED, but without one you need a certain kind of perseverance and thirst for knowledge not everyone has. I've had the pleasure of working with several folks in that category, so a formal degree isn't for everyone.
Today, it appears specialization is happening much earlier in the education process without a general foundation. Specialists will always be needed, and larger orgs can get by just fine with exclusively specialists, as long as their communication skills exist to prevent silos. Larger companies can afford this. However a good generalist can replace a team of specialists when the latter budget is out of the question; a huge win for smaller orgs, start-ups etc.
The old joke that a good generalist is the "jack of all trades, master of none" has some truth to it, but as with all things, it's a tradeoff. A team of qualified specialists will definitely produce better work within their domain expertise, but at a cost that can be prohibitive when "good enough" is within reach.
In some countries, e.g. Portugal, this is sorted how by having two university systems, and technical schools.
One wants to go straight into the work market, with focus on getting the bases, technical school.
One wants to go into the work market, with some more preparation than only the bases, polytechnic institut.
One wants to go into the work market, with overall experience, in a degree certified by the engineering order as qualified for professional title certification exam, university.
My software engineering degree had such information, that is how eventually I went from a UNIX/FOSS zealot, to someone discovering the alternative universe of Xerox PARC, ETHZ and having the seed to research the department library about systems programming languages of the past, since Fortran came to be as first alternative to raw Assembly.
Unfortunately not all degrees, or universities are well prepared to offer this, or have staff that cares about making it happen.
> Vernor Vinge was right, though-- software (and hardware) archaeologist will be a real job.
Spend some time in the coin op game community... This is all we spend our time on. Well that and cabinet repair.
But seriously though, keeping these old games alive requires serious reverse engineering of both hardware and software. Especially if they have custom chips, ASICs, undocumented features, etc. But the work has to be done or these games will be lost to time... It may be a passion project/ hobby, but it's still real work.
I like your point about digital archaeology. If it's important enough to some people to keep old arcade games running, how much more important is it to understand and keep some old mission critical computer running.
It's a lot like the tech people you see in these dystopian future movies, the ones who are like "oh yeah you have an old z80 with the qxp interface, haven't seen one of these things in years". Like Scotty from Star Trek. They're an increasingly important part of the tech ecosystem.
Having been working on writing documentation, tools and homebrew SDKs for arguably one of the most influential arcade systems ever made [1], I could not agree more with this. Hardware and software gets left behind by the original developers all the time, but that in no way makes it no longer relevant - and it certainly does not make the underlying concepts outdated. Z80 or 68000 assembly language might be completely useless to a web developer, but shall we call it "obsolete" when there are millions of these processors still around running critical infrastructure? Shall we really claim that learning to manage memory manually is futile since GC languages exist?
Besides, the PlayStation based hardware of this specific arcade PCB is not that far off from the ubiquitous 32-bit microcontrollers that underpin modern digital society; in fact the MIPS R3000 derived instruction set even bears a striking resemblance to that newfangled RISC-V thing every investor is talking about. And yet most of my 19-year-old university classmates want to stay away from "scary" embedded or kernel work in favor of web or game development.
> And yet most of my 19-year-old university classmates want to stay away from "scary" embedded or kernel work in favor of web or game development.
It seems that with more abstraction, more people throw up their hands and say "I'll never understand it", "The designers of this system did things in the stupidest way possible", "It's magic" or (in the case of the financial system) "It's a rigged game" "You're working for the man"
What all of these points of view have in common is that the people who hold them have given up trying to understand the world around them. Instead they take the intellectually lazy way out and compose their own pet theories of why things are the way they are, this is where tinfoil hat stuff comes from. This is dangerous, and is sort of the point of the original article.
I keep telling people that "the world is still knowable, still understandable". You have to put in the time and learn about it. It's easy to do, you just have to do it. There are plenty of "average" people playing in these "difficult" to understand fields.
What we desperately need to cultivate is a culture that encourages and rewards curiosity. When people are curious, they take the time to learn about the world around them, and why things are what they are.
If you don't master the world around you, it masters you.
That's certainly romantic. The real question is how much money - how much of your personal peace - you're going to give up to make space for that culture. Even if it's just putting yourself in conflict with rich people to convince or force them to pay, your asking to change the status quo when a lot of people are quite comfortable with the way things already are.
Same thing in the retrocomputing community. The video series Usagi Electric did with the Centurion minicomputer is a good example. The degree to which data about even off the shelf parts have been lost to time is frightening, let alone software.
I wouldn't call IRC decentralized, but yeah, all the new messaging systems like Slack/Discord are basically proprietary versions of IRC with only one (shitty ElectronJS) desktop client.
A friend of mine (who coincidentally once shared an office with Vernor Vinge...) remarked to me early in my career that CS programs do a terrible job of teaching the history of the discipline. Students end up not knowing the failures (or even the successes!) of the discipline, and so end up rediscovering old new things. Makes a lot of software and systems practice more faddish and hype-driven than it needs to be.
Do you have any ressources to share ?
I am a self taught developer, I do stuff at my level (typescript, lisp), also because when you work alone, such high level languages allow you to express things nicely and fast. I kinda know how a cpu works, I understand what is going on when you perform some bare metal programming, but I never barely went down the rabbit hole and performed some low level stuff.
I'm in the situation where I don't know what I should learn to be on par with graduates/undergraduates students. I am searching ressources about networking, operating systems, or anything that could enhance my proximity with low level stuff.
I'm also self-taught. I've been doing it since (roughly) 2009, and like many others I started at the highest level of abstraction with front-end web dev. I've worked with some mid-level languages like Objective-C and Java, but like you I've never really dug deeper than that.
I know this is tired and cliche at this point, but literally this week I sat down with ChatGPT and asked it to teach me how to write WAT (web assembly text format) so I could understand how memory is managed at a really low level (but not so low that I risk crashing my computer).
It turned out to be super valuable. This is where AI shines for me - I can ask it any question that pops into my head, and also validate whether what it's telling me is true by running the code and seeing whether it works. It was amazing.
If you're curious, I'm fine sharing a link to the chat:
Yeah I was running all the code to validate what it was giving me, but I also try to subtly prompt it to repeat itself so I can see if it’s being consistent.
I’m sure it’s not error-free, but the speed of learning makes up for it IMO. Like I’d been struggling for a while to learn tree-sitter, the documentation is overwhelming. I had a chat with AI and got a working solution for my problem in probably 4-6 hours, and now I can write tree sitter grammars without help. It’s really incredible.
For very foundational stuff, Charles Petzold has put out some great books.
His book Code is fantastic. He starts at simple battery and lightbulb circuits and builds and builds towards a simple CPU.
He also wrote The Annotated Turing which is a breakdown of Alan Turing seminal paper and you only need high school math to get through it.
When I was in school my favorite course was compiler design and we used Compilers: Principles, Techniques, and Tools (aka the dragon book). It’s one of the best textbooks I’ve used but that was 30 years ago. There might be something better now. Understanding parsers and lexers and (especially) state machines is something that will serve you well.
Ah, I'm interested in parsers these days (I just wrote one for parsing org data, I am a bit unsure about the architecture of the project but at least it does what I needed personally).
I am right now at a friend's place that just showed my the dragon book from its shelves, that's a funny coincidence.
I will check out Code.
(Most of the content is not actually specific to Python)
He beautifully pulls back the curtain on so many lower level concepts like virtual memory management, dynamic linking, heap/stack, fork(), copy-on-write.
The talk is broad in nature, not deep. It takes you just below the surface of many magic black boxes, and, as you put it, enhances your proximity with those topics.
For me, so many things clicked in this single talk:
- How virtual memory works (incl. paging in/out, swapping)
- Why there's those discrepancies between RSS / PSS
- What segfaults and page faults are
- What actually happens when I get errors related to dynamically linked libraries, either at build time or runtime
Godbolt [0] is an invaluable resource. But simply setting up tasks to yourself and completing them may be the best course of action. Then you'll find whatever resource you need for a concrete objective.
For example, if you have a week, I'd suggest to start "in the middle", and move up and down according to your tastes.
- Write a hello world program in C, compile it and run it. Try to compile it statically and understand the difference.
- Ask the compiler to produce an assembly file. Try to build the executable from that assembly.
- Try to write a hello world in assembly language by yourself, compile and run it.
- Write a program in assembly that prints fibonacci, or prime numbers.
- Now, forget about assembly and move upwards. Download python source code, compile it, and run a python hello world with that interpreter.
- Look at the Python source code, which is written in C, and try to follow what happens when you run your hello world.
- Try to change the "print" function of the python interpreter so that it prints your string backwards.
Depending on your experience, you may need more than a week (5 days) to complete all these steps. But that's OK! In a few months you can find a new spare week and continue your work.
Unless you're two hundred years old, you joined the development of this technology in the middle. There was already decades of progress in electronics and telecommunications theory and engineering that everything you listed was built on top of. Your predecessors likely thought it was all newfangled abstractions that obscured more fundamental knowledge, and your successors will think the same about whatever world they grew up in vs. whatever one they leave.
That's the kind of argumentation the article argues against. There's a middle ground. Or, if you prefer, there's an optimum somewhere, and right now, we're on a descending slope.
To build mobile phones, we need a lot of tech built on tech build on tech. Dwarfs on giant's shoulders. And there's a place for people who don't understand the underlying tech. I've got a colleague who doesn't understand any of it, yet can do useful frontend work. But when you stand too high above the ground, the problems in the article become real, and that will get us stuck.
Abstractions are only relevant when they are leaky. People alive today were flipping toggle switches to load individual machine codes into memory. Some of them literally knew what every single transistor in the machine was designed to do.
Programmers at the time went from thinking these kit computers are toys to watching all the old ideas like virtual memory, cache, pipelining, networking, and multiple cores get reintroduced as the anemic transistor budget exploded. None of it was particularly novel, but the slow introduction of older ideas was a great way to learn all this stuff from the ground up.
I don't need to know assembly to write a web app - it's totally magic, and it's fine.
However, if I am using an abstraction like an ORM without understanding what is happening with the database under the hood, or why it's getting slow all of a sudden, it is bound to bite me in the ass.
I shudder to think of how much money and energy the world is burning every second just because of this one alone.
A client of mine is enduring growth induced ORM hell right now. The struggle and existential threat is real.
It can take days for them to even track an evil SQL query back to the actual code. There were fetch-one calls that literally carry a million rows back over the wire for that one returned entity. Indices get skipped due to the magic translation of a row.col.toUpper()=="STRING" into SQL.. when the db collation is case insensitive in the first place.
But hey, the devs don't need to know SQL, so there's that.
Exactly. How far should we take this? Should you learn how to build a processor from vacuum tubes before understanding the abstraction of silicon? Build the power plant to power the whole thing and lay the wires?
Specialization always exists in an industrial society.
One mitigation of the “mastering computers takes 30 years” thesis is that many of the abstractions and technologies that we spent those decades learning are now obsolete, and newly minted grads are blessed with learning materials and environments that make catching up easier than ever.
I'd look at it in the opposite direction. Good abstractions are powerful when you understand what they build on. That recurses all the way down. Learning bottom up is a natural path, but there is still plenty of opportunity to explore top down. The difference is that top down discovery needs to be an intentional journey.
>We, people who really understand the technology that the world runs on, are a slowly dying breed.
And web development has the worst tech abstraction of all.
There used to be a time where any Software Engineers ( i think the term wasn't even invented then ) or simply programmers would know at least a thing or two about Hardware. We now have so much abstractions we have people working in tech who have zero knowledge on either software or hardware, or any low level stuff.
That is why not only do we need some open source or open standards ( I am not a zealot for everything open sources ), we also need to simplify everything we have today. To basically refractor everything we have learned from Hardware to Software.
To be fair, its hard for anyone to know anything about hardware these days unless its a company with a large on-prem footprint and it doesn't make financial or regulatory sense to put in the cloud or co-locate, which is a vanishingly small number. Hell I haven't been in a physical datacenter in 5 years and that's where I started my career many moons ago.
They know only full SPA frameworks, they have never seen a dump of an HTTP message, headers and verbs are abstract things to them. Hell, many of them don't know you can have fully functional websites with zero JS, including payment, video, login, etc.
I started to write an HTMX tutorial (https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1) because I noticed a lot of young coders don't understand what to do with it. They read the tweets saying it's nice, but when they look at it, it makes no sense to them.
It's really fun because I now remember how some senior coders looked at me, knowing nothing about compilation. I was struggling with Python packaging because before wheels, it required compiling a lot on linux, and it failed often. For them it was obvious: just install the headers, look you need the dev packages, wait, you don't have gcc?
Nowadays I happily patch nginx source code and compile it manually, but it took a lot of work to learn a minuscule chunk of what all those guys knew by heart.
>You can actually see this with new frontend devs. They know only full SPA frameworks, ......
Around I guess 2017 or even 2016? I used to think this is some sort internet troll comment about people never seen a dump of an HTTP message, until......
>Because they started with assembly.
It wasn't necessary starting with assembly or something low level. ( Although that certainly helped ) We have less entertainment, more time, and no Internet ( for most part ). Things also dont work all the time. And we have to spend time figuring it out ourselves. That is where all the knowledge comes from.
Having to figure it out is not specific to a generation. My generation had magazines and forums. Then came out google and SO. Now chat gpt.
The difference is where you start figuring things out.
Before, you needed to figure things out at your level, because it was the immediate area of mystery.
Now this level is generally solved, you need to figure things out at a different level:
- filter out the mass of irrelevant information, the out dates one, and the one from spam
- understand how all those complex abstractions interact. We have good resources on how each work individually, but the carthesian product of the monsters we build with them, we obviously don't
- debug some things that don't work when the magic fails, way below your level, or behind a service, where nobody is looking
The problem before was scarcity of info, lack of standardization and roughness of systems.
Now it's abundance, opacity and too much sophistication.
But everybody still have to figure things out. Just no the same challenges.
It's fair to say that these are different kinds of "figuring out" though. Learning by trial and error is distinct from learning how to find and synthesize information and each lead to very different outcomes over the long-term.
I'd argue that the "find and synthesize" generation have an advantage within contemporary software paradigms because of their experience but, without deep knowledge of the foundations they are building on, they might be disadvantaged when it comes to imagining/creating new paradigms.
Then again, first-order thinking seems to be easier when you're not marred with the traditions and conventions of the past so maybe this isn't actually a disadvantage.
Trial and error is a fundamental part of how we work as human beings. There is no learning or understanding without it and no "figuring it out" without it. So that dichotomy between "trial and error" and "find and synthesize" doesn't exist. They are the opposite sides of the same coin. You can't have one without the other.
Finding and synthesizing doesn't do you any good if you're unable to meaningfully apply it or understand what you're applying and what you're applying it to.
Trial and error also doesn't do you much good without the ability to back it with knowledge and to find the relevant knowledge.
Agree with everything except out of date info. The old stuff is the good stuff you can’t find anymore because Google thinks you don’t want it. the newly generated SEO ai shit is what you need to filter out
Judging solely by the date of publication is not the best criteria for filtering. For example, Vannevar Bush's article "As We May Think", published in 1945, is the oldest dated item in my reference library. Conway's Law comes from Melvin Conway's 1968 paper. David Parnas' 1972 paper "On the Criteria To Be Used in Decomposing Systems into Modules" still applies.
The "less entertainment" and "more broke things" is probably the critical piece of the environment.
This makes valuable anyone with long attention spans and immunity to boredom.
For me, it has taken some habit forming (and habit prevention), but I have learned so much in the past 10 years since giving up gaming and social media. HN is an occasional vice though...
Speaking as one of those guys, the first computer I programed on was a Timex 2068, a reality check is discussing industry events and noticing a bunch of clueless faces among junior devs.
> they have never seen a dump of an HTTP message, headers and verbs are abstract things to them.
When I was teaching programming, I had a fun party trick whenever we got to HTTP. I'd fire up netcat (in listen mode), then connect from a web browser and "serve" a website by hand. I'd show the students the HTTP request that came in, and just manually type out a simple HTTP response and they'd see it appear live in the browser. Its pretty magical.
And once I'd shown them that, I'd write (by hand) an HTTP request to wikipedia or something to show them how its symmetrical.
Of course, real websites increasingly do HTTP2 over TLS or something. So unfortunately its not as "real" as it once was. But if wide eyed look on my students faces is anything to go by, it was a great lesson.
Not quite the same, but I remember sending emails via telnet. I showed that to a younger dev who was somewhat blown away by the fact it wasn't via some REST API.
Yes, I actually remember another tech showing me how to type "get / http(whatever)" into telnet to check a browser and that is when a lot of things clicked into place. Basically it was the realization that computers are just sending text back and forth at blindingly fast speeds. Obviously I had some sense of what was going on before, but that was the demo that did it, and I had to learn it on the job because no one at school ever did that, which I find sort of backwards - school should be where you play with that sort of thing, but I suppose we all learn differently.
The email telnet thing was also a good learning experience. Gmail's servers are fun because you can see the designer's cute messages; if you forget your EHLO, they'll throw an error (or at least they did years ago) that it's polite to say hello first.
When I was at Fog Creek, we still had that as an interview question (with googling allowed). It was a good judge of how most folks approached something they used but didn’t necessarily understand. If the applicant already knew how to do it, that was a different signal as well.
Think ChatGPT would be allowed nowadays? It’d be my source for something so esoteric. I haven’t bothered to learn mail protocols because any message you send will be marked as spam unless it’s from a major provider.
This is not true. I run a mail server for myself and some friends. It started as an old desktop running under the desk at a university, transitioned to the back corner of a server room when I was working as a network engineer, and now it runs on a Raspberry Pi in a closet of my house. I had to pay extra to get a static IP at home, but everything has been off the shelf and DIY. My current Raspberry Pi has been running for nearly ten years, with only one interruption (the SD card failed). The idea that you can’t run your own mail server is a myth, and I think more people should do it. It is not hard, and you will learn a ton.
(I used to run mail for a large corporation, so yeah, I know a lot. But I learned how to do it by running a mail server out of my dorm room.)
Happy to hear that; thanks for correcting me. I was going off of what other people said, which I should have mentioned. It’s nice to know that it’s still possible to run a mail server.
No idea, but if it was available back then, we’d have been looking for whether you are critical about what it tells you and how you verified it was correct. That was a main idea behind the interview.
You can probably still do that by having a local proxy and communicate with it via good old HTTP/1.1 and let TLS and HTTP 2 and 3 be handled by the proxy.
This reminds me of when I was a kid, and learning to write some C to "create games". At that time, things like functions were some vague and "magical" mechanisms that I had to adhere to because that is how things are. I didn't even think more about them, taking them as "holy"; except only the one weird thing that were varargs functions, which seemed to do something strange and confusing, putting a disturbing crack into that perfect picture, that I had trouble wrapping my head around. It was only when this kid started learning assembly, by virtue of a book with an amazingly baiting title "How to write computer viruses", that things fell into place, and suddenly understanding the stack made everything clear and straightforward (if no less amazing, through appreciating the brilliant genius of the procedure call protocols!).
And indeed, the first time opening a POP3 session to an email provider through telnet was also an amazing feeling; maybe even more visceral thanks to its "live coding"/immediate feedback aspect; but personally just a small bit less foundational. So, anyway, through this whole story, what I really wanted to say is - thank you for your service and approaching your teacher's post in a great way!
Yeah I used to occasionally debug and test some things by writing manual HTTP requests over telnet, and it is something I’ve noticed new devs often just don’t get how approachable HTTP/1.1 is. Even if they have worked with headers and methods, there still seems to be a lot of magic happening in their minds.
Unfortunately chrome has stopped showing cookie headers in the network tab. You have to look somewhere else for those, and I think at that point you’re just seeing the current cookies, not the raw headers. Maybe there’s a security reason for that, but it is a further abstraction layer of “we’ll present the information a certain way” rather than “we’ll just show you all the information that the server sent back” that I expect will grow into other headers (maybe already has) and further remove us from seeing the raw details and having a clear understanding of what’s happening under the hood.
(You know it’s possible HTTP/3 cookies are even passed in a different way under the hood from other response headers, and that could be part of why they’ve separated them out. I hadn’t considered that possibility.)
Very nice intro. I like how you introduce networking layers. It seems you are starting to go down the path of explaining how computers and servers work - and that is certainly daunting. Maybe just stick with networking and introduce more of that and how real world systems (T1 and Ethernet) are just changing voltages at some point - and can be swapped in or out because of network layers. Then build up packet switched messages TCP, Telnet, HTTP
With all due respect, this is some elitist nonsense.
Sure, a frontend dev who strictly works on that won’t know much about, say, the OS layer. But they don’t have to, it’s not part of their job in any way.
There is no shortage of young people that work on areas that makes them require deep knowledge of many different layers of the stack, it is simply not necessary for every IT-related job. But one can absolutely pick it up if they want to.
I once worked with a back-end dev who didn't know what RAM was. He knew C# and ASP.NET but couldn't point to RAM on a motherboard or explain what it really did. About all he could describe it as was "temporary storage". Their lack of basic knowledge of how computers work really showed in the crap code they wrote. It was no wonder it was their code that ate up all the RAM on the server because they never put much thought into the hardware it was running on. I know this is anecdotal, but there are plenty of devs just like this out there.
I talked to a guy who "managed software development", and when I asked him about the resources om their developer machines (some kind of VM), he got angry at me when I laughed nervously when he didn't know what RAM was.
> but couldn't point to RAM on a motherboard or explain what it really did.
To be fair, the last time I "opened up" a computer and changed its RAM was about 10-12 years ago, and since that time I've only worked on a Mac Mini and on a laptop meant that "opening up" computers became a thing of the past.
This was years ago, in a Windows shop, everyone had a desktop PC. Not important that the guy didn't know what was inside his desktop, but it's the lack of understanding of the server hardware his code was to run on that was the problem. In all servers and desktop PCs ~10 years ago, RAM comes as DIMMs and would be easy to locate for anyone with a passing interest in computers. I'm not talking about Apple know-nothing fanbois here.
> Sure, a frontend dev who strictly works on that won’t know much about, say, the OS layer. But they don’t have to, it’s not part of their job in any way.
The HTTP protocol, HTML and vanilla JS are not "the OS layer", and yes, it's part of your job as a web developer to at least understand the basics of them. There are many so-called "frontend devs" nowadays who literally only know React, and if you asked them to create a basic webpage where clicking a button changes some text in an element without React, they'd be completely lost.
Usually the problem with this kind of developer is not just that they don't know, it's that they don't care. They WILL inevitably run into problems that require this basic knowledge to solve because the frameworks can only hold your hand so far, and instead of trying to figure out what's going on under the hood that's causing their issue, they will instead shrug and go "it's not part of my job" and just write a bunch of garbage spaghetti code in an attempt to work around the issue.
Shit developers used to cobble something together in PHP and had a bunch of horrid code on the web to copy and paste.
Looking up some deeper part of the stack is not that hard for a legit dev.
I think the critique is that you're applying this too narrowly to front end developers.
Pretty much anyone who has been around long enough has met someone with years and years of experience doing exactly one thing, whether that's WordPress or java CRUD or something else where they have blindingly obvious gaps in their knowledge that someone with their experience shouldn't.
The lack of deep knowledge isn't limited to a single field.
There are people who suck at their jobs in every industry. My wider point is that the skills required for a good web developer is vastly different than, say, a compiler dev. The latter is not a “webdev who is a better developer”, they need much more business-knowledge, understanding what the client meant, predicting future requirements, etc.
This is an entirely new business domain, it’s not really trying to be the same.
It is not elitist to point out the flaws in modern web devs who don't know what an HTTP protocol is and the structure of an HTTP request (headers vs body etc). I interview a lot of them who cannot explain the difference between a form submitted directly vs through Ajax but they surely know how to send a POST request through node/express.
I don’t even know the difference between a form submitted through JS vs a browser. Who cares? Unless there’s a problem, there’s no reason to know such things. And I’ve written http webservers by hand.
People here have way too much confidence in their interview questions being a good signal for experience. It’s pretty wild.
My point is that your question is some esoteric gotcha party question that may as well be out of Trivial Pursuit. But you’re treating it like anyone who doesn’t care is just being a bad engineer. There are countless ways engineers spend their time, and choosing what to work on is the most important choice of their careers. It’s on you to justify the claim that knowing the difference between a JS form submit vs browser submit matters at all, let alone that it’s a distinction that comes up in day to day life.
The issue isn’t that every single person in the IT field needs to know HTTP in detail. The issue is that people who have invested in training for a role and are applying for a role do not understand fundamental technologies that are essential to that role.
For you that would be trying to work in ML not understanding any of the theory or how it works, but having used OpenCV with some premade models a few times.
For a frontend web developer who, as a large part of their role, will need to communicate with backend systems… that’s not understanding how their FE web application actually communicates with BE systems.
And from experience… yes, this is very common. And it has a noticeable impact on their effectiveness. Trying to debug why some interaction between your application and the BE application isn’t working while thinking the dev tools network inspector is just black magic and nonsense makes your job substantially more difficult.
This does make these people bad engineers. They are not able to understand and solve a huge class of the problems that they face day-to-day and instead (in my experience) often fall back on “just try a bunch of different things until something works for reasons I don’t understand” which is a poor way to approach work and leads to overly complex, buggy, brittle systems.
I strongly believe tech interviews should be about seeing _how_ a candidate works and less about _what they know_. Everyone has holes in their knowledge, but can easily be filled by training or on the job experience if they have excellent problem solving skills. Maybe they never needed to know the protocol level but can deliver excellent ux results regardless.
I've started nearly every position in my career in a different domain without prior knowledge, including workflows, protocols and the languages that implement them. I may be an extreme case, but we exist and your process inherently excludes us. I've worked in robotics, consumer electronics, healthcare, fintech (not even in that order, although that probably would have made more sense.) I've delivered in each domain as well as my expert colleagues. But if you asked me to implement a LLM, I'd shug and tell you I don't know how, but I _would_ explain my process for HOW I would learn to.
"how_ a candidate works and less about what they know"
I do get your point but depends on the role you are hiring for. I have had candidates get upset at me (for example: bootcampers) because they couldn't explain how to submit a basic HTML form but wanted me to look at their github "portfolio" that was done during the bootcamp with React/Express Code and what not. Their title was "Front end React Developer". That is a problem and I usually wouldn't hire those people.
If you're specifically _not_ looking for react front-end developers, then the on-the-job training for those candidates on the position you're filling _can_ be cost (time) prohibitive in certain orgs to "get up to speed." Likely you should filter those candidates out earlier if that's your case. Sounds like you are, but I'd like to also point out that this strategy has it's own costs.
For example, I would take a react developer with obvious gaps and a strong willingness to learn on my back-end services team over a passable back-end dev who was set in their ways (unwilling/uneager to learn new tech, entrenched opinions stated as fact, etc.)
Naturally, my approach is not a one size fits all situation and a great deal depends on org structure and mentorship opportunities being in place for it to work. The benefit being you avoid monoculture "silos", and have more cross collaboration and transfer opportunities between teams (some may call this "full stack", I wouldn't)
Several high performers on my team are "self-taught", work comfortably across several languages today and can pickup new tech easily. They came in knowing their "one stack" at the time. If they were "bootcampers" or not seems irrelevant and somewhat reductive/offensive.
That said, if the candidate in question shows _no_ understanding of their problem domain, can't reason their way out of a paper bag, and get visibility upset from questions when you try and tease that out, that is a certainly a red flag in my book.
With respect to your example given, however, it could be argued that http details because of the abstractions in place inherent in react, aren't critical domain knowledge for what is essentially a UX dev. If they can ship an experience your users enjoy more than the candidate that can recite the HTTP 1.1 RFC and cannot, then what have you gained in hiring the latter other than maintaining a culture of pedantry?
TL;DR it comes down to how much risk your team is willing to take on, as there's always risk inherent in hiring ANY candidate, including those that don't already check all your boxes in regards to tech know-how. I'm merely stating that by overlooking candidates that don't fit your self-imposed mold, you're likely missing opportunities for rewards that can pay dividends when it does work.
Besides, it's not like getting a CS degree doesn't mean getting acquainted with all layers of the stack. They still teach assembly, C, CPU architecture, low level networking, etc. It's just that most people rather specialize in something far less tedious. And there's definitely more areas to pick from today than had ever been before.
Don't forget us over in hpc and embedded! What are these web things, and why do have stacks of them? But more seriously, I'm a very experienced systems programmer but I like stuff like eframe and trunk because it lets me serve up a gui I write in rust like it was a native app as a wasm page without having to know anything at all about web stuff or how many stacks to put the web thingies in. Perhaps someday the wasi will bypass all the js and html deep magic I don't understand, but not yet.
I rarely do any of that, usually my abstractions are container layers, a libc, and then a kernel between me and registers and dma. But that is a lot more layers than you'd think. I can't even understand the boot process on a modern machine with tpm. And i have no clue how many processors or controllers are in my computer. Every usb controller has an entire arm core in it, mice and keyboards have 8051 running c-ish code someone somewhere wrote, there is no bottom to the complexity.
I am technically q full stack dev and I don't grok how aspnet core actually generates the web requests. It's layers upon layers of magic to me.
Similar with mediatr, I know conceptually that it "just" checks for the right classes in the loaded assemblies. But still feels like some weird incantation to use, to me. And I have such huge knowledge holes across all layers.
Sometimes i feel my lack of knowledge to such a degree, that I question whether I should even be a programmer.
I don’t understand how front end devs can get so far without learning how to use the developer tool console now standard in every major browser. How can one not learn about a POST vs GET when things like request caching and back button complications are a reality? I hit these issues as a senior backend dev fumbling around with front end so many years ago so I have a hard time understanding how that isn’t even close to the experience kids these days get.
Too much trial and error and throwing everything they know at it until it gets close enough to the desired look. I have one co-worker that has taken off after I showed him how to use dev tools and another that's still trying to guess their way through the problem after showing them how to use dev tools.
I’ll never forget the time a network engineer escalated a ticket of his to me on the infrastructure side (think L4+ support) that a developer threw his way saying that his web pages were loading really slowly and that this was a high priority, high urgency project. I asked how they came to the conclusion that it was something related to the cloud infrastructure rather than something else? They said the developer didn’t really know besides to ask “network people.” So I get on a webex with the parties where even a VP is on there because his product launch is on the hook and then I ask the developer to open developer tools to which he responded “what’s that?” I think I had to mute myself for a bit while I screamed for a while that this was what qualified one to be a senior front end engineer at the company. So I stepped through and got the network transfer chart and pinpointed that all traffic was coming through quickly except there was a big initial delay while doing DNS look-ups that was super slow to get the first byte. The network engineer on the call was stunned that such a thing existed and thanked me for saving him a ton of grief because he got a LOT of ticket like this one.
This wasn’t in like 2010 when Firebug was just coming out and people still used Firefox as a rule, this was like 2015 when dev tools was in every shipping browser approved for corporate use by a technology laggard company.
So the moral of the story is just blame DNS first.
> Nowadays I happily patch nginx source code and compile it manually, but it took a lot of work to learn a minuscule chunk of what all those guys knew by heart.
I'm about half a decade into my career and I've recently tried to take note when I hit milestones or achievements, even little ones.
I was helping an intern with a tool and it didn't have support for what we needed it for. Since the tool was open source, I just cloned it and patched it to add support for what we needed. When I told the intern this, he couldn't comprehend that I was so blase about modifying this tool that seems like witchcraft to him.
But what I quoted, I feel deeply. I've worked with so many people with so much knowledge they can't possibly communicate it, and I'm only starting to really understand the tech around me. I didn't think anything of patching the tool at the time, but its important I look back so that the younger version of me can see the progress I've made.
I have worked with interns and young devs just out of school who are sharp as tacks. I had an intern a few years ago, the first task we paired on was to figure out why our MySql connections from Qt were misbehaving. We stepped through the application code, the Qt code, found where it was failing, and ended up backporting a Qt patch from upstream to fix it. Yes, I gave him some guidance, but once he caught on he was able to navigate this complex mess of C++ code across a few technology stacks and effectively reason about it and make changes.
A lot of CS skills are generalizable. Knowledge is one component you pick up with time. A good education, self-taught or otherwise, should allow you to drop into any sort of code and be effective without much spin-up. College covers CPU architecture, assembly, networking, operating systems, web, algorithms. This is not esoteric stuff, it is very standard and you can get it all in class or from textbooks free online!
That’s an underestimate, and is it really that unexpected given the huge interest in CS? Previously there was an implicit filter that only the most skilled people could make it into this new world, that was just being built. Now that it is built, and everyone wants to enter - of course the average quality will decline.
I’ve worked with around a dozen interns/co-ops, and only 2 stood out to me. The rest often made me wonder if they could reasonably handle this kind of career. I hope they could!
It’s tough if you feel a degree of responsibility for their success. Mentors are one of your greatest assets early on (and arguably later as well), and to try hard to have them succeed and thrive only to see them languish on trivial tasks is awful.
I think part of the problem is that CS education where I live is awful. The kids come out of school expecting real work to be wildly different than it is, and it hits them like a brick wall.
Maybe a difference between then and now is that "back in my day" CS still wasn't considered to be a "hot" college major. And there was believed to be more flexibility in choice of a major relative to whether you could get a decent job.
Today, there are probably a vast number of entrants who heard that CS is the ticket to a high paying job, and they are also told that an internship is a vital bullet point on their resume, if not a guarantee of a job at the internship site.
Then, as now, students studied under the constant drone of "you will never use this stuff once you finish college." They still have to decide if they're actually interested in the subject matter or not.
A good bellwether of career interest is the students in the youth symphony. They've all aced every subject in high school, plus rocket club, gentleman sports, and orchestra. The program for the end-of-season concert will have a little bio for each graduating senior, including their college interests. Half of these kids want to major in CS.
I also see plenty of smart young guns. And if i think back I was better myself when I was younger. I’m less excited by digging into problems with each passing year.
There's a huge gap, though, between the boot camp methodology of essentially "here's a visual design or a set of fields, I'm going to pattern match that onto a single React component or backend endpoint that's isolated to a single file" vs. "I'm comfortable looking up functions 8 files deep into a codebase and maintaining a mental map of what data flows where."
Once you have the latter, either by having built such a codebase, having worked in one, or even having experience with puzzles or games requiring multi-step planning and understanding of the potential failure points at each step... it's absolutely transferable. But there are also a lot of people in our industry who have memorized interview questions and see their role as churning out components. And while arguably that's not a CS education, it's being called the same, and it does a disservice to those people's careers.
You can see this with a "new" ANY dev, because there are only 4 years of college and far more years are required to learn the increasingly complex technology environment.
I graduated with a Computer Engineering degree, did assembly, C, microprocessor design, computer vision, and know a good bit about lower level stuff, how memory works, how networking works, etc. All the stuff people in this thread seem to be lamenting the lack of. But I was also a shitty employee fresh out of school because I didn't know anything at all about modern software development because there was absolutely no time to learn that stuff as well.
I still had to spend a lot of time getting good before I was worth anything, just as these "new frontend devs" will, as well.
> "* I didn't know anything at all about modern software development because there was absolutely no time to learn that stuff as well.*"
Yeah, but modern software development is trivial to learn, particularly in comparison to a computer engineering degree. You see "developers" here on HN gloating all the time about how they didn't need any post-secondary education at all to get their jobs; these frameworks are literally designed to be usable even by minimally skilled coders. You were in a much better position having to learn modern software development after a computer engineering education rather than having to learn the rudiments of computer engineering on the job after getting an education in modern web development.
I can last years with only Python, so those little detours won't expend as far as React.
It's more likely that I will do something like "a full web app from installing the OS on a fresh laptop to coding it to hosting it on server".
It would also be more beneficial: plenty of tutorials for individual techs, rarely they teach you how to integrate them together and go beyond the toy example on localhost.
But for that I need a better platform than substack. Therefore that's not going to happen anytime soon.
> I started to write an HTMX tutorial (https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1) because I noticed a lot of young coders don't understand what to do with it. They read the tweets saying it's nice, but when they look at it, it makes no sense to them.
I agree with you but HTMX? That's a big abstraction layer. A good one but still, not really a way to avoid layers of abstraction. Pure JS makes more sense.
This is not consistent with my experience with younger devs. There are teenagers doing kernel-level hacking and there's also older people who have been comfortably driving RAD tools for decades. This is fine. Software development has been gradually specializing and it will continue to do so. The tools and resources available for people who want to pursue low-level engineering are better, cheaper, and more accessible than ever.
My $management have said that there are nearly no teenagers who will apply for kernel work. You got someone I can reach out to when the option comes up again ?
If you’re managers and tech leads then why aren’t you making sure the people you’re supposed to lead obtain the understanding?
We work with both high and low level tools where I work, and in my experience there isn’t necessarily a lack of understanding among younger developers. The CS courses still teach you a lot of the basics, and to actually use what you call “hype” tools efficiently you typically need a rather good understanding of how they work.
I actually don’t think we have enough abstraction, especially not in the DevOps field. In my opinion we’ve overcomplicated the deployment procedure without making sure it was abstracted. We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge. I’m actually fine with infrastructure as code, but it needs to be templated to the point that our developers simply “order”resources based on whatever template they want to deploy so that they don’t have to keep up with the constant changes or learn how networking in enterprise organisations work.
As far as the CPU stuff goes. Young developers do a lot of low level code. We write a lot of embedded software for our solar plants, and a lot of the time, they seem far more capable and “modern” in their approaches than the “old guard” exactly because they’ve been taught the same curriculum but also all the lesson learned by the “old guard” along the way. That being said, it’s not really necessary for a lot of developers to keep an active knowledge of how an x86 CPU works, because it’s very unlike that they’ll have to work with it. It will often be far more useful to know how various types of ARM processors work, as that’s something a lot of us actively work with.
> We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge.
I am afraid that many businesses prefer the vision where the software developers do the operations too, and the company can fire the operations department. Consider how much money you could save... before it all falls apart.
Short term optimization is how managers get their bonuses. The trick is to move on to the next project before the old one falls apart. Then it becomes someone else's problem.
Agree that devs not needing to know this is how things should be, and that CTO/tech management however is not acting this way.
It is basically idiotic to me, as a highly paid senior application dev.. that they basically want me to spend any cycles on stuff that would ordinarily be done by someone making 1/5th my TC.
If I can do it more than 5x as fast/efficiently, sure, but I don't. IaC is something I worry about at the start of a project, as hamfistedly as possible, probably over-provisioning so I don't have to go back to it anytime soon. All so I can move onto what I am paid to do - delivering functionality to stakeholders. Someone who deals with IaC as part of their full-time job will be the expert who can move more quickly & correctly through it.
Sometimes this is just galaxy brain budget arb, where the infra org gets to show a cost save, to the detriment of the appdev org.
> stuff that would ordinarily be done by someone making 1/5th my TC
Did you mean for this to be condescending towards ops? Because that’s how it sounds.
Ops is not easy, at all. Even if you’re doing zero IaC, I defy the average dev to try to provision a Linux box from scratch and install, configure, tune, and maintain the stack you need to run your code. Throw IaC atop that, and now you need to understand declarative programming and OOP concepts to be able to do it efficiently. Then, there’s K8s…
Not to mention the architecture side of things. Devs love to grab whatever shiny thing they saw on a Medium blog, even when it’s a poor fit to their problem (or their problem is unoptimized code). You probably do not need a columnar DB, you just need to normalize your tables and learn relational algebra.
I do not mean to be condescending. I am being literal.
Every job is hard, and the pay is not exactly commensurate with effort.
Ops is absolutely hard and theres a lot to learn. And in doing so you generally may end up with DevOps teams with minimal to zero domain knowledge in what the company delivers to end users.
So if you have a team of people who built a deep level of expertise in something over 10, 15, 20 years and that something is business facing, domain knowledge, etc.. then it is not of value to have their time spent on other tasks.
Often these DevOps roles are filled by more junior staff at the earlier stages of their career. This is another reason the staffing is usually lower cost.
Staff SRE here: I comfortably exceed your TC by a factor likely approaching 2.0 doing harder shit than your whole team combined. That’s nothing against you, only that you don’t really know what you’re talking about here. I can safely conclude that because 20% of your app dev TC is not in any ops in the US, particularly not DevOps-school, not even “legacy-style” SA; hospital SA, for example, used to be a high six-figure, pensioned gig, until we commoditized ops as a concept in our reinventions and pushed everyone to AWS.
Hint: I wrote operating systems for fun before becoming an SRE. Imagine how much fun I have explaining context switches and CPU cache invalidation and their implications on their app’s performance, to application developers who consume frameworks and look down on my salary! Building a distributed computer at scale is much, much, much harder than your Linode tutorial expectations of what ops does.
Native software is mostly dead, so operations is software engineering now, at least those parts of software engineering you folks threw out when Docker came along and turn your nose up at, and I make more money every year cleaning up the low-hanging fruit y’all leave around, so…
It may be true your TC is so high, and I salute is so. Clearly my tone implied a level of looking down my nose I didn't intend.
Now take all the stuff you are an expert at, and imagine the CTO assigned to you - build dashboards of accounting information. You don't do UI, and you don't know accounting (maybe YOU do, but your average SRE/DevOps do not).
It wouldn't make sense right? And a lot of those tasks can be done by a junior "data analyst / BI dev". So it would be an expensive mistake as well.
Agreed. DevOps was a mistake, Full Stack was a mistake. Go back to teams of specialists. Ops doesn’t necessarily need to gate deployments - CD is a good thing when properly done - but believing that the latest AWS offering means your dev team can handle everything from the database to frontend is a fallacy.
>The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path.
The worst problem, which TFA alludes to, is that those "layers" are not really needed, they are leaky kludges upon kludges to make some lower layer unsuitable for later needs more palatable or able to handle some unforeseen use case that's against its design. Other stuff is just added as ways to sell new enterprise tooling, support and consulting.
Seeing the layers and technology stacks being added over time can give the impression of watching some organic evolution happen, when it's often merely accumulation, rough patching, and corporate attempt to push its technologies/NIH.
Yes. And we know this process from legacy code. Rather than fix the problem, it is usually easier to just write some new code to manage the old problem. We tack on new layers, modules, abstractions, workarounds all because at the time it seemed to be easier than refactoring the cause of the problem.
But it only works so far and at some point the application becomes unmaintainable mess and the effort to rewrite it from scratch starts.
Only we can't rewrite the world after it has become so unmaintainable that nobody can figure out how to change a detail of some layer inbetween the program and the transistor.
Heck, we are already stuck trying to fix the bad design of IPv4.
and how many of today’s CS-degree holders would barely understand any of it. As someone who has also “grown up with all the technology”, I’ve learned and experienced all that. But as a percentage of “software engineers”, there’s fewer and fewer that do every day.
Unfortunately, knowing all this won't help you get a job when the market demands an army of React devs and little else.
Under our current system, knowledge that cannot be monetized is useless. Worse than useless, in fact, because time spent learning low-level concepts is time that could be spent learning marketable skills.
The industry is effectively paying us all to forget.
Sure, learning the USB stack and differential signaling is fun, but unless my job involves implementing custom USB devices from scratch then it's pointless trivia that won't pay the bills.
Hey I've struggled with this notion as well.
But then I sometimes compare this with crypto - or how a bunch of kids simulated the entire financial system and made the same mistakes made 100s of years ago, with the end effect of an new generation that taught itself finance.
I suppose that is what will keep happening. At some point, someone will decide to re-write everything from scratch and re-discover lower layers of abstraction or challenge them.
A sci-fi semi-solution is to hope that all the relevant information is embodied in these LLMs being built.
A funny possible history is everyone forgetting what’s necessary and then having the relevant information they need accidentally blocked by an AI system they have no idea how to build or fix (maybe because of terms like master/slave going against elite values).
I used to think that the historical development of a subject can not possibly be the best way to learn a new topic because it is so path dependent and random. But with experience I have seen that there are very few instances where educators mixed up the historical timeline of a subject and made it more easily absorbable. It still baffles me that this is even the case for abstract fields like math
> people who really understand the technology that the world runs on, are a slowly dying breed
Not completely. Curious people will always pop up, it's just that they will take as much time as you did to learn everything.
I like to think of myself as one of the curious people. Ever since I was a kid, I had a huge desire to understand how the hell does this magic box called a computer work. Decades later, as a software engineer, no matter how much I learn, no matter how deep I go, there's always this voice in my head going "but WHY? WHY are things the way they are?".
I did start top-down instead of bottom-up, starting from the high-level languages running on modern operating systems, going more and more into the (professionally unnecessary) low-level through pure curiosity, but I still believe I will get to the bottom one day.
> For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option
As a counterpoint, I'd say that it's also chance to build on top of these abstractions without having your brain cluttered with how they're implemented. That's also how science grows.
And don't underestimate the new players. They're perfectly capable to understand the low level details if they have to, and they have much more resource available to learn too.
I am exactly the same, coming from making my own add on boards for early PCs and programming in machine codes and then following tech as it developed. Yes it is hard to find young persons with the same level of understanding but they do exists.
Where do we draw the line though? Surely those proficient in low level programming languages don't then need to become proficient in building motherboards from scratch?
I think that, at the very least, there's a huge benefit in having a bit of understanding one abstraction down from everything you interact with. Knowing what an abstraction is abstracting, and why, goes a long way in coexisting with that abstraction. A low-level programmer doesn't need to actually build motherboards from scratch, but they'd probably benefit a lot from a bit of understanding about processor caches, registers, and bus latencies.
"But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals."
I just hope when that time comes I'm well compensated for studying erudite texts on low level computer science while chilling on my phone instead of browsing social media or playing games. But honestly, I enjoy reading and learning about ABIs, C programming, network protocols, etc... more than mindless scrolling anyway.
Although I worked with computers most of my career, it wasn't in the tech sphere at all. The article meant a lot to me because of trying to wade through tons of websites for info about signing up for SS and Medicare. These sites were not set up for non-computer tech people.
Like you, I enjoy learning techie stuff, but most of my friends don't. Because, IMO, most websites are written for younger people, they feel almost helpless.
This is a non-problem. If there's a shortage of people working at X level of abstraction (where X is possibly very low) then salaries for those with that knowledge will go up and more people will seek it out.
Free markets don't solve every problem but they solve this one.
>We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals.
I'm gonna guess this same message, in different words, has been repeated often throughout history for as long as people had the ability to reflect on their life and the next generation.
So really, what's different this time? What's the difference between forgotten technology that nobody needs and an unpublished author who's works were forgotten? I think the answer is none. We just think it's more important because this is the industry we care about.
We're actually really good at reinventing the wheel in this industry, so even if some software is forgotten, you can probably bet it'll be reinvented or reverse-engineered if it's really needed.
While we should preserve technology knowledge (because it's pretty cheap to do so), in terms of the economy, sometimes things are a stepping stone and should be forgotten.
>We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners.
There will still be people at e.g faang/semiconductor companies
that do this stuff day2day and even push advances in those areas and have already created solid training materials for younger employees
If you're worried that this knowledge will disappear, then feel free to write it up in form of tutorials.
Excellent point. Giving my perspective as a relative newbie (~5 years) to the industry with no comp. sci. background.
It was around the year 3-4 mark that I decided to knuckle down and try to improve my fundamentals (data structures, algorithms, memory models, concurrency, CPU architecture and some network fundamentals) by reading popular papers and literature and writing all of my personal projects in C and C++. I’m about two years into this study and while it’s been immensely rewarding, I’ve found it to be a huge undertaking while juggling life and a full-time job.
What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work (still Python and JS). I would love to be able to put these concepts into practice for many hours of my working day but it’s difficult to move from general web-stack development to more performance-oriented development (embedded, low-latency, OS, etc.).
My guess is that this is one of the reasons we have ended up in this situation. You can get away without knowing the fundamentals (a good sign of progress?) and that if you really do want to pursue these areas that promote building this kind of knowledge as part of your career, the barrier to entry is quite high and the positions are fewer than say a decade or two ago. I find it a shame because in my eyes, these areas are the most interesting and exciting areas of programming. It’s an art.
> What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work
I think this is because of the effect OP is describing: no matter how much time you spent studying, the odds that you learned about the specific thing that will make the difference in your work this week are slim because the technology is so broad and deep. Odds are that what you choose to intentionally study is irrelevant to your work.
You've picked a slice to learn about, but those who were working on it as it got layered know it all. Once you have that kind of knowledge, the odds of being able to explain some unexpected behavior approach 100%, but acquiring that knowledge took decades.
I've found a method that does work pretty well is to dig really deep into the topics that come up in your actual work. Instead of a random sample of fundamentals, find the parts of your job that feel like magic and explain that magic. Skipping over V8 and straight to the CPU might not be useful for a JavaScript dev very often, but a deep understanding of V8 is relevant more often than you might think.
I agree about the high barrier to entry, but I disagree about the modern job market.
There’re huge areas of the software industry where performance matters to this day. Examples include videogames, content creation, video processing, engineering software, science related HPC, and now AI.
However, transitioning from web development to these areas gonna be hard. That software doesn’t run in web browsers, the code is either desktop apps, deep inside web backends, or supercomputer programs. And high-performance C++ is often not enough for them, depending on the area ideally you might also need GPU graphics and/or GPGPU compute.
I was lucky I have never worked on web apps. I know TCP/IP reasonably well, I have some general understanding of HTTP and HTML, but I’m totally clueless about the modern frontend technologies.
What I’ve found most helpful for understanding how something affects Python specifically (but this holds true for nearly any language) is godbolt [0].
Find a chunk of code you want to optimize, simplify it as much as possible, then put it into godbolt. Match the interpreter version to yours. Then go look up the bytecodes and see what they do. Try changing the Python version to see how it differs, or a different approach to the problem (e.g. a map instead of a list comprehension).
This takes an enormous amount of time, of course, but you can learn quite a bit.
This sounds plausible but I don't think it's correct. The body of knowledge you refer to is large but not that large. I think by the time a teenager with a persistent interest in computers is 25 or so they can accumulate a similar level of expertise, and without no lifing it.
I believe that the first lesson plan for any subject should be to follow its historical progression. That's not the be-all and end-all but, whether you're talking about computer science or physics or economics, it's a good starting point because it's one proven way to deal with the field's complexity. You just have to be clear that the simpler models from older times are pedagogical tools, not current understanding.
I think this is unduly pessimistic. Take low-level kernel work, for instance. Between Linux, Microsoft, and Apple, there are lots of people who understand it. It's just a smaller proportion than used to understand it. The absolute number of people has probably grown.
But we don't need every programmer to understand low-level kernel stuff, precisely because the kernel abstraction is good enough at hiding that stuff. Most of us don't need to care (whereas 40 years ago, many more did need to care).
Personally, I myself never want to care about low-level kernel things. When I'm trying to write a program to do something, if I can write it without having to worry about low-level things, that's a win.
I have that knowledge but don’t know how to monetize it. Kids are coming out of school with 5 YOE and making double my salary with almost 30 YOE. And they ONLY know how to do React.
For example, I am NOT a cloud expert, and barely a novice, by any measure. When dealing with various CloudOps/Cloud Architect/etc types, without fail, the good ones are also complete Linux gurus who would be at home architecting on-prem infrastructure as well.
Unfortunately there are a lot of "experts" who are only comfortable working in abstraction and so as soon as we have an issue that scratches below the surface, it turns into an all-hands-on-deck mayday situation, looping in other teams/experts to try and save them.
I've also seen horrific implementations by abstraction enjoyers.
Imagine a continuous integration environment of the following - k8s where one pod simply runs a forever shell script that is basically "git checkout master;git pull; sleep 300;". Then the other pod actually runs the application on top of the shared storage that the first pod writes to. This script, unsurprisingly, fails silently and hangs frequently. To debug this you need to go through some cloud auth portal and click through some web UI to then open a console in your browser (which doesn't support copy/paste).
The idiot version of doing this on-prem would be a single small VM that has a cron running every 5 minutes, which invokes your deployment script, with cron configured to email on failure. This would be simpler, more reliable (not relying on a forever script), and more supportable (it emails on fail!). And it uses like 30 year old tech that just fkn works.
I totally relate to your point of view and experience. I'm so happy to ignore all the software related to the attention economy. What I find left over is the computing environment that I dreamt of when I was 12. I marvel that it took a whole generation of man hours to get to this point.
I think we are real inflection point between an analog world and a digital world. I also think that the Bronze Age never went away and analog techniques will just develop into craftsmanship.
I haven't posted a link to these Autonetics parts in while so I will do so again.
What I love about Autonetics is that they were solving the inertial guidance problem in whatever computing form they could get a hold of.
I'm working on a proper website but the images linked here represent the inertial guidance problem being solved with discrete components arranged in circuits and later those circuits integrated in silicon.
Older workers are supposed to pass along this knowledge to younger workers in an expedited fashion. Acquisition of the understanding of increasingly complex systems must be optimized to operational needs. One of the many, many gripes Millennials have with Baby Boomers and Gen-Xers is that they jealously guard institutional knowledge in the name of job security, for years, and then just... retire. If you're lucky, your company will shell out massive consultation fees to have them come back and drip-feed what they know to younger personnel.
It's not just the natural order of things that such knowledge passes out of this world. It's one of many collective and affirmative decisions by people born before 1980 to take as much with them as possible when they go. I would be skeptical if it were just one thing, but after affordable college, affordable housing, affordable healthcare, affordable retirement, and a viable biosphere, it's become a pattern. That you seem to understand this on some level and don't care to try to mke a difference is even worse.
An important thing to remember is that not only did you get to watch the layers be added, you also have decades of experience. In today's field, where we double in number every N years, that's enough time to accumulate enough knowledge to feel like a demigod—to have more than 20 years of experience is extremely rare in the industry right now. That won't always be the case: eventually we'll stop growing exponentially and we'll find an equilibrium, and eventually the people with 30 years of experience will stop being a vanishingly small minority. While that may not give us the kind of breadth that your generation has, I would expect many of them to be able to master the specific slice of the tech tree that they actually build on top of.
30 years is plenty of time to learn the everything from the browser down to the CPU, or the HTTP framework down, or the database down, and probably enough to learn two such slices. Part of why it seems impossible now is just because most developers not only don't have the historical context, they haven't yet had any time.
its more than just time. the sad part about this evoluation is that younger people are enculturated to believe that the foundations that they are standing on represent some sort of inviolate physics and not just some technical decision someone made 20 years ago.
i think whats required to move forward isn't just some dusty studying of these physics, but a reawakening to an understanding that these are just systems with tradeoffs, and we can choose to make other systems just as easily.
thats the only way the sandpile collapses and we get .. for example .. secure operating systems and network protocols. or systems that were designed to exploit the massive amount of concurrency available in modern machines.
> younger people are enculturated to believe that the foundations that they are standing on represent some sort of inviolate physics and not just some technical decision someone made 20 years ago.
This is simply not my experience of young people or of being young. There's a reason why radical movements are led and populated largely by the young—to the extent they err, it's on the side of "let's tear it all down and start over".
For the last 10 years you have been adding stuff that did nothing but increase the complexity. Now you wonder why new people can't keep up with the complexity.
I am not conflating anything. For me it was my day to day work for decades. Some new technology happened and I had to learn it to be on top of my game.
Trying to learn all this in 1/10th of the time to start your career is what requires perseverance.
I think you may be confusing what you see with the cause.
For example: You and I HAD to be willing to go through that, and we were fascinated by it. There are still plenty of great FE and BE young devs who see something and instantly ask "How does this work" and keep going down. The difference is, most people in employment don't HAVE to do this like all people in our early career did. If those shortcuts to getting paid existed in our early careers, many of us would have taken the money and not cared. It's not an experience or a "You had to be there" thing, it's simple curiosity and motivation that sets people apart in this regard.
> We, people who really understand the technology that the world runs
on, are a slowly dying breed. We are still here as tech leads,
managers, directors, business owners. But there will be a point in
time when we will go on retirement and there will be only precious
few people who had perseverance to really understand all those
things by diving into obscure, historical manuals
Some people like to say this is a perennial problem that every
generation bemoans. I get why that might seem intuitive, and for the
longest time I've hoped they're right and I'm merely getting old.
Sadly they are wrong. Long term historical patterns of empire collapse
follow this over-reach and under-education cycle that leads to a
catastrophic rebuild capability gap that is triggered at some point.
Thomas Thwaite's "The toaster project" [0] is a wonderful commentary
on this. I cited it in a talk I did about education in world that
will no longer pay for teaching kids how to use chips and breadboards
in skinflint universities that prefer to teach them Microsoft Excel or
use some crappy simulators of everything [1].
When it comes to cybersecurity we can only obtain defence in depth if
we have knowledge in depth, and there are ever fewer of us around who
understand how computers actually work. It's more than a little
unnerving to meet CISOs and folks with high-flying titles who have no
idea why using a phone app to control a Fortune 500 company infra
might not be a good idea.
I suspect this happens in all technical professions as things advance: you learn first principles and low level implementations in a formal training program/degree, but the demands of the industry to be productive require that you use and trust abstractions.
Even my doctors don't know some things about various drugs they prescribe me, because I can afford to spend the time reading the literature that only pertains to me. I'm sure their biochemistry background means they could definitely outperform me if they wanted to do a deep dive, but they know the important parts, and they know how my condition stacks up in the compendium of total knowledge and clinical experience they have. I don't think anything important is lost if they cannot at the drop of a hat tell me the half life and mechanism of action if I grill them on it.
This is the perfect articulation of something I think about all the time. It is so much easier if you were in it while it was happening.
I've been trying to write a blog post about this, and how to address the problem specifically for front end development, where it's absolutely ludicrous. It's a very serious problem, especially when you think about security.
You have all these people clamoring to get cloud certifications so they can work in IT but its so complex. A certification isn't enough. There just aren't enough people with enough knowledge and practical experience to fill all the needs that companies have.
I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc.
If by "understand" you mean "have a passing familiarity with" then.. sure, but if you mean "my understanding is reflective of reality" then there's absolutely no way.
We, people who really understand the technology that the world runs on, are a slowly dying breed.
There are more people doing "low level" programming right now than were doing it a few decades ago. The only knowledge that is fading is knowledge that's applicable to legacy and/or retro systems.
I think there is a fundamental issue with the article and that's the author stating devops or devsecops is someone taking care of both development and operations. That's just not true.
Increases in the levels of abstractions are necessary. The human brain doesn’t become much more capable over the years, but the number of available tools does so you need abstractions to allow for focus. I don’t see why this makes the future bleak per se. A programmer working on a business problem can be extremely productive in a high-level language even without understanding EUV, compilers, assembly, instruction sets, kernels, USB protocol, HTTP, dies, substrates, and much more. The same holds for pilots. They don’t need to know everything about aerodynamics, tensile strengths, aluminium, rubber, GPS, and much more.
However, a discussion over the misalignment of incentives in modern society can definitely be had. I would trust the pilot generally much more than a business software developer, researcher, or banker because the pilot will be the first to arrive in a crash.
This is an insightful article, although I don't necessarily agree with the view that "everyone" needs to know everything from first principles to be good at their job.
Talking about abstractions, during my past month, I was reading nand2tetris, and it's a compelling experience if you understand the exercise you are doing, which is not about building a computer from first principles; it's much more than that.
It makes you really understand what's going on behind the layers of abstracting that have been raised. Sometimes, understanding every layer of complexity is impossible, but depending on the area you are in, we need at least to try to understand the roots of it.
However, this is not for everyone; ask a musician if they are really in the weeds of why the instrument is producing music (the physics behind it!). They are probably aware that it's vibrating air, but, in general, they won't know the theory behind it.
Everything is built on abstractions, and that's OK, with time we will add even more on top of what we have; now, the issue is when those abstractions lock you in with a mindset that prevents you from creating something new without relying on those same abstractions that help you build stuff.
Many discoveries and inventions were made because they knew the layer of abstractions on top of it, and they just started again from scratch. Even Figma, the software, is built on a new core of concepts based on how the web was working back in the time [1]
> ...ask a musician if they are really in the weeds of why the instrument is producing music (the physics behind it!). They are probably aware that it's vibrating air, but, in general, they won't know the theory behind it.
A big difference is that, unlike computing, their instrument probably won't stop working because of some subtle change to physics introduced by a seemingly-unrelated change made to the universe by some other party in the musical instrument / air / molecules / atoms / quarks "stack". Theirs is a world with some assurance of stability.
Ours is a field built upon shifting sands. Knowing what the foundations are that the edifice you've constructed sits upon allows you to affect repairs when it crumbles unexpectedly.
Musical instruments can and do break for unknown reasons and it's exactly as reasonable to expect musicians to crack out their machine tools to fix them as it is to expect a javascript dev to fix a kernel bug.
Computers are a relatively new invention. There are still people around from the times when knowing the entire stack from top to bottom was not just valuable but necessary. They worked during periods when abstractions were far leakier than they are today and far less reliable but also far simpler.
Some of those people have persisted with the attitude that knowing the stack top to bottom is still necessary as the complexity of those stacks have grown in complexity geometrically. This does not signify wisdom, only age.
- Knowing how to change strings is comparable to knowing how to use the browser console
- Knowing music theory is comparable to understanding programming language theory
- Being competent on the fretboard is comparable to being competent in one programming language. Strumming/fingerpicking could be considered another language.
---
And then for the controversial one...
- Only knowing how to use music-making software is like only knowing how to use low-code applications for development.
Thanks! That one frequently trips me up. If I think about it for a moment I know "his affect" is different than "the effect something has", but, as is sadly and normally the case, I power thru writing w/o thinking as much as I should.
> However, this is not for everyone; ask a musician if they are really in the weeds of why the instrument is producing music (the physics behind it!). They are probably aware that it's vibrating air, but, in general, they won't know the theory behind it.
Not a great metaphor. First, most instruments were invented by people who had no knowledge of the theory, not even of sound involving vibrating air. Second, many musicians do understand some of how their instrument works. It's not hard to understand how a guitar works, how to tune it, and how to fix a broken string. Most can't fix a broken body or neck, but they do understand why it stopped being playable. Hell, I knew a flutist who simply started to build her own recorders. Zero knowledge of physics, but after a few years producing great sounding period instruments.
I think everyone needs to know everything they do professionally from first principles. And if you are to build abstractions, they also need to be based on first principles in order to be easy to reason about.
Sometimes I'm learning a framework, and I'm in the part of "...you write this easy syntax and it outputs pure HTML!", which is great. But then the next line you read "also, when you use [obscure symbol], it does [obscure thing] in order to respect [obscure concept]". You have no idea what to do other than start googling those 3 new things, none of which are explained properly or at all in the current "super simple framework" documentation. I think even naming should make sense.
> However, this is not for everyone; ask a musician if they are really in the weeds of why the instrument is producing music (the physics behind it!). They are probably aware that it's vibrating air, but, in general, they won't know the theory behind it.
Knowing the nitty-gritty of the acoustics theory and the related math may not help you much as a musician... (that being said, if you're building a home studio or doing any mixing/mastering, you'll find it helpful to learn about physics of standing waves etc).
However, the details of how frequencies come to form the well-tempered scale with all the tradeoffs and imperfections, and the low-level details of music theory, would be something many musician nerds would actually know.
While I agree with the sentiment, I also struggle to find the right approach of peeling back abstractions and understanding the lower levels. Learning how something works under the hood takes time and can be a big commitment. How to do decide when to look below the abstraction? I’m most often interested in how things work under the hood, but I can’t always dig deep, which does bother me.
> It was clear, just by looking at how bad everything was performing, that something was wrong.
Meh, zoomers didn’t invent incompetence and slow, ugly software. Not everyone is cut out to write aerospace grade software and, thankfully for those people, not everyone needs it.
I think a good first step towards improvement in this space is by separating the terms 'abstraction' and 'indirection'.
Programmers too often add indirections which don't provide abstraction:
* The programmer wants to POST an object to a web server.
* The programmer also wants to think about it at the level of POSTing an object to a web server.
* And yet the programmer creates an HttpClient.java and an AbstractClientManager.java an ClientManager.java.
* These extra classes are just pointless indirection - not only do they not prevent the programmer from needing to reason about POSTs, but they actively make it harder to do so.
In contrast, if you have a collection type (Set, History, Permissions) and want to treat it as a monoid, that's a huge leap in abstraction, with only one level of indirection.
Because almost all programmers tend to add not only abstractions, but also shortcuts. And as everybody knows, shortcut is a longer but more challenging route.
I think the reason people do that stuff is because they do provide abstraction, but they only do it in UML diagrams, not in the code.
I've found the shittest programmers to work with are the ones that think visually, because in their head they just deleted two boxes and four lines, but in reality all they did was add 40 lines of code that do nothing.
Yes. Anything that is destroyed during the process of compilation is not architecture. It is just a method of code organisation, and we've turned the subject into a holy war, pulling in mindshare that should be spent on more important problems (e.g. we have fibre lines and Ghz multi-core processors, but why are user interactions slower than they were in 1995?).
80% of code organisation problems require doing a couple of things right: directory design (directories, file names, file contents), function design (purpose, naming, parameters, context and return type). I've been coding for almost 25 years and I've encountered the balance "20%" extremely rarely. Even then, 100 lines of simple code is easier to read and modify than 20 lines wrapped in clever abstraction (e.g. I once had the dubious honour of refactoring a broken and unmaintainable state-machine driven code base back to a simple series of if/else/switch/case statements, and the improvement in the development and troubleshooting times seemed almost unfair for such a simple "trick").
Folks that have really strong opinions on issues of taste, but then they output apps that send a couple of thousand requests ( with several second latency ) are extremely difficult to reason with.
This an amazing way to concisely explain the difference between indirection and abstraction. Works for me at least.
Anything undone by the compiler (perhaps even inlined) is just indirection. Everything beyond is actual abstraction (which cannot be broken down further by the compiler as it’s far too limited in its understanding).
We do? In my neighborhood, we can’t get fiber lines because of some combination of NIMBYism and political back-scratching allowing a cable company to continue to serve us shit slow and unreliable internet. In the middle of a city!
I’m more interested in solving _those_ problems than how long a request round trip takes.
This, a manager and domain manager for everything. One of the reasons I love go is that is inherently encourages being direct. All of the verbosity that people complain about it is infinitely offset by simply doing the thing. Some examples:
- how directory structure is part of the code organization of the language, and how it's something you have no choice about (so have to spend zero brain power on), and works out of the box.
- the concept of exported/unexported fields (no need complex logic/annotations, ineffective _prefixes)
- named returns eliminating the need for explicit, so useful when exiting early
- default values (+ named returns)
Any of these *manager concepts just seem out of place in Go.
> Question everything. Especially things that don't make any sense to you. Don't just assume that someone else knows better - that's how you quickly turn into a blind follower.
That’s what I do by default since I was a kid, and I can tell you the social pressure not to is significant. I recall an interview I did once, and one reason I failed it was "questioning everything". It didn’t even felt like it, I was just asking questions about their system, it’s supposed to be basic curiosity. At my current job there’s this architect that explicitly asked me for continuous feedback, but now shows subtle signs that maybe I went a little too far.
And that's probably healthy for society to have a check. If the questions about the sky keep the questioner in bed they might not do the things they need to ensure their continued existence. In one respect that might be food but in a business that would be delivering change or supporting the business.
Another example: as a child you understand what + means and take it for granted. As part of Maths undegrad you start understanding the type of operators and the axioms of maths. The child was perfectly find completing maths problems without that depth.
Ah you went there eh. Yeah an alternative form on my reply would have been to point out that kids are the ones who question everything. Adults learn to identify the questions which will expand knowledge, and ask those.
My approach is to just question everything "else" where there is a chance I might be able to do something about it. This is why I'm not popular because I don't have much interesting trivia for party conversations. But I already have come to terms with that. This doesn't help with staying up late though.
Personally, I find pithy comebacks like this to sound convincing to but to actually be mostly devoid of utility. They tend operate as a sort of social persuasion tool, but I'd like to Get Things Done, which requires deep domain knowledge.
One thing I've been noticing with my current clients is the vast amount of churn generated by them only having a patchwork understanding of their overall architecture. Some grok the low-level infra, some the deployment levels, others the backend pieces, and others the frontend stuff. However, since there is no story collecting the pieces into a cohesive whole, we see them treadmilling through different tech stacks in an attempt to gain "velocity". They're definitely running fast; it's just they're not getting anywhere very quickly.
It's a pattern I see a lot. On the flipside, when clients have devs who mostly grok the entirety of their system, then we can focus on the questions that really matter: purpose, market fit, empirical evidence from users, _etc._
Questioning things costs time right now, not questioning costs 10 times that in the future.
> It's a pattern I see a lot. On the flipside, when clients have devs who mostly grok the entirety of their system, then we can focus on the questions that really matter: purpose, market fit, empirical evidence from users, _etc._
So uh, not questioning everything eh...
Sounds like you agree with me. That questions should be reserved for the meaningful and insightful.
asking why the sky is blue when you don’t know is perfectly fine and even admirable when everyone else seems to know and you don’t. i would expect such a person to at least try some research first and then share what they learned and what questions they still have, but if someone is asking earnestly and i have time, then why shouldn’t i help?
i don’t thjnk the logical extreme you position here is correct; if it’s an interest for the person why shouldn’t they pursue it? questioning everything for me is just getting to the point that it’s not magic anymore.
i probably don’t need to know the entire kernel code base to figure out why my eth0 won’t stay up. it might help a bit to know some of it if i’m getting weird kernel-ey messages in the logs.
The question was not "why is the sky blue", but "why is the sky a different shade of blue".
Every day the sky will be slightly different. There exists a reason why, and given effort you could identify the differences and relations.
Or you could go to work.
> It's a pattern I see a lot. On the flipside, when clients have devs who mostly grok the entirety of their system, then we can focus on the questions that really matter: purpose, market fit, empirical evidence from users, _etc._
Subconsciously, I think some people know their system designs can’t hold up to sustained scrutiny. And their technical aesthetic is weak, oftentimes it is a mental symlink to “best practices” and “what HN/X says.” So talking about the way it has grown, and the why, can provoke anxiety.
So you end up in this weird place where it feels like there is a big pull to make tech about everything but the mechanics of what you actually do on a day to day basis. Or rather, it’s viewed in a very transactional way: “I learn this skill to get that job.”
> Questioning everything gets results, not friends.
I'd argue that it depends how it's done. I used to work with a brilliant Data Scientist who had a PhD in AI. She asked so many questions — often about fundamentals — that it was like working with the Riddler. She generated a ton of insight as a result and identified missteps before they happened. Despite questioning everything she was arguably the most liked person at the company because of how she approached it. It didn't feel like being grilled, but rather that she was super interested in people and what they were working on.
My point is that questioning everything can get results and friends if done correctly.
Not surprised that they are a "she". Guys tend to be more aggressive in their questioning, and guys are also more likely to be considered a rival (to other guys, at least).
To make this gender neutral: When questioning, don't be aggressive, and try to obfuscate that you are a potential rival.
One thing to keep in mind is that software for a dialysis machine requires a different approach from some overly generic CRUD app exposing how many mansions you have or something.
There is a lot of asinine software in the world and that’s fine. Lots of “real problems” are pretty asinine and don’t require heroic engineering feats.
Just slapping some bullshit together is good enough in a frightening number of cases.
Hate it too myself, but I have come to accept I can either prove them wrong by building my own business and outcompeting them through my supposed superior engineering or just swallow that I’m yelling at the clouds.
> Hospitals don't check software quality when making purchasing decisions (much beyond "this button doesn't work")
This! The worst piece of low quality garbage I've seen from the inside is hospital software. Nobody really cares or understands software on the hospital side, so anything goes. The product is inherently complex, so there are only a small number of large companies who don't care either since they can rely on customer lock-in, no matter how crappy their service is.
The thing with software in which bugs can kill people is that if there is a bug and it does kill or hurt someone, the engineers who made this bug will be charged with a felony.
What else can you do but look at files and code to figure out problems? And why use a monospace font for prose? I agree otherwise though, it can bite you in the arse if you don’t understand the layers of abstraction you’re dealing with. It’s the reason I sort of don’t like browser-based stuff. We already have a full OS stack for doing stuff, why add another on top of it?
It's not only developers we had a regress in skill in the art/vfx world as well. If there are no tutorials with step by step instructions on youtube most people can't resolve visual/art/vfx problems anymore. It's about the most basic things if you get people with "art" degrees today maybe 3 out of 10 can draw or sketch. If they are not in front of a computer they can't communicate visually. They are also not able to adept or solve problems on the fly which is really important if you supervise on set.
At a certain point software engineering may come to resemble medicine more than mathematics, where the lower layers of operation are known to be poorly understood and innovations emerge from tinkering rather than derivations from first principle.
It seems like many important innovations we use today were derived from first principle.
Rust uses algebraic datatypes with ML-style type condtructors and Hindley-Milner type inference to achieve, for example, types like Option<Box<T>> or Option<&T> or Option<&mut T>, so that you're forced to check for the case of a null pointer (None), and if it's not in an Option, then it's guaranteed not to be null. This was originally conceived of mathematically in the 1980s and then implemented in ML.
The Hindley Milner type system was devised in 1969 from the typed lambda calculus.
Golang was influenced a lot by Communicating Sequential Processes, a formal system for describing concurrent behavior in systems, first described in 1978 by Tony Hoare.
Graph theory is a field of math that comes up a lot in computer science, in marketing, networking, compilers (register allocation and functional programming language interpretation).
The only innovation I can think of that came purely from tinkering is Rust's borrow checking system, and they're still trying to formalize that while improving it.
Derivations from first principle are always important.
563 comments
[ 2.6 ms ] story [ 345 ms ] threadI know Javascript, React and Express like the back of my hand. I can solve absolutely any problem I've run into with that stack, I can't even remember the last time I was stuck on something. But I've ended up on like 5 projects in a row with stuff like Typescript, NextJS and NestJS because they are red hot on the fad scale right now. Despite that, I still don't feel like I know much about them because I set that bar reasonably high, and out of all the people forcing me to use tech I don't want to use, none of them even come close to reaching that bar.
I'm sick of asking fairly basic questions about frameworks and libraries I'm not as familiar with, and getting back "oh, I dunno, have you tried...?" 15 times in a row before solving the problem. That's not an efficient way to work, no matter what 90% of the industry seems to think.
In the example at the end, people don’t necessarily need to know what files and folders are so its not bad that they dont know, it’s better for a new developer that they have a blank slate to learn about trees of storage paths, as the skeumorph is outdated. just like the save icon being a floppy disk was so outdated that instead of replacing it, people realized we dont actually need save icons anymore and everything should just always save
But the problem today is that very few people, in tech not just the general public, understand the low level stuff, especially among new people entering the field.
Furthermore software abstractions tend to be far more leaky than abstractions in the physical world you mention. And when they leak you need someone who understands what's going on under the hood.
Actually I don't think this knowledge is most important for people forming their own businesses at all - they have many other problems to sort out. But I think it absolutely is valuable working for someone else. These people are the "goto guys" and, ultimately, a "technical insurance policy" for the companies they work for.
A company producing tech products without at least a few people on staff who can dig into and fix almost anything is in a difficult position ultimately.
> Some students today apparently don't even know what files and folders are
…to a slashdot thread that links to a PC Gamer article that rephrases and links to a Verge article that links to Stack Exchange questions.
> Replaced docker with VMs
This doesn't make sense to me, as Docker and VMs are not the same thing. Replace LXC with VMs, sure.
It won't be better than docker, for sure, but if you're going to use VMs and stay sane, you'll need to reinvent it.
But operationally yeah I agree, the stricter separation could make it easier to use a VM to get stuff done.
How is Clojure a layer of abstraction that had to be removed and replaced with Java? What do you think that Clojure was abstracting exactly?
Same question for Typescript. In fact, Typescript is a layer on top of Javascript so I could say that you actually added one more here.
The post and the url reminds me of the Unix Hater's Handbook. https://web.mit.edu/~simsong/www/ugh.pdf
The problem with the abstraction tower is not that it is a tower, but that it is square, when you can't tell where you are relative to the abstractions above or the abstractions below, then your abstraction gradient is too small and you are just generating mush. We have too much mush.
It’s enough to make a technologist erect a silo.
I had to explain the problem with a single wire signalling bits in a series, recipient having to de-serialize them into some data structure. Then I had to explain that TCP emulates such a single wire using small packets.
I think that they have understood, but it was a funny feeling explaining this to someone who routinely deserializes form and JSON data, then serializes them into SQL queries, then deserializes query results in order to serialize them into templates or JSON.
The layers and layers of indirection obscured what HTTP is for me, and it took me too long to understand it.
Have a look at https://learnbchs.org/easy.html for a real "oh that's what HTTP is?" moment.
I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge.
As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to the problems and logic behind all those new solutions. Understand how we come to have JavaScript and the browser mess we are in and so many other curious features of todays digital world.
I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc.
The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path.
For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option.
We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals.
I am continuously hiring new people, mostly senior devs and tech leads. Maybe one in a hundred has any understanding of what virtual memory is. Or why one process can't access memory of another process and, if you really want it, how to set it up.
Two decades ago that was common knowledge.
It went away just as the knowledge of how networking works. Currently, if they try to open a connection and it does not work they are pretty much at a complete loss to what happened, where and how to fix it. Aside maybe from a simple problem like DNS resolution error (and not even that -- most devs don't understand how DNS propagation works, for example).
So if those problems happen, they are mostly deferring to me as a guy who can solve every problem. I see no clear path to achieving a goal of having someone else to learn to do the same.
If at least was able to purchease a home I wouldnt have this pressure.
I guess in the US is different because people have higher salaries.
You have to have the time/resources to dive that deep. Or maybe it's your job, line some of the folks in a local company that work in hard problems.
But even they are not that well payed.
It's certainly much easier to find the time before you have to support yourself
Finding time to learn if this is not your day job is another problem, but a solvable one if you're willing to learn.
It’s not too different from pro tennis players, who typically start playing before the age of 8.
For those at the top of the game, computing is a calling as much as a job. They will do substantial learning on their own initiative.
Your latter point is far more important to the matter. Those who treat it as a passion more so than a job, are more likely to be the trendsetters. Growing up and being free from responsibilities makes it easier for that thing to become your passion.
And let's also not forget, a few decades ago, computers weren't exactly a cheap thing for parents with little understanding to let their kids tinker with at will. Being born in a family with enough wealth to get a computer, enough wealth / understanding to let a kid tinker with it, was an immense boon. A long with anything that type of family tends to have going for it alongside wealth. It's not that far-fetched an idea that it's the other things, rather than the early age interest of the kid itself, that got them into such a position later in life.
I think it's important to keep things in perspective here - when I took my first programming job the first week was setup. On my team, our onboarding has you branch, review and merge on your first day. Those abstractions allow for people to focus on the areas they're working on, and iterate there. A game developer doesn't need to understand the complexities around js type conversions, any more than a js developer needs to understands rendering thread latency in a browser.
On every team I've ever worked on, that headspace from the team would have been much better served by understanding the product and project, IMO.
It's not about any particular detail, obviously.
He himself does not ever have had to previously see /proc/pid/mem before in his life in order to solve a problem involving it.
However many times a day he gets a question of any kind, that's how often.
also, devs seem to not understand that a computer network is basically a massive distributed system (on the routing level anyways), and that things inside this system can and do fail nearly constantly.
things like latency, endpoint connectivity and available bandwith is not a fixed given!.
So many devs in my experience seem to write code which assumes the network is always available with the same characteristics as the point in time in which they wrote the software.
Not to even mention devs are even entire companies building HA mechanisms which rely on ethernet connectivity to work, making it very hard if not impossible to stretch them properly across network segments/locations without stretching ethernet. (which brings its own set of problems).
Assume everything is broken, all the time.
When "the cloud" came out, I could easily understand the magic behind it all, so it made a lot of sense to me and it still does.
I feel "computer literate", however, even a lot of solid but younger engineers I work with struggle with lower level concepts and just don't have the knowledge and experience with things like networking protocols to be able to solve hard problems.
I'd love to hear from somebody with experience in scientific or other disciplines and IT who could contrast their fields with ours. I only know that I have had to go out of my way to pick up a historical component to my IT education, whereas it seems like a historical foundation is part of traditional math, science, and engineering training.
Vernor Vinge was right, though-- software (and hardware) archaeologist will be a real job.
And then there's the mix between the two in languages that rely heavily on powerful type systems.
I know I have read about it in books as a fiction, but I assume it must have already happened here before.
Maybe its an oxymoronic question to ask, seeing as if its lost we might not even know its lost, but more in the vein of "We put the lime in the mortar because this is what we have always done", unaware of the actual properties of lime when interacting with concrete.
Usually following the fall of empires. Ibn Khaldun (1332-1406) wrote:
"Perhaps they have written exhaustively on this topic, and their work did not reach us. There are many sciences. There have been numerous sages among the nations of mankind. The knowledge that has not come down to us is larger than the knowledge that has. Where are the sciences of the Persians that ‘Umar ordered to be wiped out at the time of the conquest? Where are the sciences of the Chaladaeans, the Syrians and the Babylonians, and the scholarly products and results that were theirs? Where are the sciences of the Copts, their predecessors? The sciences of only one nation, the Greeks, have come down to us, because they were translated through Al-Ma’mun’s efforts. He was successful in this direction because he had many translators at his disposal and spent much money in this connection."
Maybe not quite in the context you were asking, but philosopher Alasdair MacIntyre argues as the premise in his book After Ethics that this is what happened to the philosophy of ethics after antiquity.
Copypasting from its Wikipedia article [0]:
> [After Ethics] begins with an allegory suggestive of the premise of the science-fiction novel A Canticle for Leibowitz: a world where all sciences have been dismantled quickly and almost entirely. MacIntyre asks what the sciences would look like if they were re-assembled from the remnants of scientific knowledge that survived the catastrophe.
> He claims that the new sciences, though superficially similar to the old, would in fact be devoid of real scientific content, because the key suppositions and attitudes would not be present. "The hypothesis which I wish to advance", he continues, "is that in the actual world which we inhabit the language of morality is in the same state of grave disorder as the language of natural science in the imaginary world which I described." Specifically, MacIntyre applies this hypothesis to advance the notion that the moral structures that emerged from the Enlightenment were philosophically doomed from the start because they were formed using the aforementioned incoherent language of morality.
[0] https://en.wikipedia.org/wiki/After_Virtue
The original machine had a base-plate of prefabulated aluminite, surmounted by a malleable logarithmic casing in such a way that the two main spurving bearings were in a direct line with the pentametric fan. The latter consisted simply of six hydrocoptic marzlevanes, so fitted to the ambifacient lunar waneshaft that side fumbling was effectively prevented. The main winding was of the normal lotus-o-delta type placed in panendermic semi-bovoid slots in the stator, every seventh conductor being connected by a non-reversible tremie pipe to the differential girdlespring on the "up" end of the grammeters.
— John Hellins Quick, "The turbo-encabulator in industry", Students' Quarterly Journal, Vol. 15, Iss. 58, p. 22 (December 1944)
Anyways, that just to say that people in post-Roman Empire Europe lived out the reality of lost knowledge. They lived with the remains of incredible art, architecture like the coliseum, public works like the aqueduct, etc. But they wouldn’t have known how to reproduce those works. How to make concrete, etc. was knowledge that was lost to them.
The decline in big public works were probably the result of the loss of an empire that could cocentrate resources.
I thought concrete did continue to be used, but differently?
This all seems to spring from the myth of "dark ages"
The Antikythera mechanism comes to mind. It dates to the 2nd century BC. Nothing like it was constructed afterwards, until the 14th century.
https://en.wikipedia.org/wiki/Antikythera_mechanism
The thing is, we tend to lose the knowledge that is deemed useless. Anything considered useful is well spread. The problem is when our opinion doesn't match the reality, we have something around a generation to prove it, or the chance is gone.
Mathematics is full of examples of things that were discovered again and again, until somebody found a use for them and they entered our textbooks.
Actual IC design (VLSI etc) is still a purely analogue field and all digital technology is fundamentally analogue at its core.
The reality in that sphere is there is little reason to limit oneself to the constraint of expertise in analog circuit designs when one can achieve the same functional outcome digitally, and use those skills more broadly.
Only where component sourcing is artificially limited, or risks of digital operation sufficiently large, and where the job market will support it, does it make sense to proceed in growing in analog circuit design expertise.
Meanwhile these designs are nearly all being functionally lapped by those in general industry, which participates in all the accelerating gains of digital technologies.
Much as I'd like that, I disagree.
The momentum has always been against that. We're members of a cult of the new, which does not value or even acknowledge the old. Which is why we keep reinventing the same solutions but with different tech stacks.
This drives developer value down which drives developer salaries down, so I don't see it changing.
FWIW, my history of math class as an undergrad was a math elective that almost nobody else took. Other math students didn't want to learn history and other history students didn't want math. And the curriculum stopped at the 18th century, before things got really interesting.
This just isn't true. We have Heartbleed because we refuse to move off how we wrote software 30 years ago[0]. HTTP was invented in the 1980s[1]. REST in 2000[2]. TCP/IP in the 1980s[3]. Ethernet the same[4].
I'm pretty sure we acknowledge and value all of those things. Our field, depending on how you define it, is only about 70 years old. Those things are pretty old in those terms.
[0] https://queue.acm.org/detail.cfm?id=3212479
[1] https://en.wikipedia.org/wiki/HTTP
[2] https://ics.uci.edu/~fielding/pubs/dissertation/rest_arch_st...
[3] https://en.wikipedia.org/wiki/Internet_protocol_suite#Adopti...
[4] https://en.wikipedia.org/wiki/Ethernet
Some of this just seems to be due to the maddening popularity contest of technologies and frameworks within companies. There is a frantic rush to make sure you're doing everything that your competitors are doing, but not much introspection regarding whether things will actually help your company. You're just keeping ahead for the sake of keeping ahead.
I think today it’s probably more true than ever. Schools are under pressure to turn out grads who will be productive on day one and I think it’s shortsighted.
The problem was that there was a shortage of developers, and that most universities still only had CompSci degrees, not Software Engineering degrees, coupled with those degrees were 3-4 years long... So enter the code bootcamp, where you're basically given a hammer, and taught that every problem can be solved with it..
Now those bootcamps aren't all like this, and there were and are good ones out there..
...but from what I've seen (I want to stress that this has happened rarely compared to the rest) a graduate of one of faster bootcamps, that learned a single language, like JavaScript, and try and solve every problem with whatever answer they found someone else upvoting.. without any amounts of thinking if it's the right solution for this specific problem.
The problem is as your professor said. Do you want an education or training... And also as you said, that you can't turn out effective software engineers with a quick or short amount of education.
Is a Software Engineering degree really a thing? Perhaps outside the US? Can you point to a notable university that has this major? A quick search gives me the impression this is an online correspondence program type of thing.
At UCSD in the late 90s we had a Software Engineering class but that was it. Otherwise internships were supposed to prepare you for your careeer.
They do have a full SWE track as a Master’s program, which I did, and quite enjoyed.
Penn State has one.
I cannot stress how important my education was to my current understanding and success as a software engineer in the field. A poor education is a huge detriment. When I joined the workforce I found many colleagues without one ill prepared for some of the basics, like debugging kills, using tools outside their domain expertise, etc.
That said I don't think a formal education is REQUIRED, but without one you need a certain kind of perseverance and thirst for knowledge not everyone has. I've had the pleasure of working with several folks in that category, so a formal degree isn't for everyone.
Today, it appears specialization is happening much earlier in the education process without a general foundation. Specialists will always be needed, and larger orgs can get by just fine with exclusively specialists, as long as their communication skills exist to prevent silos. Larger companies can afford this. However a good generalist can replace a team of specialists when the latter budget is out of the question; a huge win for smaller orgs, start-ups etc.
The old joke that a good generalist is the "jack of all trades, master of none" has some truth to it, but as with all things, it's a tradeoff. A team of qualified specialists will definitely produce better work within their domain expertise, but at a cost that can be prohibitive when "good enough" is within reach.
One wants to go straight into the work market, with focus on getting the bases, technical school.
One wants to go into the work market, with some more preparation than only the bases, polytechnic institut.
One wants to go into the work market, with overall experience, in a degree certified by the engineering order as qualified for professional title certification exam, university.
Unfortunately not all degrees, or universities are well prepared to offer this, or have staff that cares about making it happen.
Spend some time in the coin op game community... This is all we spend our time on. Well that and cabinet repair.
But seriously though, keeping these old games alive requires serious reverse engineering of both hardware and software. Especially if they have custom chips, ASICs, undocumented features, etc. But the work has to be done or these games will be lost to time... It may be a passion project/ hobby, but it's still real work.
I like your point about digital archaeology. If it's important enough to some people to keep old arcade games running, how much more important is it to understand and keep some old mission critical computer running.
It's a lot like the tech people you see in these dystopian future movies, the ones who are like "oh yeah you have an old z80 with the qxp interface, haven't seen one of these things in years". Like Scotty from Star Trek. They're an increasingly important part of the tech ecosystem.
Besides, the PlayStation based hardware of this specific arcade PCB is not that far off from the ubiquitous 32-bit microcontrollers that underpin modern digital society; in fact the MIPS R3000 derived instruction set even bears a striking resemblance to that newfangled RISC-V thing every investor is talking about. And yet most of my 19-year-old university classmates want to stay away from "scary" embedded or kernel work in favor of web or game development.
[1] https://psx-spx.consoledev.net/konamisystem573/
It seems that with more abstraction, more people throw up their hands and say "I'll never understand it", "The designers of this system did things in the stupidest way possible", "It's magic" or (in the case of the financial system) "It's a rigged game" "You're working for the man"
What all of these points of view have in common is that the people who hold them have given up trying to understand the world around them. Instead they take the intellectually lazy way out and compose their own pet theories of why things are the way they are, this is where tinfoil hat stuff comes from. This is dangerous, and is sort of the point of the original article.
I keep telling people that "the world is still knowable, still understandable". You have to put in the time and learn about it. It's easy to do, you just have to do it. There are plenty of "average" people playing in these "difficult" to understand fields.
What we desperately need to cultivate is a culture that encourages and rewards curiosity. When people are curious, they take the time to learn about the world around them, and why things are what they are.
If you don't master the world around you, it masters you.
During the crypto boom I distinctly remember hearing someone say something about creating a decentralized Discord. I was like... that's IRC.
Nobody said that. But spending a week is definitely an option to all programmers, and many would benefit from it.
https://training.linuxfoundation.org/training/a-beginners-gu...
A motivated person can run through it in a couple weeks while working a day job.
I know this is tired and cliche at this point, but literally this week I sat down with ChatGPT and asked it to teach me how to write WAT (web assembly text format) so I could understand how memory is managed at a really low level (but not so low that I risk crashing my computer).
It turned out to be super valuable. This is where AI shines for me - I can ask it any question that pops into my head, and also validate whether what it's telling me is true by running the code and seeing whether it works. It was amazing.
If you're curious, I'm fine sharing a link to the chat:
https://chat.openai.com/share/583bf23b-b43d-4566-956a-e92b6f...
I’m sure it’s not error-free, but the speed of learning makes up for it IMO. Like I’d been struggling for a while to learn tree-sitter, the documentation is overwhelming. I had a chat with AI and got a working solution for my problem in probably 4-6 hours, and now I can write tree sitter grammars without help. It’s really incredible.
His book Code is fantastic. He starts at simple battery and lightbulb circuits and builds and builds towards a simple CPU.
He also wrote The Annotated Turing which is a breakdown of Alan Turing seminal paper and you only need high school math to get through it.
When I was in school my favorite course was compiler design and we used Compilers: Principles, Techniques, and Tools (aka the dragon book). It’s one of the best textbooks I’ve used but that was 30 years ago. There might be something better now. Understanding parsers and lexers and (especially) state machines is something that will serve you well.
Ah, I'm interested in parsers these days (I just wrote one for parsing org data, I am a bit unsure about the architecture of the project but at least it does what I needed personally). I am right now at a friend's place that just showed my the dragon book from its shelves, that's a funny coincidence. I will check out Code.
Python, Linkers, and Virtual Memory by Brandon Rhodes (Python core dev)
https://www.youtube.com/watch?v=twQKAoq2OPE
(Most of the content is not actually specific to Python)
He beautifully pulls back the curtain on so many lower level concepts like virtual memory management, dynamic linking, heap/stack, fork(), copy-on-write.
The talk is broad in nature, not deep. It takes you just below the surface of many magic black boxes, and, as you put it, enhances your proximity with those topics.
For me, so many things clicked in this single talk:
- How virtual memory works (incl. paging in/out, swapping)
- Why there's those discrepancies between RSS / PSS
- What segfaults and page faults are
- What actually happens when I get errors related to dynamically linked libraries, either at build time or runtime
- Actually understanding the output of top / ps
Godbolt [0] is an invaluable resource. But simply setting up tasks to yourself and completing them may be the best course of action. Then you'll find whatever resource you need for a concrete objective.
For example, if you have a week, I'd suggest to start "in the middle", and move up and down according to your tastes.
- Write a hello world program in C, compile it and run it. Try to compile it statically and understand the difference.
- Ask the compiler to produce an assembly file. Try to build the executable from that assembly.
- Try to write a hello world in assembly language by yourself, compile and run it.
- Write a program in assembly that prints fibonacci, or prime numbers.
- Now, forget about assembly and move upwards. Download python source code, compile it, and run a python hello world with that interpreter.
- Look at the Python source code, which is written in C, and try to follow what happens when you run your hello world.
- Try to change the "print" function of the python interpreter so that it prints your string backwards.
Depending on your experience, you may need more than a week (5 days) to complete all these steps. But that's OK! In a few months you can find a new spare week and continue your work.
[0] https://godbolt.org/
To build mobile phones, we need a lot of tech built on tech build on tech. Dwarfs on giant's shoulders. And there's a place for people who don't understand the underlying tech. I've got a colleague who doesn't understand any of it, yet can do useful frontend work. But when you stand too high above the ground, the problems in the article become real, and that will get us stuck.
Programmers at the time went from thinking these kit computers are toys to watching all the old ideas like virtual memory, cache, pipelining, networking, and multiple cores get reintroduced as the anemic transistor budget exploded. None of it was particularly novel, but the slow introduction of older ideas was a great way to learn all this stuff from the ground up.
However, if I am using an abstraction like an ORM without understanding what is happening with the database under the hood, or why it's getting slow all of a sudden, it is bound to bite me in the ass.
I shudder to think of how much money and energy the world is burning every second just because of this one alone.
It can take days for them to even track an evil SQL query back to the actual code. There were fetch-one calls that literally carry a million rows back over the wire for that one returned entity. Indices get skipped due to the magic translation of a row.col.toUpper()=="STRING" into SQL.. when the db collation is case insensitive in the first place.
But hey, the devs don't need to know SQL, so there's that.
Specialization always exists in an industrial society.
And web development has the worst tech abstraction of all.
There used to be a time where any Software Engineers ( i think the term wasn't even invented then ) or simply programmers would know at least a thing or two about Hardware. We now have so much abstractions we have people working in tech who have zero knowledge on either software or hardware, or any low level stuff.
That is why not only do we need some open source or open standards ( I am not a zealot for everything open sources ), we also need to simplify everything we have today. To basically refractor everything we have learned from Hardware to Software.
They know only full SPA frameworks, they have never seen a dump of an HTTP message, headers and verbs are abstract things to them. Hell, many of them don't know you can have fully functional websites with zero JS, including payment, video, login, etc.
I started to write an HTMX tutorial (https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1) because I noticed a lot of young coders don't understand what to do with it. They read the tweets saying it's nice, but when they look at it, it makes no sense to them.
It's really fun because I now remember how some senior coders looked at me, knowing nothing about compilation. I was struggling with Python packaging because before wheels, it required compiling a lot on linux, and it failed often. For them it was obvious: just install the headers, look you need the dev packages, wait, you don't have gcc?
Nowadays I happily patch nginx source code and compile it manually, but it took a lot of work to learn a minuscule chunk of what all those guys knew by heart.
Because they started with assembly.
Around I guess 2017 or even 2016? I used to think this is some sort internet troll comment about people never seen a dump of an HTTP message, until......
>Because they started with assembly.
It wasn't necessary starting with assembly or something low level. ( Although that certainly helped ) We have less entertainment, more time, and no Internet ( for most part ). Things also dont work all the time. And we have to spend time figuring it out ourselves. That is where all the knowledge comes from.
The difference is where you start figuring things out.
Before, you needed to figure things out at your level, because it was the immediate area of mystery.
Now this level is generally solved, you need to figure things out at a different level:
- filter out the mass of irrelevant information, the out dates one, and the one from spam
- understand how all those complex abstractions interact. We have good resources on how each work individually, but the carthesian product of the monsters we build with them, we obviously don't
- debug some things that don't work when the magic fails, way below your level, or behind a service, where nobody is looking
The problem before was scarcity of info, lack of standardization and roughness of systems.
Now it's abundance, opacity and too much sophistication.
But everybody still have to figure things out. Just no the same challenges.
I'd argue that the "find and synthesize" generation have an advantage within contemporary software paradigms because of their experience but, without deep knowledge of the foundations they are building on, they might be disadvantaged when it comes to imagining/creating new paradigms.
Then again, first-order thinking seems to be easier when you're not marred with the traditions and conventions of the past so maybe this isn't actually a disadvantage.
Finding and synthesizing doesn't do you any good if you're unable to meaningfully apply it or understand what you're applying and what you're applying it to.
Trial and error also doesn't do you much good without the ability to back it with knowledge and to find the relevant knowledge.
This makes valuable anyone with long attention spans and immunity to boredom.
For me, it has taken some habit forming (and habit prevention), but I have learned so much in the past 10 years since giving up gaming and social media. HN is an occasional vice though...
When I was teaching programming, I had a fun party trick whenever we got to HTTP. I'd fire up netcat (in listen mode), then connect from a web browser and "serve" a website by hand. I'd show the students the HTTP request that came in, and just manually type out a simple HTTP response and they'd see it appear live in the browser. Its pretty magical.
And once I'd shown them that, I'd write (by hand) an HTTP request to wikipedia or something to show them how its symmetrical.
Of course, real websites increasingly do HTTP2 over TLS or something. So unfortunately its not as "real" as it once was. But if wide eyed look on my students faces is anything to go by, it was a great lesson.
The email telnet thing was also a good learning experience. Gmail's servers are fun because you can see the designer's cute messages; if you forget your EHLO, they'll throw an error (or at least they did years ago) that it's polite to say hello first.
This is not true. I run a mail server for myself and some friends. It started as an old desktop running under the desk at a university, transitioned to the back corner of a server room when I was working as a network engineer, and now it runs on a Raspberry Pi in a closet of my house. I had to pay extra to get a static IP at home, but everything has been off the shelf and DIY. My current Raspberry Pi has been running for nearly ten years, with only one interruption (the SD card failed). The idea that you can’t run your own mail server is a myth, and I think more people should do it. It is not hard, and you will learn a ton.
(I used to run mail for a large corporation, so yeah, I know a lot. But I learned how to do it by running a mail server out of my dorm room.)
And indeed, the first time opening a POP3 session to an email provider through telnet was also an amazing feeling; maybe even more visceral thanks to its "live coding"/immediate feedback aspect; but personally just a small bit less foundational. So, anyway, through this whole story, what I really wanted to say is - thank you for your service and approaching your teacher's post in a great way!
Unfortunately chrome has stopped showing cookie headers in the network tab. You have to look somewhere else for those, and I think at that point you’re just seeing the current cookies, not the raw headers. Maybe there’s a security reason for that, but it is a further abstraction layer of “we’ll present the information a certain way” rather than “we’ll just show you all the information that the server sent back” that I expect will grow into other headers (maybe already has) and further remove us from seeing the raw details and having a clear understanding of what’s happening under the hood.
(You know it’s possible HTTP/3 cookies are even passed in a different way under the hood from other response headers, and that could be part of why they’ve separated them out. I hadn’t considered that possibility.)
https://www.networksfromscratch.com/
It will take me 10 years to finish it at the pace I’m going though :)
And for once in my life, I'm actually going to sign up for your email list because I want to know when this thing is done!
Sure, a frontend dev who strictly works on that won’t know much about, say, the OS layer. But they don’t have to, it’s not part of their job in any way.
There is no shortage of young people that work on areas that makes them require deep knowledge of many different layers of the stack, it is simply not necessary for every IT-related job. But one can absolutely pick it up if they want to.
No wonder a lot of modern software is terrible.
To be fair, the last time I "opened up" a computer and changed its RAM was about 10-12 years ago, and since that time I've only worked on a Mac Mini and on a laptop meant that "opening up" computers became a thing of the past.
The HTTP protocol, HTML and vanilla JS are not "the OS layer", and yes, it's part of your job as a web developer to at least understand the basics of them. There are many so-called "frontend devs" nowadays who literally only know React, and if you asked them to create a basic webpage where clicking a button changes some text in an element without React, they'd be completely lost.
Usually the problem with this kind of developer is not just that they don't know, it's that they don't care. They WILL inevitably run into problems that require this basic knowledge to solve because the frameworks can only hold your hand so far, and instead of trying to figure out what's going on under the hood that's causing their issue, they will instead shrug and go "it's not part of my job" and just write a bunch of garbage spaghetti code in an attempt to work around the issue.
Pretty much anyone who has been around long enough has met someone with years and years of experience doing exactly one thing, whether that's WordPress or java CRUD or something else where they have blindingly obvious gaps in their knowledge that someone with their experience shouldn't.
The lack of deep knowledge isn't limited to a single field.
This is an entirely new business domain, it’s not really trying to be the same.
People here have way too much confidence in their interview questions being a good signal for experience. It’s pretty wild.
You are proving my point. You should care.
My point is that your question is some esoteric gotcha party question that may as well be out of Trivial Pursuit. But you’re treating it like anyone who doesn’t care is just being a bad engineer. There are countless ways engineers spend their time, and choosing what to work on is the most important choice of their careers. It’s on you to justify the claim that knowing the difference between a JS form submit vs browser submit matters at all, let alone that it’s a distinction that comes up in day to day life.
For you that would be trying to work in ML not understanding any of the theory or how it works, but having used OpenCV with some premade models a few times.
For a frontend web developer who, as a large part of their role, will need to communicate with backend systems… that’s not understanding how their FE web application actually communicates with BE systems.
And from experience… yes, this is very common. And it has a noticeable impact on their effectiveness. Trying to debug why some interaction between your application and the BE application isn’t working while thinking the dev tools network inspector is just black magic and nonsense makes your job substantially more difficult.
This does make these people bad engineers. They are not able to understand and solve a huge class of the problems that they face day-to-day and instead (in my experience) often fall back on “just try a bunch of different things until something works for reasons I don’t understand” which is a poor way to approach work and leads to overly complex, buggy, brittle systems.
I think you do. GP just phrased it oddly.
The distinction is between a traditional HTML form and browser page load, vs XMLHttpRequest.
Ultimately it's all HTTP though and on that level, there is no difference whatsoever.
I've started nearly every position in my career in a different domain without prior knowledge, including workflows, protocols and the languages that implement them. I may be an extreme case, but we exist and your process inherently excludes us. I've worked in robotics, consumer electronics, healthcare, fintech (not even in that order, although that probably would have made more sense.) I've delivered in each domain as well as my expert colleagues. But if you asked me to implement a LLM, I'd shug and tell you I don't know how, but I _would_ explain my process for HOW I would learn to.
(Edit: typo)
I do get your point but depends on the role you are hiring for. I have had candidates get upset at me (for example: bootcampers) because they couldn't explain how to submit a basic HTML form but wanted me to look at their github "portfolio" that was done during the bootcamp with React/Express Code and what not. Their title was "Front end React Developer". That is a problem and I usually wouldn't hire those people.
For example, I would take a react developer with obvious gaps and a strong willingness to learn on my back-end services team over a passable back-end dev who was set in their ways (unwilling/uneager to learn new tech, entrenched opinions stated as fact, etc.)
Naturally, my approach is not a one size fits all situation and a great deal depends on org structure and mentorship opportunities being in place for it to work. The benefit being you avoid monoculture "silos", and have more cross collaboration and transfer opportunities between teams (some may call this "full stack", I wouldn't)
Several high performers on my team are "self-taught", work comfortably across several languages today and can pickup new tech easily. They came in knowing their "one stack" at the time. If they were "bootcampers" or not seems irrelevant and somewhat reductive/offensive.
That said, if the candidate in question shows _no_ understanding of their problem domain, can't reason their way out of a paper bag, and get visibility upset from questions when you try and tease that out, that is a certainly a red flag in my book.
With respect to your example given, however, it could be argued that http details because of the abstractions in place inherent in react, aren't critical domain knowledge for what is essentially a UX dev. If they can ship an experience your users enjoy more than the candidate that can recite the HTTP 1.1 RFC and cannot, then what have you gained in hiring the latter other than maintaining a culture of pedantry?
TL;DR it comes down to how much risk your team is willing to take on, as there's always risk inherent in hiring ANY candidate, including those that don't already check all your boxes in regards to tech know-how. I'm merely stating that by overlooking candidates that don't fit your self-imposed mold, you're likely missing opportunities for rewards that can pay dividends when it does work.
I rarely do any of that, usually my abstractions are container layers, a libc, and then a kernel between me and registers and dma. But that is a lot more layers than you'd think. I can't even understand the boot process on a modern machine with tpm. And i have no clue how many processors or controllers are in my computer. Every usb controller has an entire arm core in it, mice and keyboards have 8051 running c-ish code someone somewhere wrote, there is no bottom to the complexity.
Similar with mediatr, I know conceptually that it "just" checks for the right classes in the loaded assemblies. But still feels like some weird incantation to use, to me. And I have such huge knowledge holes across all layers.
Sometimes i feel my lack of knowledge to such a degree, that I question whether I should even be a programmer.
This wasn’t in like 2010 when Firebug was just coming out and people still used Firefox as a rule, this was like 2015 when dev tools was in every shipping browser approved for corporate use by a technology laggard company.
So the moral of the story is just blame DNS first.
I'm about half a decade into my career and I've recently tried to take note when I hit milestones or achievements, even little ones.
I was helping an intern with a tool and it didn't have support for what we needed it for. Since the tool was open source, I just cloned it and patched it to add support for what we needed. When I told the intern this, he couldn't comprehend that I was so blase about modifying this tool that seems like witchcraft to him.
But what I quoted, I feel deeply. I've worked with so many people with so much knowledge they can't possibly communicate it, and I'm only starting to really understand the tech around me. I didn't think anything of patching the tool at the time, but its important I look back so that the younger version of me can see the progress I've made.
A lot of CS skills are generalizable. Knowledge is one component you pick up with time. A good education, self-taught or otherwise, should allow you to drop into any sort of code and be effective without much spin-up. College covers CPU architecture, assembly, networking, operating systems, web, algorithms. This is not esoteric stuff, it is very standard and you can get it all in class or from textbooks free online!
It’s tough if you feel a degree of responsibility for their success. Mentors are one of your greatest assets early on (and arguably later as well), and to try hard to have them succeed and thrive only to see them languish on trivial tasks is awful.
I think part of the problem is that CS education where I live is awful. The kids come out of school expecting real work to be wildly different than it is, and it hits them like a brick wall.
Today, there are probably a vast number of entrants who heard that CS is the ticket to a high paying job, and they are also told that an internship is a vital bullet point on their resume, if not a guarantee of a job at the internship site.
Then, as now, students studied under the constant drone of "you will never use this stuff once you finish college." They still have to decide if they're actually interested in the subject matter or not.
A good bellwether of career interest is the students in the youth symphony. They've all aced every subject in high school, plus rocket club, gentleman sports, and orchestra. The program for the end-of-season concert will have a little bio for each graduating senior, including their college interests. Half of these kids want to major in CS.
Once you have the latter, either by having built such a codebase, having worked in one, or even having experience with puzzles or games requiring multi-step planning and understanding of the potential failure points at each step... it's absolutely transferable. But there are also a lot of people in our industry who have memorized interview questions and see their role as churning out components. And while arguably that's not a CS education, it's being called the same, and it does a disservice to those people's careers.
I graduated with a Computer Engineering degree, did assembly, C, microprocessor design, computer vision, and know a good bit about lower level stuff, how memory works, how networking works, etc. All the stuff people in this thread seem to be lamenting the lack of. But I was also a shitty employee fresh out of school because I didn't know anything at all about modern software development because there was absolutely no time to learn that stuff as well.
I still had to spend a lot of time getting good before I was worth anything, just as these "new frontend devs" will, as well.
Yeah, but modern software development is trivial to learn, particularly in comparison to a computer engineering degree. You see "developers" here on HN gloating all the time about how they didn't need any post-secondary education at all to get their jobs; these frameworks are literally designed to be usable even by minimally skilled coders. You were in a much better position having to learn modern software development after a computer engineering education rather than having to learn the rudiments of computer engineering on the job after getting an education in modern web development.
LMAO c'mon. This basically undercuts anything else you say here. Go teach your grandma react and see how trivial it actually is.
It's more likely that I will do something like "a full web app from installing the OS on a fresh laptop to coding it to hosting it on server".
It would also be more beneficial: plenty of tutorials for individual techs, rarely they teach you how to integrate them together and go beyond the toy example on localhost.
But for that I need a better platform than substack. Therefore that's not going to happen anytime soon.
I agree with you but HTMX? That's a big abstraction layer. A good one but still, not really a way to avoid layers of abstraction. Pure JS makes more sense.
We work with both high and low level tools where I work, and in my experience there isn’t necessarily a lack of understanding among younger developers. The CS courses still teach you a lot of the basics, and to actually use what you call “hype” tools efficiently you typically need a rather good understanding of how they work.
I actually don’t think we have enough abstraction, especially not in the DevOps field. In my opinion we’ve overcomplicated the deployment procedure without making sure it was abstracted. We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge. I’m actually fine with infrastructure as code, but it needs to be templated to the point that our developers simply “order”resources based on whatever template they want to deploy so that they don’t have to keep up with the constant changes or learn how networking in enterprise organisations work.
As far as the CPU stuff goes. Young developers do a lot of low level code. We write a lot of embedded software for our solar plants, and a lot of the time, they seem far more capable and “modern” in their approaches than the “old guard” exactly because they’ve been taught the same curriculum but also all the lesson learned by the “old guard” along the way. That being said, it’s not really necessary for a lot of developers to keep an active knowledge of how an x86 CPU works, because it’s very unlike that they’ll have to work with it. It will often be far more useful to know how various types of ARM processors work, as that’s something a lot of us actively work with.
I am afraid that many businesses prefer the vision where the software developers do the operations too, and the company can fire the operations department. Consider how much money you could save... before it all falls apart.
Short term optimization is how managers get their bonuses. The trick is to move on to the next project before the old one falls apart. Then it becomes someone else's problem.
It is basically idiotic to me, as a highly paid senior application dev.. that they basically want me to spend any cycles on stuff that would ordinarily be done by someone making 1/5th my TC.
If I can do it more than 5x as fast/efficiently, sure, but I don't. IaC is something I worry about at the start of a project, as hamfistedly as possible, probably over-provisioning so I don't have to go back to it anytime soon. All so I can move onto what I am paid to do - delivering functionality to stakeholders. Someone who deals with IaC as part of their full-time job will be the expert who can move more quickly & correctly through it.
Sometimes this is just galaxy brain budget arb, where the infra org gets to show a cost save, to the detriment of the appdev org.
Did you mean for this to be condescending towards ops? Because that’s how it sounds.
Ops is not easy, at all. Even if you’re doing zero IaC, I defy the average dev to try to provision a Linux box from scratch and install, configure, tune, and maintain the stack you need to run your code. Throw IaC atop that, and now you need to understand declarative programming and OOP concepts to be able to do it efficiently. Then, there’s K8s…
Not to mention the architecture side of things. Devs love to grab whatever shiny thing they saw on a Medium blog, even when it’s a poor fit to their problem (or their problem is unoptimized code). You probably do not need a columnar DB, you just need to normalize your tables and learn relational algebra.
Ops is absolutely hard and theres a lot to learn. And in doing so you generally may end up with DevOps teams with minimal to zero domain knowledge in what the company delivers to end users.
So if you have a team of people who built a deep level of expertise in something over 10, 15, 20 years and that something is business facing, domain knowledge, etc.. then it is not of value to have their time spent on other tasks.
Often these DevOps roles are filled by more junior staff at the earlier stages of their career. This is another reason the staffing is usually lower cost.
Hint: I wrote operating systems for fun before becoming an SRE. Imagine how much fun I have explaining context switches and CPU cache invalidation and their implications on their app’s performance, to application developers who consume frameworks and look down on my salary! Building a distributed computer at scale is much, much, much harder than your Linode tutorial expectations of what ops does.
Native software is mostly dead, so operations is software engineering now, at least those parts of software engineering you folks threw out when Docker came along and turn your nose up at, and I make more money every year cleaning up the low-hanging fruit y’all leave around, so…
Now take all the stuff you are an expert at, and imagine the CTO assigned to you - build dashboards of accounting information. You don't do UI, and you don't know accounting (maybe YOU do, but your average SRE/DevOps do not).
It wouldn't make sense right? And a lot of those tasks can be done by a junior "data analyst / BI dev". So it would be an expensive mistake as well.
This is my point.
The worst problem, which TFA alludes to, is that those "layers" are not really needed, they are leaky kludges upon kludges to make some lower layer unsuitable for later needs more palatable or able to handle some unforeseen use case that's against its design. Other stuff is just added as ways to sell new enterprise tooling, support and consulting.
Seeing the layers and technology stacks being added over time can give the impression of watching some organic evolution happen, when it's often merely accumulation, rough patching, and corporate attempt to push its technologies/NIH.
But it only works so far and at some point the application becomes unmaintainable mess and the effort to rewrite it from scratch starts.
Only we can't rewrite the world after it has become so unmaintainable that nobody can figure out how to change a detail of some layer inbetween the program and the transistor.
Heck, we are already stuck trying to fix the bad design of IPv4.
It reminds me of:
https://github.com/alex/what-happens-when
and how many of today’s CS-degree holders would barely understand any of it. As someone who has also “grown up with all the technology”, I’ve learned and experienced all that. But as a percentage of “software engineers”, there’s fewer and fewer that do every day.
Under our current system, knowledge that cannot be monetized is useless. Worse than useless, in fact, because time spent learning low-level concepts is time that could be spent learning marketable skills.
The industry is effectively paying us all to forget.
Sure, learning the USB stack and differential signaling is fun, but unless my job involves implementing custom USB devices from scratch then it's pointless trivia that won't pay the bills.
I suppose that is what will keep happening. At some point, someone will decide to re-write everything from scratch and re-discover lower layers of abstraction or challenge them.
A funny possible history is everyone forgetting what’s necessary and then having the relevant information they need accidentally blocked by an AI system they have no idea how to build or fix (maybe because of terms like master/slave going against elite values).
Not completely. Curious people will always pop up, it's just that they will take as much time as you did to learn everything.
I like to think of myself as one of the curious people. Ever since I was a kid, I had a huge desire to understand how the hell does this magic box called a computer work. Decades later, as a software engineer, no matter how much I learn, no matter how deep I go, there's always this voice in my head going "but WHY? WHY are things the way they are?".
I did start top-down instead of bottom-up, starting from the high-level languages running on modern operating systems, going more and more into the (professionally unnecessary) low-level through pure curiosity, but I still believe I will get to the bottom one day.
As a counterpoint, I'd say that it's also chance to build on top of these abstractions without having your brain cluttered with how they're implemented. That's also how science grows.
And don't underestimate the new players. They're perfectly capable to understand the low level details if they have to, and they have much more resource available to learn too.
I just hope when that time comes I'm well compensated for studying erudite texts on low level computer science while chilling on my phone instead of browsing social media or playing games. But honestly, I enjoy reading and learning about ABIs, C programming, network protocols, etc... more than mindless scrolling anyway.
Like you, I enjoy learning techie stuff, but most of my friends don't. Because, IMO, most websites are written for younger people, they feel almost helpless.
Free markets don't solve every problem but they solve this one.
I'm gonna guess this same message, in different words, has been repeated often throughout history for as long as people had the ability to reflect on their life and the next generation.
So really, what's different this time? What's the difference between forgotten technology that nobody needs and an unpublished author who's works were forgotten? I think the answer is none. We just think it's more important because this is the industry we care about.
We're actually really good at reinventing the wheel in this industry, so even if some software is forgotten, you can probably bet it'll be reinvented or reverse-engineered if it's really needed.
While we should preserve technology knowledge (because it's pretty cheap to do so), in terms of the economy, sometimes things are a stepping stone and should be forgotten.
There will still be people at e.g faang/semiconductor companies
that do this stuff day2day and even push advances in those areas and have already created solid training materials for younger employees
If you're worried that this knowledge will disappear, then feel free to write it up in form of tutorials.
It was around the year 3-4 mark that I decided to knuckle down and try to improve my fundamentals (data structures, algorithms, memory models, concurrency, CPU architecture and some network fundamentals) by reading popular papers and literature and writing all of my personal projects in C and C++. I’m about two years into this study and while it’s been immensely rewarding, I’ve found it to be a huge undertaking while juggling life and a full-time job.
What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work (still Python and JS). I would love to be able to put these concepts into practice for many hours of my working day but it’s difficult to move from general web-stack development to more performance-oriented development (embedded, low-latency, OS, etc.).
My guess is that this is one of the reasons we have ended up in this situation. You can get away without knowing the fundamentals (a good sign of progress?) and that if you really do want to pursue these areas that promote building this kind of knowledge as part of your career, the barrier to entry is quite high and the positions are fewer than say a decade or two ago. I find it a shame because in my eyes, these areas are the most interesting and exciting areas of programming. It’s an art.
I think this is because of the effect OP is describing: no matter how much time you spent studying, the odds that you learned about the specific thing that will make the difference in your work this week are slim because the technology is so broad and deep. Odds are that what you choose to intentionally study is irrelevant to your work.
You've picked a slice to learn about, but those who were working on it as it got layered know it all. Once you have that kind of knowledge, the odds of being able to explain some unexpected behavior approach 100%, but acquiring that knowledge took decades.
I've found a method that does work pretty well is to dig really deep into the topics that come up in your actual work. Instead of a random sample of fundamentals, find the parts of your job that feel like magic and explain that magic. Skipping over V8 and straight to the CPU might not be useful for a JavaScript dev very often, but a deep understanding of V8 is relevant more often than you might think.
There’re huge areas of the software industry where performance matters to this day. Examples include videogames, content creation, video processing, engineering software, science related HPC, and now AI.
However, transitioning from web development to these areas gonna be hard. That software doesn’t run in web browsers, the code is either desktop apps, deep inside web backends, or supercomputer programs. And high-performance C++ is often not enough for them, depending on the area ideally you might also need GPU graphics and/or GPGPU compute.
I was lucky I have never worked on web apps. I know TCP/IP reasonably well, I have some general understanding of HTTP and HTML, but I’m totally clueless about the modern frontend technologies.
Find a chunk of code you want to optimize, simplify it as much as possible, then put it into godbolt. Match the interpreter version to yours. Then go look up the bytecodes and see what they do. Try changing the Python version to see how it differs, or a different approach to the problem (e.g. a map instead of a list comprehension).
This takes an enormous amount of time, of course, but you can learn quite a bit.
[0]: https://godbolt.org
But we don't need every programmer to understand low-level kernel stuff, precisely because the kernel abstraction is good enough at hiding that stuff. Most of us don't need to care (whereas 40 years ago, many more did need to care).
Personally, I myself never want to care about low-level kernel things. When I'm trying to write a program to do something, if I can write it without having to worry about low-level things, that's a win.
For example, I am NOT a cloud expert, and barely a novice, by any measure. When dealing with various CloudOps/Cloud Architect/etc types, without fail, the good ones are also complete Linux gurus who would be at home architecting on-prem infrastructure as well.
Unfortunately there are a lot of "experts" who are only comfortable working in abstraction and so as soon as we have an issue that scratches below the surface, it turns into an all-hands-on-deck mayday situation, looping in other teams/experts to try and save them.
I've also seen horrific implementations by abstraction enjoyers.
Imagine a continuous integration environment of the following - k8s where one pod simply runs a forever shell script that is basically "git checkout master;git pull; sleep 300;". Then the other pod actually runs the application on top of the shared storage that the first pod writes to. This script, unsurprisingly, fails silently and hangs frequently. To debug this you need to go through some cloud auth portal and click through some web UI to then open a console in your browser (which doesn't support copy/paste).
The idiot version of doing this on-prem would be a single small VM that has a cron running every 5 minutes, which invokes your deployment script, with cron configured to email on failure. This would be simpler, more reliable (not relying on a forever script), and more supportable (it emails on fail!). And it uses like 30 year old tech that just fkn works.
I think we are real inflection point between an analog world and a digital world. I also think that the Bronze Age never went away and analog techniques will just develop into craftsmanship.
I haven't posted a link to these Autonetics parts in while so I will do so again.
What I love about Autonetics is that they were solving the inertial guidance problem in whatever computing form they could get a hold of.
I'm working on a proper website but the images linked here represent the inertial guidance problem being solved with discrete components arranged in circuits and later those circuits integrated in silicon.
https://en.wikipedia.org/wiki/Autonetics
https://www.icloud.com/sharedalbum/#B0YG4TcsmGWIVSf
Some Nixdorf 820 photos
https://www.computerhistory.org/revolution/memory-storage/8/...
https://www.icloud.com/sharedalbum/#B0Y59UlCqNHCjx
It's not just the natural order of things that such knowledge passes out of this world. It's one of many collective and affirmative decisions by people born before 1980 to take as much with them as possible when they go. I would be skeptical if it were just one thing, but after affordable college, affordable housing, affordable healthcare, affordable retirement, and a viable biosphere, it's become a pattern. That you seem to understand this on some level and don't care to try to mke a difference is even worse.
30 years is plenty of time to learn the everything from the browser down to the CPU, or the HTTP framework down, or the database down, and probably enough to learn two such slices. Part of why it seems impossible now is just because most developers not only don't have the historical context, they haven't yet had any time.
i think whats required to move forward isn't just some dusty studying of these physics, but a reawakening to an understanding that these are just systems with tradeoffs, and we can choose to make other systems just as easily.
thats the only way the sandpile collapses and we get .. for example .. secure operating systems and network protocols. or systems that were designed to exploit the massive amount of concurrency available in modern machines.
This is simply not my experience of young people or of being young. There's a reason why radical movements are led and populated largely by the young—to the extent they err, it's on the side of "let's tear it all down and start over".
You say you were lucky to be there in the beginning and that young people don't have this advantage.
Then you conflate your own luck with perseverance.
Trying to learn all this in 1/10th of the time to start your career is what requires perseverance.
> We, *people who really understand the technology* that the world runs on, are a slowly dying breed
Thus one has to conclude that young people, not having perseverance, do not really understand technology (and lacking such basic skill, never will).
For example: You and I HAD to be willing to go through that, and we were fascinated by it. There are still plenty of great FE and BE young devs who see something and instantly ask "How does this work" and keep going down. The difference is, most people in employment don't HAVE to do this like all people in our early career did. If those shortcuts to getting paid existed in our early careers, many of us would have taken the money and not cared. It's not an experience or a "You had to be there" thing, it's simple curiosity and motivation that sets people apart in this regard.
Of course, it's bad if the western world fully outsources certain components of the stack.
But not everybody needs to understand the full stack. Maybe you don't know how the silicon crystals are made and that is okay.
Some people like to say this is a perennial problem that every generation bemoans. I get why that might seem intuitive, and for the longest time I've hoped they're right and I'm merely getting old.
Sadly they are wrong. Long term historical patterns of empire collapse follow this over-reach and under-education cycle that leads to a catastrophic rebuild capability gap that is triggered at some point.
Thomas Thwaite's "The toaster project" [0] is a wonderful commentary on this. I cited it in a talk I did about education in world that will no longer pay for teaching kids how to use chips and breadboards in skinflint universities that prefer to teach them Microsoft Excel or use some crappy simulators of everything [1].
When it comes to cybersecurity we can only obtain defence in depth if we have knowledge in depth, and there are ever fewer of us around who understand how computers actually work. It's more than a little unnerving to meet CISOs and folks with high-flying titles who have no idea why using a phone app to control a Fortune 500 company infra might not be a good idea.
[0] http://www.thetoasterproject.org/
[1] https://www.youtube.com/watch?v=AxoSskalOrM
Even my doctors don't know some things about various drugs they prescribe me, because I can afford to spend the time reading the literature that only pertains to me. I'm sure their biochemistry background means they could definitely outperform me if they wanted to do a deep dive, but they know the important parts, and they know how my condition stacks up in the compendium of total knowledge and clinical experience they have. I don't think anything important is lost if they cannot at the drop of a hat tell me the half life and mechanism of action if I grill them on it.
I've been trying to write a blog post about this, and how to address the problem specifically for front end development, where it's absolutely ludicrous. It's a very serious problem, especially when you think about security.
You have all these people clamoring to get cloud certifications so they can work in IT but its so complex. A certification isn't enough. There just aren't enough people with enough knowledge and practical experience to fill all the needs that companies have.
If by "understand" you mean "have a passing familiarity with" then.. sure, but if you mean "my understanding is reflective of reality" then there's absolutely no way.
We, people who really understand the technology that the world runs on, are a slowly dying breed.
There are more people doing "low level" programming right now than were doing it a few decades ago. The only knowledge that is fading is knowledge that's applicable to legacy and/or retro systems.
However, a discussion over the misalignment of incentives in modern society can definitely be had. I would trust the pilot generally much more than a business software developer, researcher, or banker because the pilot will be the first to arrive in a crash.
Talking about abstractions, during my past month, I was reading nand2tetris, and it's a compelling experience if you understand the exercise you are doing, which is not about building a computer from first principles; it's much more than that.
It makes you really understand what's going on behind the layers of abstracting that have been raised. Sometimes, understanding every layer of complexity is impossible, but depending on the area you are in, we need at least to try to understand the roots of it.
However, this is not for everyone; ask a musician if they are really in the weeds of why the instrument is producing music (the physics behind it!). They are probably aware that it's vibrating air, but, in general, they won't know the theory behind it.
Everything is built on abstractions, and that's OK, with time we will add even more on top of what we have; now, the issue is when those abstractions lock you in with a mindset that prevents you from creating something new without relying on those same abstractions that help you build stuff.
Many discoveries and inventions were made because they knew the layer of abstractions on top of it, and they just started again from scratch. Even Figma, the software, is built on a new core of concepts based on how the web was working back in the time [1]
[1] https://madebyevan.com/figma/introducing-vector-networks/
A big difference is that, unlike computing, their instrument probably won't stop working because of some subtle change to physics introduced by a seemingly-unrelated change made to the universe by some other party in the musical instrument / air / molecules / atoms / quarks "stack". Theirs is a world with some assurance of stability.
Ours is a field built upon shifting sands. Knowing what the foundations are that the edifice you've constructed sits upon allows you to affect repairs when it crumbles unexpectedly.
Computers are a relatively new invention. There are still people around from the times when knowing the entire stack from top to bottom was not just valuable but necessary. They worked during periods when abstractions were far leakier than they are today and far less reliable but also far simpler.
Some of those people have persisted with the attitude that knowing the stack top to bottom is still necessary as the complexity of those stacks have grown in complexity geometrically. This does not signify wisdom, only age.
- a musician knowing how to fix their instrument vs a programmer knowing how to fix their keyboard/computer
Or
- a musician fixing some harmony, or progression vs a programmer fixing a bug in code
Otherwise it does not feel like a fair comparison.
- Knowing how to change strings is comparable to knowing how to use the browser console
- Knowing music theory is comparable to understanding programming language theory
- Being competent on the fretboard is comparable to being competent in one programming language. Strumming/fingerpicking could be considered another language.
---
And then for the controversial one...
- Only knowing how to use music-making software is like only knowing how to use low-code applications for development.
dodges tomatoes
Finally get to use this bit of knowledge: 'effect' (the verb) was the word you wanted there.
Not a great metaphor. First, most instruments were invented by people who had no knowledge of the theory, not even of sound involving vibrating air. Second, many musicians do understand some of how their instrument works. It's not hard to understand how a guitar works, how to tune it, and how to fix a broken string. Most can't fix a broken body or neck, but they do understand why it stopped being playable. Hell, I knew a flutist who simply started to build her own recorders. Zero knowledge of physics, but after a few years producing great sounding period instruments.
Sometimes I'm learning a framework, and I'm in the part of "...you write this easy syntax and it outputs pure HTML!", which is great. But then the next line you read "also, when you use [obscure symbol], it does [obscure thing] in order to respect [obscure concept]". You have no idea what to do other than start googling those 3 new things, none of which are explained properly or at all in the current "super simple framework" documentation. I think even naming should make sense.
Knowing the nitty-gritty of the acoustics theory and the related math may not help you much as a musician... (that being said, if you're building a home studio or doing any mixing/mastering, you'll find it helpful to learn about physics of standing waves etc).
However, the details of how frequencies come to form the well-tempered scale with all the tradeoffs and imperfections, and the low-level details of music theory, would be something many musician nerds would actually know.
Meh, zoomers didn’t invent incompetence and slow, ugly software. Not everyone is cut out to write aerospace grade software and, thankfully for those people, not everyone needs it.
Forces you to at least admit to yourself that you probably lack a lot of knowledge. Makes you humble and hopefully qurious.
Programmers too often add indirections which don't provide abstraction:
In contrast, if you have a collection type (Set, History, Permissions) and want to treat it as a monoid, that's a huge leap in abstraction, with only one level of indirection.I've found the shittest programmers to work with are the ones that think visually, because in their head they just deleted two boxes and four lines, but in reality all they did was add 40 lines of code that do nothing.
https://news.ycombinator.com/item?id=26956298
80% of code organisation problems require doing a couple of things right: directory design (directories, file names, file contents), function design (purpose, naming, parameters, context and return type). I've been coding for almost 25 years and I've encountered the balance "20%" extremely rarely. Even then, 100 lines of simple code is easier to read and modify than 20 lines wrapped in clever abstraction (e.g. I once had the dubious honour of refactoring a broken and unmaintainable state-machine driven code base back to a simple series of if/else/switch/case statements, and the improvement in the development and troubleshooting times seemed almost unfair for such a simple "trick").
Folks that have really strong opinions on issues of taste, but then they output apps that send a couple of thousand requests ( with several second latency ) are extremely difficult to reason with.
Anything undone by the compiler (perhaps even inlined) is just indirection. Everything beyond is actual abstraction (which cannot be broken down further by the compiler as it’s far too limited in its understanding).
We do? In my neighborhood, we can’t get fiber lines because of some combination of NIMBYism and political back-scratching allowing a cable company to continue to serve us shit slow and unreliable internet. In the middle of a city!
I’m more interested in solving _those_ problems than how long a request round trip takes.
- how directory structure is part of the code organization of the language, and how it's something you have no choice about (so have to spend zero brain power on), and works out of the box.
- the concept of exported/unexported fields (no need complex logic/annotations, ineffective _prefixes)
- named returns eliminating the need for explicit, so useful when exiting early
- default values (+ named returns)
Any of these *manager concepts just seem out of place in Go.
That’s what I do by default since I was a kid, and I can tell you the social pressure not to is significant. I recall an interview I did once, and one reason I failed it was "questioning everything". It didn’t even felt like it, I was just asking questions about their system, it’s supposed to be basic curiosity. At my current job there’s this architect that explicitly asked me for continuous feedback, but now shows subtle signs that maybe I went a little too far.
Questioning everything gets results, not friends.
Others might have accepted the unknown and ate breakfast.
Another example: as a child you understand what + means and take it for granted. As part of Maths undegrad you start understanding the type of operators and the axioms of maths. The child was perfectly find completing maths problems without that depth.
One thing I've been noticing with my current clients is the vast amount of churn generated by them only having a patchwork understanding of their overall architecture. Some grok the low-level infra, some the deployment levels, others the backend pieces, and others the frontend stuff. However, since there is no story collecting the pieces into a cohesive whole, we see them treadmilling through different tech stacks in an attempt to gain "velocity". They're definitely running fast; it's just they're not getting anywhere very quickly.
It's a pattern I see a lot. On the flipside, when clients have devs who mostly grok the entirety of their system, then we can focus on the questions that really matter: purpose, market fit, empirical evidence from users, _etc._
Questioning things costs time right now, not questioning costs 10 times that in the future.
So uh, not questioning everything eh...
Sounds like you agree with me. That questions should be reserved for the meaningful and insightful.
i don’t thjnk the logical extreme you position here is correct; if it’s an interest for the person why shouldn’t they pursue it? questioning everything for me is just getting to the point that it’s not magic anymore.
i probably don’t need to know the entire kernel code base to figure out why my eth0 won’t stay up. it might help a bit to know some of it if i’m getting weird kernel-ey messages in the logs.
Every day the sky will be slightly different. There exists a reason why, and given effort you could identify the differences and relations.
Or you could go to work.
> It's a pattern I see a lot. On the flipside, when clients have devs who mostly grok the entirety of their system, then we can focus on the questions that really matter: purpose, market fit, empirical evidence from users, _etc._
So... stopping. Not questioning everything eh.
So you end up in this weird place where it feels like there is a big pull to make tech about everything but the mechanics of what you actually do on a day to day basis. Or rather, it’s viewed in a very transactional way: “I learn this skill to get that job.”
I'd argue that it depends how it's done. I used to work with a brilliant Data Scientist who had a PhD in AI. She asked so many questions — often about fundamentals — that it was like working with the Riddler. She generated a ton of insight as a result and identified missteps before they happened. Despite questioning everything she was arguably the most liked person at the company because of how she approached it. It didn't feel like being grilled, but rather that she was super interested in people and what they were working on.
My point is that questioning everything can get results and friends if done correctly.
To make this gender neutral: When questioning, don't be aggressive, and try to obfuscate that you are a potential rival.
There is a lot of asinine software in the world and that’s fine. Lots of “real problems” are pretty asinine and don’t require heroic engineering feats.
Just slapping some bullshit together is good enough in a frightening number of cases.
Hate it too myself, but I have come to accept I can either prove them wrong by building my own business and outcompeting them through my supposed superior engineering or just swallow that I’m yelling at the clouds.
A sizeable percentage of which is made without source control, unittests, and just slapped together like the rest you described
Hospitals don't check software quality when making purchasing decisions (much beyond "this button doesn't work")
This! The worst piece of low quality garbage I've seen from the inside is hospital software. Nobody really cares or understands software on the hospital side, so anything goes. The product is inherently complex, so there are only a small number of large companies who don't care either since they can rely on customer lock-in, no matter how crappy their service is.
"Fix IT: See and solve the problems of digital healthcare"
https://www.waterstones.com/book/fix-it/harold-thimbleby/978...
I guess something like certified engineering may alleviate some of these issues.
If we build bridges like this.. oh.
Honestly, I liked it. The font defaulted to Consolas on my Windows machine and it was both readable and not too distracting, oddly enjoyable.
People increasingly just know about using third-party technology instead of building their own.
One approach to fix this is to ban usage of third-party technology entirely.
Rust uses algebraic datatypes with ML-style type condtructors and Hindley-Milner type inference to achieve, for example, types like Option<Box<T>> or Option<&T> or Option<&mut T>, so that you're forced to check for the case of a null pointer (None), and if it's not in an Option, then it's guaranteed not to be null. This was originally conceived of mathematically in the 1980s and then implemented in ML.
The Hindley Milner type system was devised in 1969 from the typed lambda calculus.
Golang was influenced a lot by Communicating Sequential Processes, a formal system for describing concurrent behavior in systems, first described in 1978 by Tony Hoare.
Graph theory is a field of math that comes up a lot in computer science, in marketing, networking, compilers (register allocation and functional programming language interpretation).
The only innovation I can think of that came purely from tinkering is Rust's borrow checking system, and they're still trying to formalize that while improving it.
Derivations from first principle are always important.
https://sites.google.com/site/steveyegge2/practical-magic