> There is another fundamental that is ignored by a lot of computer users. I think everyone who spends a lot of time
behind a computer should learn how to touch type, since typing is still the main way of interacting with your computer.
I sort of agree, but anyone typing out code at 120WPM isn't programming, they're writing. Programming is an intellectual activity, not a physical one so learning to touch type is worthwhile but might only gain you a few minutes a day of productivity over someone doing 40WPM in a home-grown hunt-and-peck style of typing?
I don't agree. Touch typing is not about the speed. It is about letting you keep your eyes on the screen while you're typing. It enhances your "headspace" so much, that you don't have to micro-context-switch all the time between the keyboard and the screen.
I've seen this argument before and I don't think I agree. Surely there's a lot of times when I program that are heavy into thinking, but there's also a lot of mechanical work. Refactoring? Typing speed will help you. Going to try out a new approach? Typing speed will help you to see whether it's true or not.
Everything else being equal, typing speed will help you try out stuff faster.
Yeah... even if you don't have to type fast, not needing to look at the keyboard when you type will at least help you to concentrate on what's on your screen and stay in the "flow".
All else being equal sure typing faster is only going to make you more productive but a slower typer who is better at solving complex problems is always going to be more valuable than a fast typer who is not as good. I don’t think I’ve ever encountered a programmer that was too slow to simply be productive and valuable to a business which makes it questionable advice for anyone to follow.
I'm a developer who is also a pretty fast typist, and I have to say that's not my experience. I wouldn't want to hunt and peck, but my sense is that higher than, like, 40 GWAM probably yields diminishing returns.
Language might make a difference there. Writing C# with a QWERTY keyboard, you still have to interrupt a normal typing flow for your parentheses, braces, etc.
I suppose it's helpful if you're writing long comments, but for the most part, I rarely hit sustained bursts of my peak typing speed when writing code.
Also typing fast does not mean writing fast. Think about the speed tests you took. You probably kind of zoned out and just bashed it out. But when actually writing/coding you stop and think about it a bit. I used to hit 70-80. But now I maybe manage around 20-40, probably less. Because I have to think at the same time and can not just 'zone out' and copy. Knowing the keys is a massive help though. With coding it is more 'i know an interface that does something'. Ok start using this object oh yeah wait it is a pointer not a local so different call syntax, oh and then the call I need is inside that object but that is local to that object, oh wait lets go read the docs this thing has 4 params and that 3rd one is always a bit tricky... No way you are hitting 40+wpm unless you just happen to use only calls you use all the time and have memorized all of the params that go into them and have the code basically already written in your head.
I'd go a step further. In my experience writing C# I don't really need to type. Like I'll just sit on CTRL + Space and the arrow keys most of the time.
To be controversial, a language that rewards typing speed seems to me like a failure of tooling, why am I manipulating text like a caveman (ok.. ok.. Mesopotamian) rather than with the symbols that the language actually deals with.
Way too many times I have seen fast typers who would spend as much time adding text as immediately deleting it after that and typing something else in its place, over and over. (Maybe that’s the way they think, I don’t know.)
I don’t know, when the thought about how to structure something is in my head, I want to type it out as fast as possible. My brain isn’t ideally suited to keeping complex constructs at the top of my thoughts for a long time.
You touch type not to break a world record. You touch type because you can write without thinking about the keyboard, just like you don't think when using or arms or legs, and it's faster to correct an error that you made.
Imagine that you want to correct a block of code or a whole paragraph. If you don't touch type, maybe you will think twice before rewriting it.
I don't mind being slow early on when I'm doing something new, because I know that I will get muscle memory for things I repeat a lot and become faster. I also use an editor that supports mapping keys (currently vim).
Besides, only Java can make any programming task I/O bound ;p
It's not about clock time, it's about reducing the latency (and friction) between thinking about code and having it written.
I might think for a couple minutes and then come up with ten lines of code, and once they're in my head I want them in my editor now before I lose track of them.
What comes naturally when using a computer daily is usually bad touch typing, i.e. not having to look at your fingers, but typing with only four or maybe six of them using weird, possibly unhealthy, techniques.
i developed a pretty fast way of typing naturally, using 7 or 8 fingers.
unfortunately it caused wrist strain.
i could not use split ergonomic keyboards because some keys where "mapped" to the wrong hand for that.
i also could not simply learn "proper" touch typing on a qwert layout because my typing habit was too strong.
in the end i learned touch typing on dvorak to fix my wrist issue. maybe after 10 years i'll have unlearned qwert typing enough to go back? (not sure i would, i do prefer dvorak, it just feels silly to type at snail speed on others ppls keyboards).
I can't stand those supposedly ergonomic keyboards that have been chopped in half, and I didn't understand why anyone would want one until I recalled people learned touch typing.
My naturally acquired typing style involves me holding my hands hands in just about the same angle as those "ergonomic" keyboards, but just using a regular keyboard. Fingers rest on LShift, A, W, D, and K, O, P, Ø with both thumbs on Space.
Should be noted that's on a "105-key" Nordic keyboard with US layout when not writing Norwegian, so the Ø becomes '.
For me this has worked quite fine for over 20 years. I haven't bothered measuring my typing speed because it's never the limiting factor.
Depends what you mean by "touch typing". I was typing pretty fast at a young age (young age compared to know) but got my arse handed to me in a speed contest, for fun, at home, by a friend who was a secretary and did learn proper touch typing technique. She was like: "Oh you think you type fast uh?".
So I simply asked her to teach me the proper technique and now, since decades already, I'm touch-typing the proper way.
Having said that: you can be very fast using improper touch-typing technique.
However, it might be a good shortcut for people learning. I was taught at a youngish age to attempt to touch type (box over the keyboard and a tutor program) I fucking hated it, and I was slow, it was dull and painful.
However, Then I had a need for typing, so I got faster and faster, and the initial structure that I got from the rigid teaching, allowed me to learn faster than my peers.
You don't, but boy do I love not focusing on how to type something and just having my thoughts flowing into the screen without me thinking how to do that.
Yes, someone really do needs to. I used computers daily for 10 years without learning to touch-type, reaching about 24wpm. Then I took a typing course, and in 9 weeks of 5 hours a week I got up to 40wpm, faster than the fastest hunt-and-peck typists. A couple of years later I was at 50wpm, and then I started using IRC and was up to 90 within a few months.
People can practice a skill for a long time, but if they don't focus on improving, they tend to just harden their existing habits.
Any activity on a computer is aided by reducing the friction involved in getting what is in your brain into the computer and from the computer back into your brain.
Sure, you can program with punch cards and no display. People did it for years. But having three monitors and a quality keyboard that you’ve mastered will be much more efficient.
I started with hunt-peck style typing and can now type without looking at the keyboard because of muscle memory. As you mentioned, typing has never seemed like a productivity impediment to me since I don't think I ever had to type at 120WPM when programming.
Programming is a uniquely bad fit for formal touch typing because we type a lot of symbols that don't fit on the home rows. I'm typing this comment without looking at the keyboard, but I'm not "touch typing" because my hands aren't kept on the home row.
Need to use the arrow keys? Outside the touch typing range. Mouse? Outside touch typing. Function keys? Numpad? Touchbar (blech)?
I have never felt limited by typing speed. Informal touch typing is good enough. Quite often it's the computer or network that can't keep up, anyway.
40WPM is fine, and far beyond actual hunt and peck.
But I have encountered one IT professional (not a programmer) who despite being in the industry for 20+ years, could only manage about 15WPM, if that. His style seemed to include legitimate hunting for the next key, and only two fingers.
At that speed it was starting to become problematic: It was painfully slow to watch him do anything or help him with a problem.
Not an absolute deal breaker, as long as such a person is good at their job and is efficient with what they type (taking advantage of copy/pasting or scripting), it's workable.
If I can touch-type at 120WPM, then I can churn out a great deal more documentation per unit time than another engineer that can only type at 40WPM. It's less burdensome for me to document well, to write expositional documents and to communicate regularly than my colleague that needs to budget their typing more carefully.
While I might not be bottlenecked when working on a tricky piece of code at 40WPM, I'll likely be bottlenecked elsewhere often.
Typing fast is useful but not necessarily enhancing productivity all that much. Even writing documentation is more about thinking than typing fast. Documentation must be concise and understandable.
There's also an argument here about having (and signalling) respect for one's peers, craft and tools.
The keyboard is a tool.
To me, it feels disrespectful for a person to expect these high wages when they don't even make the moderate effort involved in learning to use this tool efficiently.
In my experience, this usually signals that a person isn't actually interested in continuously improving, feels overly entitled and they tend to be a nightmare to work with.
Most places I've worked require substantial pair programming or at least working through problems together on a regular basis.
If you can't type, you're going to waste a lot of people's time and they may be too polite to say anything, but it's frustrating for everyone around you.
A carpenter who claims to be experienced, but can't work a hammer efficiently would be ridiculed on a job site.
I'm not proposing we ridicule anyone, but being unwilling to type and claiming to be competent at a job that requires an enormous amount of typing does have a ridiculous look (to me).
Of course, I have worked with people working through a variety of disabilities, including dyslexia that make touch typing basically impossible. I'm not talking about them here.
I'm talking about people who just refuse to make the effort for whatever reason. It's a bad look.
PSA for newer players: Regardless of how you may feel about this subject, failing to demonstrate at least moderate typing skill in an interview is often a hard pass unless there's some explanation requiring an ADA accomodation.
Typing should be a very small amount of time of a developer's job even when pair programming.
>A carpenter who claims to be experienced, but can't work a hammer efficiently would be ridiculed on a job site.
It's just not remotely the same. How quickly and accurately a carpenter can hammer has a large impact on the quality and speed of the work, while typing should never be the bottleneck for a developer
> If you can't type, you're going to waste a lot of people's time and they may be too polite to say anything, but it's frustrating for everyone around you.
I don't think in my career I have ever been alerted to anyone's typing speed being slow.
Yes, I've heard endless noise about "the perfect keyboard" and the ins and outs of spring types and junk. But I've never noticed the output of someone because they type slowly.
And to be honest, I really couldn't care. Can they code? can they keep to a deadline? are they nice to work with?
if the answer to all three is yes, then I couldn't give a shit about how they type.
> but can't work a hammer efficiently would be ridiculed on a job site.
Only if they need to use a hammer effectively though. If they are laying veneer, carving, using a screwdriver like a sensible person, then a hammer is only rarely needed.
> But I've never noticed the output of someone because they type slowly.
I do, when I'm pairing with someone, or in any synchronous meeting environment. It may not effect the 'output' of delivered code (for a developer) but it can hinder their ability to pair effectively.
I agree that some (maybe lots of?) people feel differently about this and of course, that's okay too.
To be clear, I'm not saying anyone needs to type 120 WPM, I'm talking about a single finger on each hand, eyes down, hunting for the next key to press.
Over the 20+ years of my career, I've been exposed to hundreds, maybe thousands of different folks doing different kinds of computer work.
And yes, I agree that there are likely people who can kind of get the job done without being able to operate a keyboard efficiently. I also agree, that it helps a lot if they're nice to work with.
But I have not yet met a single hacker who has demonstrable and deep knowledge of computers and the software that runs them who cannot type.
I'm sure they must exist, and it wouldn't surprise me if Donald Knuth himself dropped into this thread to let me know that he hunts and pecks!
I'm just saying, that I have not personally met or observed someone operating at the highest levels of this field who cannot type with a reasonable speed.
[update]
Almost forgot about Gatekeeping, Yes!
Gatekeeping is an essential part of interviewing. In fact, that's the entire point. We interview people for roles with the specific intent of bringing on people who we believe (or at least hope) have higher competence and closing the gate to people who have lower competence.
It's incredibly difficult work and it's not perfect and we have a lot of work to do, to make it better and more inclusive, but expecting competence in basic, role-relevant skills is unambiguously reasonable IMO.
Yep. You can’t find the best programmer in the room by measuring everyone’s typing speed. But the person with the slowest typing speed in the room will rarely be the best programmer. Unless they have RSI or something, typing slowly is just a simple, low effort, obvious signal someone hasn’t spent many hours in front of a computer yet. Programming skill takes time.
Would you say I am being disrespectful because when I learned to type at the age of 12 I wasn't following any official guide and learned to touch type with only ~6 fingers? I type at around 120 WPM on average. Is it just the speed, or do I have to conform to some specific definition of "touch typing?"
Of course the particular technique is irrelevant (to me at least).
I most enjoy working with people who care deeply about their work, and in my experience, those people are often (not always), people who also care about being efficient. In fact, if you're writing code at 120 wpm, maybe you've found a technique worth sharing and teaching! :-)
But how do you know if someone has dyslexia or is just not making an effort? Or has wrist, shoulder, whatever problems? Unless the typing is hurting their output, who cares? And at that point it is not the typing speed that is the problem, but that their output is low.
To be clear, I'm not insinuating that anyone needs to type 120WPM. I'm talking about single-finger, head down, hunting for the location of a key.
People with mild to moderate dyslexia (in my experience) and people with injuries are generally able to demonstrate competence with the keyboard and computer operation.
I have dealt with moderate to severe RSI, and I have worked with a relatively large number of people dealing with RSI over the years and in my experience injuries don't mask operational competence. These behaviors look different from, "where was that key again?"
Dyslexia often surfaces in a variety of early communications and over the years, working with some especially brilliant people challenged with dyslexia, I've learned to assume (early on) that typos are not a signal for carelessness, but potential indicators of dyslexia.
I'm aware dyslexia is a spectrum, and at the extreme it can be fully debilitating for some activities. I haven't had much exposure to that cohort, but I would be happy to make reasonable accommodations and certainly would not consider it disqualifying.
Thanks for bringing this up, I think it's helpful to be able to clarify these details!
One of the best and extremely productive programmers I have known had cerebral palsy. He typed slowly, with one finger, while holding the right hand steady with the left.
What about writing that in ancillary to the actual entry of code into the computer?
In the modern workplace there is often quite a bit of communication via a text chat system. A chat where everyone types at 120 WPM is about as fast as the low end of normal conversation speed (120-150 WPM). A chat with several people only doing 40 WPM is much less pleasant.
Email doesn't have the quick back and forth of chat, so slow typing usually won't be directly noticeable to your colleagues, but it can take up a lot of your time.
Certainly you can be a highly productive programmer if you only hunt and peck, but it's a real handicap; you have to do a lot more of your programming in your head and a lot less in the computer. (Though 40wpm is unheard of, you might get 30.)
I think the best analogue is blindness: it dramatically reduces the bandwidth of the information channel from the computer to you, but there are highly productive blind programmers. Hunt-and-peck programmers are in a similar situation.
> For the majority of programmers out there, something being RESTful means that you can feed it chunks of JSON over HTTP. The minority, who have spent some time reading about it, will complain that it’s a design principle defined in the Roy Fielding’s thesis and it has nothing to do with JSON or HTTP 3. How many of you have actually read his thesis? All of it? No skipping?
I've been railing about this for years. Please stop calling it a REST endpoint. In 95% of the cases it is a custom HTTP endpoint and that's it.
Probably pick a different hill to die on. The meaning of "REST" in common parlance is now just HTTP+JSON, nothing more. At best, the url might represent a noun.
I'm not going to die on that hill, but I will gleefully call an API "RPC-over-HTTP" as a kind of silly trolling when this style of JSON-over-HTTP RESTful is equated with cool.
I actually wonder where this came from. It seems like a few popular Spring Boot projects started putting "rest" somewhere in the url path of every http endpoint that returns a serialized data structure in the response body rather than browser-renderable markup, but were they the first to do this?
I try to just call these things an "http API" when talking about it myself, just to mean an endpoint that is intended for consumption by other applications, not by humans, but I'm not gonna fight other people using language the way it works in practice, where words just mean whatever enough people agree they mean, regardless of what the very first user of the word meant.
It’s interesting, many western corporations used to do that, but after decades it eventually was delayed to training during summer breaks or jobs right after university as it is often assumed that even after university they lack any real world knowledge. Cheaper and often a loyal employee, still works but not as mainstream as it used to be.
Dojo’s approach seem to be one step beyond, but it requires even longer commitment to the employee and an asymmetrical market opportunity and economic chance. Definitely will help people and become a win win.
I've worked at visual effects studios working on feature films, and in that industry (where proprietary workflows rule) there is always an education department. Even if one used to work there, being gone for a year is enough to make one useless as the workflows change radically, VFX technologies move fast. When I spent time consulting at web shops, I was dismayed at the undocumented and loose workflows, plus no process documentation in sight. I get the sense the web is the worse of modern tech exploitation work practices. Hell, working in video games looks better.
Undocumented is much worse than your comment seems to realize. Wait until you are in a situation when the only people that know a mission critical procedure quit or die. This can cause entire large companies to crumble, as well as the reason so much confusion exists in development, and is also a reason the industry relies on leet code testing - if you don't know something before being hired, nobody can explain it there because they are all fixing fires or on not-really-understanding auto-pilot.
It's actually a good practice. I think the intelligence community sort of do this too. On the other hand I think a more practical equivalent is to hire internal candidates who show some interests but didn't get the chance to work on real problems and train them.
I've worked with several companies that tried that, in the end once the juniors became experienced and skilled enough, they took offers from other companies with a much higher salary.
The issue is that training/skilling juniors or fresh university graduates costs you money and slows you down (as you need to assign senior engineers/mentors to assist them). It is not unlikely that a newly joined junior employees only costs you money and decreases throughput/value for your company in the first 6-12 months, but you still pay them a (decent) salary. So once you got them skilled, you as a company would probably need to raise the salary to market/senior level to keep them - given the mass amount of recruiter approaches we in IT get. In that case, you could just start off hiring seniors only in the first place.
This is an issue I see everywhere, nobody wants to hire freshly graduated and everybody is only shopping around for seniors.
>took offers from other companies with a much higher salary.
Seems like paying at least market rates once they are trained up would solve that problem. Once someone develops a skill, a company can't expect to keep them at intern or junior salary levels.
That was exactly my point. But why hire & train a junior dev when there is no company loyalty and the money spent on training is basically lost. Instead management focus on directly hiring seniors. Of course that doesn't work as a whole, because if no one hires juniors there will be a point where there will be no seniors anymore. I also feel little to no pity when those companies complain, that they can't find seniors.
> Don’t ask questions that can be answered with a web search.
These days, that’s equivalent to saying “don’t ask questions about what the day to day job requires one to know”.
Sites like stackoverflow contain answers to problems that people encounter in their day to day jobs, and if you eliminate those as potential questions, then what should you ask? And what is the value in asking some obscure question, other than making the interviewer feel smart and special?
Here's a question you can't properly answer with a web search for a data engineering role
"I'm emitting 10TB of data a day with this schema {X} - it's produced in Avro. Currently we upload it into S3. But, instead, I want to stream into Apache Druid, but also want to make it available for ad-hoc analysis by BAs.
Your question turns into a question of "have you worked with Avro, S3, Apache Druid..." which will filter out competent people that don't have experience with the specific list of tools.
The keywords there were "streaming" and "also ad-hoc analysis".
Answering those questions requires no knowledge of Avro or Druid, but rather, how to build streaming pipelines, and also make the data available for a query engine.
But that said, if you're in data engineering and are unaware of what Avro is, you're probably not at the level of knowledge I'd expect, and I'm happy to filter you out.
Likewise, if you're unfamiliar with S3, why are you even applying for a data engineering role?
But how can I be familiar with S3 without any data engineering work? Or maybe I should put it like this, what exactly does "familiar with S3" mean? Does it mean that I used it to store something (I mean even a non-programmer can do that), or that I should be able to manipulate (i/o) the store using Python with Boto (Again this can be done with a tutorial), or something else?
I believe that "something else" usually indicates some previous data engineering work though, so it's an egg and chicken problem.
I think it's a bit too specific. But for sure someone can relate to similar experience. Something like "For sure I haven't dealt with Druid specifically but in my job I tackled with a similar problem for a different set of tools...".
BTW from my experience we are streaming a lot of data (similar to 10TB but definitely more) produced in JSON then converted to Avro using Spark and finally loaded into Vertica and the analysis pipeline. I have never used Druid but from Googling it is a column based data store which I guess is not too different from Vertica. Is there any specific difficulty you think would be encountered with Druid? Because the wording seems to indicate that Druid is not particularly good with ad-hoc analysis.
> Because the wording seems to indicate that Druid is not particularly good with ad-hoc analysis
Bingo ;)
I was trying to phrase it like I've heard people do so in interviews.
And tbh, a candidate saying "I'm not familiar with Druid, but it sounds like you can't use it do ad-hoc queries with it? Can you tell me more?" would be a very good sign.
The best ones are the one that take the CV and for each project listed they ask you to say how it was and then some tehnical challenges you encountered. Depending on how you lay out the problems, a competent interviewer will know at what level you are.
This is really the least that I can ask for new hires. I don't mind if they want me to repeat what I have taught them because I have been there and know that been taught != fully understand and it takes a few practices to imprint that in one's brain.
But for God's sake please Google before asking any general question. Some questions are literally within the first 5 results of the first page.
Maybe I shouldn't blame them though, sometimes I did that too :/
> “Yes, every programmer should know how their hardware works as one cannot write an efficient software without that knowledge 4, and I wouldn’t be able to do my job if I didn’t know anything about computer internals, yak, yak, yak…”
I agree that you don't need to know anything about von Neumann Architecture and the likes in order to be a Web Dev.
But I wouldn't want to hire someone who doesn't have basic knowledge about data structures and runtime complexity. It's just too easy to have something work fine in development, but choke in production due to accidentally quadratic behaviour.
Usually I had funny but opposite experience when doing consulting for performance optimizations.
At some points either teams converged to people without basic knowledge of data structures or they rushed with something that works (when N in O(N) was small) then scaled up. They get used to lower performance slowly like boiled frog.
Then someone is tasked with optimization, but instead of attacking data structure at hand, usually those people tried to optimize pieces of the puzzle for smaller gain, then they ended up with small hacks to shave some execution time, with unmaintainable code which lead to serious bugs.
This is the kind of observation that should be central to CS, but isn't.
While academic algorithm astronauts write papers which include Greek math symbols and can be proven correct, most people do... something else.
Optimising the something else would be a huge benefit to the industry. But no one even looks at it. It's just a thing that happens, and there's no research into fixing it or making it better.
I feel like this is moving in the opposite direction! I think the issue is that most of the time the "something else" people write with poor algorithms is actually an application of the correct, efficient algorithms from formal papers written by academics, but they don't know it and thus don't use it.
Not knowing how databases work is a pretty common issue I come across with colleagues. Even basic things like normal forms (which take about 5 minutes to understand), and what transactions are. I think it’s why non-RDBMS datastores are so popular, because they generally just ignore all the problems RDBMS are trying to solve, so the developer isn’t inconvenienced by thinking about them.
Non rdbms are popular because at scale everything that's nice about "classic" rdbms are what makes them ultra slow.
edit: the features I refer to are stuff like foreign keys, cascades, dynamic fields, complex joins, etc. Once your stuff is spread out you loose some of these by definition and if you're still sticking to a single DB with massive stores you're not getting much humpf out of it anymore.
I 100% agree there are way too many fools that think their DB is big and try non rdbms too early and waste too much time. But I still think their lack of recent popularity is due to their own limitations.
Not really, they remain fast enough, there’s just no easy way to scale them past a certain point. Or there wasn’t anyway, I believe with spanner and CockroachDB we’re getting pretty close.
But 99% of projects do not come even close to the scale where you need that.
There are very few RDBMS usecases where scale is a legitimate performance concern. Index scans scale to a very, very high cardinality.
There are use cases that aren’t best suited to an RDBMS at any scale, like managing a collection of denormalized documents, or a graph. But in my experience, considering these factors isn’t what’s motivating adoption of Mongo, or Dynamo. Your key lookups will always be fast (until you get a hot shard), but when you cram a highly relational set into a document db, eventually you’re forced to realize that there’s a lot of mutations and access patterns that you simply can’t implement, because you chose the wrong tool for the job.
Either that or you’ll decide that it’s a good idea to implement your own concurrency control mechanism on top of your document database. Which I’ve personally seen at least 3 people try and do.
Completely agree and have been burned this way as well. If you need engineers but cant even find competent ones (something ive seen before), then the only choice is to put them on prototype work/non-critical projects
That's true to some extent, but I am quite sure even the worldest-of-class engineers have tripped up on a question or two and went on to get a rejection. If such people truly are talented then they'll have no trouble getting hired elsewhere and the companies in question would be blissfully unaware what they missed out on.
I am not sure the failure of any company has ever hinged on the hiring decisions of a few developers. If you take a look into the downfall of Sears you'll see long-term strategic missteps that aren't related to recruitment in their software engineering teams.
Also I don't think a company deserves to fail if they've got a bit too much faith in a slightly flawed recruitment process. The entire point of us sharing posts like this on here and discussing them is to learn and even help each other, isn't it?
Just say everything is O(2^N) and you’ll be technically correct most of the time. If the interviewer says that it’s wrong, they don’t know their fundamentals.
Yeah this is right. Big O can be thought of as meaning “eventually always smaller than a multiple of” so f(n) = O(g(n)) as n -> inf means that there is some N (the “eventually” part) and M (the “multiple of” part) such that, for every n > N, |f(n)| < M g(n).
Yeah I agree that there is no need to go overly academic in interviews. I wouldn't ask about the different O-Notations and what they mean.
But if I give a code sample that has three nested for loops over some array and doing string comparisons, the candidate should at least have a vague idea that this might be problematic and could better be solved using sets or maps.
Yeah I agree that it could be better to explore this sort of stuff by looking at actual code. To be honest though you could come at it from more of an academic angle if the candidate has a more theoretical background. It's just about how the question is presented, being flexible with the response and ready to adapt if things don't go as planned. Maybe I'm kind of derailing the topic here to grind my own axe around some people's attitude towards recruitment :D
If you’re writing database applications in a commercial environment then knowing what ACID means is part of your job.
If you don’t know that, for example, debiting one account and crediting another needs to be done inside a transaction - or the converse, what you need to do if you don’t have access to transactions - then you shouldn’t be anywhere near a database.
Fundamentals are actually important. You don’t have to know everything, but you should know the fundamentals of the field you work in.
I dislike asking what acronyms mean, as even I can't always remember the letters, although I know about the necessity of transactions and how to implement them.
Such as "I" - what is it again? "Isolation"? "Transactions are independent". Well--yes.
The important part you need to know in that case isn't the definition of ACID, but the actual transactional properties of the database you're working with. And even among developers working with databases, it varies how well you actually need to understand transactions.
And I would guess that many developers working with relational databases don't actually understand the more subtle differences between transaction isolation levels. I think asking backend developers to know the basics of transactions is reasonable. But e.g. knowing the exact difference between READ COMMITTED, REPEATABLE READ and SERIALIZABLE (and what your specific database actually implements there) might not be necessary in many cases. Though it certainly is good to be able to spot when you're about to enter territory where these differences might matter.
I agree that a backend developer working with databases should know what transaction are. But I don't really need the definition of ACID, and you can really push this question into details that are not relevant for all developers.
I agree to a point. But ACID is part of the nomenclature for relational databases. I feel that if I ask a developer what ACID means, it doesn’t matter to me if they know the specific meaning of each letter in the acronym, but it does matter that they’ve heard of the term and can explain the implications of it.
The point of jargon is to allow concise communication between experts. Knowing the jargon is part of the job.
I don't see the connection between your reply and the linked article's content. A major point is already made right at the start when an ACID definition is quoted from a book - which makes it clear that that term is too nebulous and, in conclusion, that simply regurgitating some definition is not helpful in practice.
On the other hand, if you are actually able to design for all those complex points you may still not have one short definition to quote available, which would not speak against you at all. Not having a short definition for ACID to quote may happen to both real experts as well as people who really don't know anything. Because for real experts that definition is too simplified to be of any use.
I saw similar issues a lot in school and in academia, when having to answer some test questions: When you actually know a lot more you may actually get many answers wrong, because the one asking used a much simplified model (and therefore more inaccurate than yours) as basis for creating their question. For example in biology, there are some key tenets, but when you go deeper you learn about all the additional quirks and exceptions. Using the simplified tenet an answer to a question "can this or that happen" would be never, but for someone who knows a lot more that answer would be wrong. As an expert, you no longer use that simplified model you learned early in school.
That was the point of the article, as shown by this example quote:
> How many similarly vague “fundamentals” we have floating around us? Plenty: RESTful, MVC, SOLID are the first things that come to my mind. If you start a discussion about the meaning behind those terms, you will find yourself in the middle of a food fight where everyone will have a slightly different opinion and no consensus will ever be reached.
All I have top do is read HN whenever such a topic comes up. It usually ends up with a lot of back and forth of people who are all experts but have a slightly different angle, and the initially clear definition falls apart when you read the discussions.
I think the takeaway from. The article is that if this I'd the case then interviews should drop all the meaningless jargon and just ask a candidate to sketch out a solution to a problem where they would have to think about atomization of DB operations.
"Yet, despite coming up with a distributed archiving system that works in production". The thing is, did he? Really? I know plenty of people who get things into production and by working they mean, doing something that appears like what it's supposed to do. I usually find these are the people who blame everything else for why it's not working. A red flag is usually the phrase, "X isn't ready for prime time". The translation for that is, "I found something that I couldn't figure out and probably will never figure out so you better let me choose something else". These are the people who habitually bounce around from framework to framework. Oddly the more cutting edge it is the better. That way they're on the cutting edge so you need to expect problems, better than everyone else, and because they're better than everyone else you need to listen to them when they tell you, "It's not ready for prime time."
Believing everything you read on the internet is true as long as you don't have contrary evidence isn't a good idea. I'm not saying he didn't, I'm saying you don't have any evidence, other than his own assertion, that he did. Considering the article is railing against learning "the fundamentals" his assertion is very self serving and should be taken with a bit of skepticism.
Most of your time isn't spent in fundamentals though. I find a lot of my time is spent looking up some kind of best practice, or discovering what libs other people are using for a problem.
Also, things may be fundamental, but not everything is built up from fundamentals. Maybe math is the best place to look at this. When you go to school, you just do it. Number lines, equations, calculus, matrices. Even through the engineering course I did, which had all sorts of advanced topics, nobody mentioned ZFC or Peano or Cantor or anything like that. And while it's true that if you dig under the stuff that you get taught there's a whole interesting world, it doesn't seem to stop anyone from using math.
With coding, a lot of it is having the right attitude. Something's broken, I'm going to try to understand why and what other people have done to fix similar issues. There's a resourcefulness that beats classroom teaching in terms of usefulness. You want that person who is capable of understanding everything that might happen, what he needs to do to fix it, etc. That person is never going to know or have heard of everything to start with, but they can deal with things when they happen.
But of course it's easier to test whether someone has heard of this or that thing. Perhaps this is why senior devs find it easier to get a job than junior ones, employers are more likely to think that they can work through issues.
Very true. Also at times when you're looking stuff up you get told what the fundamentals are, pretty handy. Eg you are looking at data structures and the docs keep telling you what the Big-O is. So eventually you decide to look it up.
In a google interview I explained CAP theorem using the speed of light. The interviewer didn't understand. He clearly wanted me to regurgitate the official definition.
The speed of light puts a hard constraint in the simultaneity of data availability in a network. You can't have data replicated faster than c no matter how fast is your hardware.
My exact response after reading the first quote. It is indeed fundamental, and something I am mindful of with every database query I execute. But I haven’t thought about the exact definition and its mnemonic in over a decade.
If I was hiring your response would be sufficient. Assuming you could expound a bit on the point of it and offer some example scenarios. So many interview objections are about extreme interpretations of these questions, which do happen but are not normal.
Yeah, I assume the quoted interviewer attempted to further clarify the question and its intent. “Explain ACID” might be misheard or misunderstood by the subject. “Well, I don’t know, I guess I get reflux any time I’m doing an interview.“
I don't think anyone reasonable actually suggest that one has to memorize the exact book definitions, but simply to be demonstrate that they understand the important parts.
Not being able to give any explanation or sign that you've at least heard of it is a completely different story from not being able to recall what the letter C stands for (consistency IIRC?).
As the saying goes: If you think you understand something, and you can't explain it to someone else, that you probably don't really understand it.
Yes, you can write some storage solution or yet another database without knowing ACID terms but you will still end up somewhere in the CAP triangle. Using common terms like these makes it easier to communicate with other engineers and makes it more obvious that a lot of things have already been solved and perhaps there’s a library that you can use.
In the same spirit you should not write your own cryptographic algorithm and for example you could develop an API that is not RESTful but it’s more interoperable when you are using standards and prior knowledge.
I personally would not hire a guy to work databases if he insists on not having to learn about ACID.
I'm somewhat ashamed to say that, when I interview, a lot of people who claim "distributed systems experience" get some variant of a line of questions that ends in the user experience of systems that do not aim for partition-tolerance.
You just assume network partition is never happening. If calls are synchronous, assume they always reach other side and finish. If async, also assume successful outcome immediately after initiating the call.
Put quite many people say "you cannot have all three of Conistency, Available, Partition Tolerance", which sounds like you can keep any two / drop one at will.
But the right starting point is: "Partitions will happen. When they do, do you drop A or C?"
> It turns out my day to day work doesn’t require deep knowledge about database internals and I can mostly treat them as a black box with an API 2.
Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?
I myself had a similar fuckup with MongoDB a decade ago, because I used it "because it's fast", but failed to read the small letters, and wasn't aware that it achieves that speed by buffering the data for async save, and so has no guarantees it will actually be saved at all. So I lost a lot of data when traffic suddenly jumped, and since those were affiliate clicks lots of people got very pissed about not getting paid, and it almost ruined both my client's business as well as my own, as it was our core client.
Lack of understanding of (or even caring to know about) DB internals is also why like 90% of projects I've seen has wrong indices, or often no even a single index set at all. Because, hey, it's someone else's job to think about it, but in reality budgets are limited and team don't have a dedicated DBA or even a capable devops person to notice it, and it just ends up in the production like that.
So, in summary: no you don't need to know every in-and-out of every technology out there, no one can do that, but there's a valid requirement that one has to be familiar to those that they actually use - at least those that can seriously bite one's ass and cost a lot of money. And all DB related stuff is very much like that, that is probably the most critical part of the whole system, so no, it's definitely not a black box.
>Because of such attitude of the previous dev team my client ended up with DB integrity ruined
Same thing here, our legacy codebase avoided using transactions, and to this day, after migrating to a new codebase, we still get reports from clients here and there about errors which stem from inconsistent data
Damn, and I've always felt like I must have some kind of OCD whenever I'm getting a refresher on MVCC/MGA isolation levels and their guarantees and the associated auxiliary locking (say, select for update with lock in Firebird, for example) just to be sure I'm not writing something stupid. Simply not using transactions at all is something I can't even fathom. That's basically no better than blindly mutating shared in-memory data structures in a multi-threaded program. Who would write something like this?
Exactly. Maybe not everyone needs to be an expert in databases and distributed systems, but if you’re developing web technologies, then someone needs to be the expert. They will need to have the vocabulary to communicate with the rest of the team.
I think it is possible that an engineer knows about the need to use transactions while doing certain kinds of DB updates while not knowing the exact definition of the ACID acronym.
I suspect the author of this article would have no objections to probing if a candidate understands transactions and indices.
Exactly, I've spent years optimizing RDBMS and columnar databases and I couldn't at this moment fully explain the definition of ACID to you without having a wee google refresher.
I mean, I learned ACID's definition, nodded because it made sense, realised that MySQL's default (at the time) MyISAM storage engine wasn't ACID, it flat out failed at C, so encouraged my LAMP stack using friends to switch to InnoDB, then switched to Postgres myself.
Likewise, I use a lot of distributed systems, but I don't think in terms of the CAP theorem on the regular.
Rather I just think about all the fun ways each distributed system can die in, and while ZK is CP (although I'm dubious about the P), and Kafka is CA, the failure mode is roughly the same - you lose enough cluster members and shit breaks.
I know that my k-safety (k being how many nodes I can lose and still have a system) for ZK is N - (N +1)/2 (assuming truncating int division), and as for Kafka, well... it really depends on the failure.
But the usual conservative design for a Kafka cluster is n * 3 replicas on a 3 AZ stretch cluster with min.insync.replicas set to num.replicas - 1,leading to a k safety of 1.
I'm a fan of the 2.5 stretch cluster, if I have to do a stretch cluster, I generally prefer separate clusters replicating.
So for a 2.5, you'd go 2 AZs with M brokers and N ZK nodes, where M % 2 == 0 && N % 2 == 0, with one ZK in the third AZ as the tiebreaker.
You can now lose an entire AZ, and still have a quorum, while paying 33% less inter-AZ traffic from your Kafka clients.
Basically, theory underpins all my work, but the theory and the reality are quite different things.
When I read that sentence, I gasped and thought the same thing.
I've spent a career writing thousands of lines of application code that would have never been needed if the database had been designed properly. But it wasn't because the deployer didn't think they needed "deep knowledge" that day.
We stand on the shoulders of giants who figured out the fundamentals just for us.
The worse the violation (different data types in the same column, ugh!), the more application code (10X) will be needed.
The more code written, the more more maintenance (another 10X) needed.
And none of it ever gets replaced, just migrated to something else years later (if we're still in business).
ASIDE: As for the interview question, I would never ask what "ACID" meant. I'm more interested in understanding than recall. I would just give the candidate a poorly designed database and ask what they would do to fix it.
And that's why you have senior engineers and junior ones.
If you are trying to hire, expecting every junior dev to know all this stuff means you'll skip a lot of candidates who can contribute a lot to your business.
They shouldn't be trusted with writing things from scratch. But if your codebase is mature enough, getting db connections, making endpoints atomic and stuff like that already should have a convention so your junior engineer wouldn't have to do it on their own.
> I've spent a career writing thousands of lines of application code that would have never been needed if the database had been designed properly.
And then there's the time that I rewrote a small database application, but couldn't really reduce the byzantine schema, as that's just how our business works. However, I did reduce the main query from 727 lines to 93, and made it run twice as fast. And then, of course, the project was canceled because of politics, and the group still struggles with their terrible application, 3 years later.
I should really write a book about terrible management in Fortune 250's.
The author says, "I wouldn’t be able to explain from the top of my head what the ACID term means".
I think there is a big difference between not knowing something off the top of your head and being completely unaware of it.
If you have awareness of something, you can make decisions based on that awareness without necessarily having a deep understanding. When you have no awareness is when the big issues arise.
For example, with your MongoDB fuckup if you had even a vague idea about the underlying behaviour you might have been able to consider the failure scenario ahead of time and prevent it from occurring. A deep understanding of how MongoDB works probably wasn't required to prevent this incident.
Same for indexes. If you're aware of them, you can trigger the thought process of "this query is pretty slow, maybe I should look into adding an index because I know that speeds things up". You don't necessarily have to understand how a B-tree works for an index to be effective.
> The author says, "I wouldn’t be able to explain from the top of my head what the ACID term means".
> I think there is a big difference between not knowing something off the top of your head and being completely unaware of it.
The next level is being aware that the “I” doesn't quite apply in most databases with their default transaction settings, true isolation/serialisability being sacrificed for performance, though the distinction rarely matters is can be significant.
> You don't necessarily have to understand how a B-tree works for an index to be effective.
How a b-tree operates can be safely kept as a black-box thing, the important level of detail that many miss in my experience is how compound indexes can (and more importantly, can't) be used. Others include that in most databases defining a foreign key relationship does not create a supporting index by default, and that careless use of functions in joining & filtering clauses often blocks index use. If you have a DB specialist on the team there is probably no need for other devs to be intimately aware of even these details, but they should at least be aware enough of their existence than they know to ask the local expert (or look things up elsewhere) to check their working when the issues might affect what they are working on.
As a bit of a local DB specialist who has met some bad ones, I should point out that it is important that we, and other local “exports”, be approachable so the other devs feel they can run things by us for verification/improvement.
To some degree the more competent the person in using ACID databases, the less likely they are to remember what 4 random words are hidden behind ACID. If someone uses a database that makes ACID guarantees there is no benefit to remembering what they are. It is just how the database works.
People only need to know about ACID if they are arguing about whether to use a database that doesn't implement it, or switching between databases some of which do and some of which don't provide strong ACID guarantees.
I disagree entirely with this, ACID only makes guarantees at the database level itself not the system as a whole; it's possible that using ACID in the database could render the system as a whole more vulnerable than not using it. Take for example MySQL/InnoDB which offers durability (the D in ACID). If you have a write heavy database you could end up in a situation where your server ends up stalling waiting for MySQL to commit the write to disk. In effect, your server ends up building a large buffer of queries to commit to MySQL and sits there waiting for MySQL to become available. If your server or system crashes, all of that data gets lost.
You may have been much better off, both performance and correctness, to relax MySQL's durability from flushing on every write, to flushing/committing every 1 second (using innodb_flush_log_at_trx_commit). Doing so improves performance by multiple orders of magnitude (it can be the order of 10-100x) to the point that your server is no longer sitting there queuing up queries for writes to complete. The database loses some guarantees about durability but the system as a whole gains better durability.
I could make similar arguments about the other properties as well, the point is that a competent person should actually know this stuff, understand it, not just memorize an acronym and boil the entire issue down to whether a database has ACID or not. It's not a binary matter, it's more of a continuum where each property can be tuned to various degrees. MySQL and Postgres both offer many options to tune the various ACID guarantees from having complete conformance to relaxing them with various upper bounds. Relaxing guarantees about a local property of the system can improve the guarantees of system as a whole.
> this query is pretty slow, maybe I should look into adding an index because I know that speeds things up
On my current work database is riddled with indices over various status columns. Those indices are not selective and basically useless. Worse of all: it's not immediately obvious that those indices are bad, because inserts/updates are fast enough, but they slow system down as they accumulate and there's no easy way back.
Sadly, a good DBA would be able to give you information on these things, but the DBA like system admins is a fading "not needed" position. I know HN hates stored procedures, but having an application use stored procedures instead of on-the-fly queries allows a team to do the proper optimizations. Databases evolve with use and added records, and few teams devote resources to stay ahead of the changes.
I personally love stored procedures, and Postgres, MSSQL, Sybase and Oracle all provide pretty good languages to implement them in.
In my experience the most reliable and long living projects treat the DB as its own separate product, whose interface to data mutation is via stored procs. The db can then be sanely designed with good data integrity rules (foreign keys, unique indexes and data column constraints).
Then external callers use the stored procs as their API, and then mostly don't have to worry about the internals. You can easily create a web based client, REST calls or a native GUI app that all work directly off the same db.
However they do have to worry about :
-connection timeouts
-query timeouts
-deadlocks
> If you have awareness of something, you can make decisions based on that awareness without necessarily having a deep understanding. When you have no awareness is when the big issues arise.
I think in many cases, it's that even just having a vague awareness of something prompts you to double-check more often or to do something about it when you aren't really sure.
As an example, I find myself reading about the pros and cons of various PostgreSQL index types over and over again. It isn't really all that important for me to retain every detail about them for immediate recall but knowing that there's enough of a difference motivates me to read the documentation or run a quick web search before shooting myself in the foot.
I tend to think that testing people on this kind of deep knowledge isn't really helpful nor is it indicative of how developers really operate. What you need to test them on is what steps they would take when they aren't really sure and how they learn from it.
Yeah that’s the difference between a “blank stare” in the interview and a decent response. Anyone applying for a database position should know that relates to transactions and yadda-yadda. They should know why transactions are important, etc.
But… I’d like to say this is the kind of problem these quick-boot coding schools fall short on. Sure one can learn programming language X, but that doesn’t mean one has any sort of fundamentals in algorithms, data structures or general database technology.
> For example, with your MongoDB fuckup if you had even a vague idea about the underlying behaviour you might have been able to consider the failure scenario ahead of time and prevent it from occurring.
This is going to require being conversant with the terms in that ACID acronym.
> A deep understanding of how MongoDB works probably wasn't required to prevent this incident.
That deep understanding is exactly what was required. If you don't understand how Mongo works, you don't understand the ways it can fail to do what you think it will do.
> If you're aware of them, you can trigger the thought process of "this query is pretty slow, maybe I should look into adding an index because I know that speeds things up".
So why not add an index for everything ahead of time, then every query will be fast, right?
Or maybe you need to have a sense of the cost of adding an index, then. Which may require knowing the different kinds of indexes and what the speed/cost tradeoffs are.
I think in all these cases, not every developer has to understand these things, but if you don't have anyone in your organization that understands them and is thinking about them, things can blow up for you at unexpected times that make your customers very unhappy. And then you might need to hire someone who understands them to fix the mess.
Well, the interviewer did not complain about getting answer partially wrong, they complained about getting a blank stare back. The interviews are not like school exams, they are not graded by the number of exactly correct answer.
I don't remember what each letter of acronym means either. But if I was interviewee in this situation, I'd just start talking about transactions from user perspective -- how they either completeley pass or completely fail, and how even with power failure, you are not going to see partial data.
If I were interviewer, I'd accept this answer. No one cares about hearing exact words (0), "user will never see only a part of transaction" is good enough even if person never said the word "atomicity"
(0) According to some sources, Google runs scripted interviews where interviewer is non-technical, and grades answers against a checklist.
Anecdotal evidence: I didn't know what ACID meant, but I know each of the things:
• the transaction is the minimal (atomic) unit, and must never be split up;
• all transactions (and everything else too) must map valid states to valid states;
• parallel database transactions should be viewed as a directed asynchronous graph over valid states (like Git)
• store stuff on the hard drive rather than RAM (i.e. the hard part of making databases; filesystems are horrible!)
I have fairly deep understanding of these things; I just didn't know the cool acronym mnemonic. (Database engine devs don't tend to use it in their blog posts about the internals of SQLite or Postgres.) “If they didn't learn how I learnt, they don't know anything” is one of my pet peeves.
You're right but you or more precisely that interviewer is also wrong.
Understanding the way your storage works is crucial, yes. But you ensure people understand it not by asking "Explain ACID to me" but by asking to design a certain system and observing them figuring out requirements and asking questions. You then ask follow up question about consistency and reliability, you probably even can ask about benefits of using transactions but I wouldn't because there's plenty of databases that can be used reliably that don't have transactions.
Life is more complex than ACID, that's it. Investigate if person understands life, not that they were able to memorise bunch of magic acronyms.
For the life of me I don't remember what SOLID is and I totally fail to parse any written explanation of liskov substitution principle but whenever I see code that violates those I immediately go "please don't do this, that'll cause problems in the future".
> I totally fail to parse any written explanation of liskov substitution principle
Off the top of my head, "all child classes should be usable interchangeably when dealing with an interface/base class". I guess that's too imprecise to qualify as "written explanation"?
This doesn’t sound correct to me. The original statement is, Let ϕ(x) be a property provable about objects x of type T. Then ϕ(y) should be true for objects y of type S where S is a subtype of T. Here, objects of a base class are replaced with objects of derived classes.
The GP said "usable interchangeably", which is vague enough to admit a wide variety of interpretations, including the more rigorous one Koshkin posted.
Exactly, the acronyms should be viewed as tools for the interviewer, not as trivia for the interviewee to know. Asking the interviewee to explain ACID or SOLID is fine, but if they don't know the acronyms that shouldn't be an instant fail, it just means the interviewer has to put in a little more effort. The interviewer should go deeper and say "Given we're using this data to do this, which of the options of the MongoDB insert should we use?" or for SOLID, show them a piece of code and ask "What criticism do you have for this code? How would you refactor it?".
Given how some people study for these interviews, you might be better off going deep either way, to avoid getting rehearsed answers on these trivia questions.
> Understanding the way your storage works is crucial, yes. But you ensure people understand it not by asking "Explain ACID to me" but by asking to design a certain system and observing them figuring out requirements and asking questions. You then ask follow up question about consistency and reliability, you probably even can ask about benefits of using transactions but I wouldn't because there's plenty of databases that can be used reliably that don't have transactions.
Who has time for that? Interviewers design interview for themselves, not for the interviewees. It's basically a trivia game at this point of time. A prime example of "Everything is obvious once you know the answer" syndrome.
Now that is up to the skill of an interviewer how will they approach it.
Often you're time-limited and can't do actual code testing, and acronyms and buzzwords can be useful to do the initial filtering quickly. Of course, not expecting for candidates to reproduce what each letter means, but to show they've heard of it and have some idea what it's about.
> For the life of me I don't remember what SOLID is and I totally fail to parse any written explanation of liskov substitution principle but whenever I see code that violates those I immediately go "please don't do this, that'll cause problems in the future".
In my book this is a perfectly valid answer to that question, actually far better than just citing the book definition blindly. Interviews are all about trying to read between the lines, and often the whole impression of competence that candidate leaves to you is far more important than the actual answers.
> Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?
There's a difference between knowing the internal implementation, and the pros/cons of different usage patterns. What you're talking about is having mastery over an interface. This is distinct from knowing the internal implementation.
As someone who used to work as a hardware engineer, I can assure you that there's a tremendous amount of hardware internal-implementation-details that you're ignorant of. But that doesn't get in the way of you being you a great developer, as long as you spend some time understanding the interface and the recommended ways to use the hardware black-box. The same applies to what the OP is describing.
My statement is that it was a terrible example, but the core principle stands. Yes, we should all definitely understand how the storage layer works (and anything else that could corrupt data) but no one needs to have the laundry list of acronyms memorized.
I think your comment is basically just reacting to the title or a snippet of the article and not really giving the OP due respect. For example transactions are clearly a part of the api for a SQL database so not having deep knowledge and not using transactions are not equivalent things.
I think the article also touches on the fact that when you look up the definition of ACID you get a rough description of the acronym and then a description of how it is mostly a marketing term and all the ways real systems can deviate from it. So really the deep knowledge is not so much what ACID means as what it doesn’t.
The article is basically complaining that knowing what ACID stands for is not really a fundamental and that knowing how to use a database is. So I don’t think you really disagree with the OP much either.
If you’re implementing interactions with a fairly new database (let’s face it. 10 years ago MongoDB was basically experimental) then it is indeed incumbent on you to understand in greater details what you’re doing.
But if you’re building a CRUD web app, that’s maybe only used internally by your company and backed by SQL Server you probably have to never give it a thought.
Either way, lack of knowledge of a particular acronym, one which is described as basically meaningless in texts that describe it, is not a deal breaker of any sort. And it doesn’t apply just to ACID but most acronyms in tech today, such as REST/SOLID, etc like the author points out.
> But if you’re building a CRUD web app, that’s maybe only used internally by your company and backed by SQL Server you probably have to never give it a thought.
Until you get complaints that the hour-long reconciliation transaction you run in the morning is locking out all the users, or there's an API a customer hits that's erroring out 1% of the time because Sybcoughuh, Wonderful Microsoft SQL Server has detected a deadlock, and this is causing us to lose customer orders, or...
No, you really need to know what ACID means, and what tradeoffs are made to balance ACID with other considerations in your chosen database implementation. Lumping ACID in with vague heuristics like SOLID on the basis that they're both acronyms is just plain ignorance.
> Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?
Sure, you're right, but I'd argue that transactions are is hardly "database internals". Transactions are day 2 of "using a database 101".
You don't have to know what ACID means or how transactions are implemented to use them correctly.
I'd rather hire a candidate that knows how to use transactions but has no idea what ACID is than one who knows what ACID means and how it's implemented but doesn't know when or why they should use transactions in real life. (Of course the second case is rare, but this is just an example).
I get the feeling that many people do not even know what problems databases solve. It seems to be common to treat them as an extension to the application's memory that somehow isn't lost between restarts, and the properties, operational caveats and other capabilities of the database are not even considered.
I don't think that transactions count as deep database knowledge. Those are the absolute fundamentals that most developers need on a daily basis and not some dusty computer science stuff. If you have never heard of them it is fair to say that you should take a step back and learn the basics of your profession before being allowed to touch a production database. But the thing is of course - you don't know what you don't know.
> But, that’s only because you are a noob and you don’t deal with the things that I have deal with.
> Someone on the internet
In conclusion: not all software problems require the same implementation.
I don’t run a DB for some deterministic applications I have that can regenerate the same outputs such I don’t need to store it all. I could, in theory, regenerate an entire data set with a seed. Maybe app developers are ignorant and lazy for not designing systems that do that!
What does your one story have to do with “software engineering” as a field?
Sounds like people are conflating problem solving and personal career demands.
When I read it, my first thought was that "an API" in this case was SQL.
Understanding why we should use database transactions is not the same, to my mind, as understanding database internals. When the author said they can treat the database as a "black box", my thinking was the they didn't necessarily need knowledge of the database's implementation. People can understand how to effectively work with a database in the general sense (i.e. use transactions) without a deep knowledge of every implementation.
When we start talking about MongoDB, I think that's really interesting because it is so different from our traditional, SQL-first databases like PostgreSQL, Oracle, etc. Our understanding of SQL-first databases isn't all that useful when we try to apply it to MongoDB.
At that point I think we are in agreement: it's unreasonable to think we can treat a MongoDB database as a black box if our base knowledge is only SQL and SQL-first databases. We probably do need someone who has already dug in and dealt with the implementation, how MongoDB really gets things done.
One of projects I was called to help some time ago set up a service with over a thousand servers. They reached out to help improve the performance because in their opinion the network and network filesystem were too slow.
They invested incredible amount of time in learning technology and scaling their application but forgot about need to learn fundamentals -- data structures and efficiency.
Fast forward 1,5 years, the application as I left it ran on a single server using about 10% of its capacity. A second server is just a hot standby backup.
The service went literally from being able to process tens of transactions to a hundred thousand transactions per second on a single node.
More than that, we threw away most of the exotic technology that was used there -- greatly improving team productivity. The implementation is simpler than ever with layers upon layers of microservices replaced with regular method calls and a lot of infrastructure basically removed without a need to replace it with anything.
People who do not learn from history (fundamentals) are doomed to repeat the same mistakes.
Something people seem to forget when designing systems based on what they've learned is that modern individual machines have approximately the power of the top supercomputer ~20 years ago, and a few racks of modern machines can in many ways match the top supercomputer of ~10 years ago. Approaches to solve large problems are continually changing and it's almost always worth a big-picture design session when looking at a familiar-seeming problem. By the time students graduate from a 4-year program computing is about 10X better than when they started.
I still remember some interviews within the last decade where single-core machines with a few GB of RAM were a default assumption for whiteboard designs, or that spinning disks were the default.
Well, I catch myself that I am not "updating" the state of my understanding of hardware. Just recently I caught myself putting a lot of effort into solving a problem that only exists if persistent storage is too slow to be used for calculations. Then I facepalmed myself hard when I realized that I can just move that entire 500GB data structure to an NVMe and treat it almost as if it was in memory.
But in general I think that the problem isn't that people are not "updating" their understanding. Even 10 years ago it wasn't a huge problem getting hundreds of thousands of transactions per second on a single modest machine.
The problem rather is people relying on more and more layers of abstractions for vary small gains.
Example: I get that Python is a nice language (for somebody that does not know Lisp). But is it worth it to choose Python for a little bit improvement in productivity for a problem that requires a lot of throughput, to then suffer performance issues, to then spend many times more effort on trying to improve performance? I don't think so.
Or more in my space: is Spring Data (Java) worth the very incremental productivity improvements if it completely destroys your application performance? The application I described in my parent post used Spring Data MongoDB which kinda means it was fetching entities one by one which is extremely costly.
By replacing it with bare MongoDB reactive driver and ensuring data is being streamed in large batches (why read one user data if you can read 10k at a time) and getting rid of costly aggregations in favour of application side processing we have improved throughput by many orders of magnitude WHILE reducing load on MongoDB.
Granted, there is a little bit of additional complexity (on the order of 10% more of application code) but just the performance improvements mean that the team can breathe and focus on other problems like modelling the domain correctly.
> Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?
I would consider transactions part of the api. I use them all the time in postgres and I'd hardly call myself an expert in the low level details
I don't think use of transactions or any other query or modeling hygiene are "internals". You can just as easily consider SQL to be an API (or really a DSL) for the black box of DB internals. In most cases, you're better off not looking inside the box because DB vendors only guarantee behavior at the API level. You can always identify and improve bottlenecks if/when they actually impact application function. I can write a load of crummy queries, expose them in an API and then just apply some clever caching and the DB will just never be a bottleneck no matter how little I understand it.
I mean sure, yes, there's also programmers out there who've been working for years and look at me like I'm speaking in tongues when I talk about closure.
The author's casual arrogance about "pixel pushers" only caring about "hotdog stand colors" is just as damaging to the profession -- as it infects the hardest part of getting things right most of the time: teamwork.
I think you misunderstood his point, I think what he's saying is that even good back end devs are horrible at designing UI and you need different people with different specialisations on a team.
I'm horrible at designing UI too. That's simply not the job of building modern interfaces across multi-disciplinary teams.
I care far, far more about data-structures, eventual-consistency, data-design, and types than CSS and divs.
These days getting the design to look nice is my two-hour reward for nine days of data-stitching hell, writing backend-for-frontend lambda functions, and negotiating API contracts that actually support the product I've been asked to build.
We need to stop assuming that just because a programmer works on client code, that they are in any way less capable that a programmer that works on server code. My point stands: It's absolutely toxic to collaboration, which is the key to success.
I think he was saying that the GUI programmer doesn't need to know how to use a profiler for their work to be valuable, and that if you're good at optimizing you don't need to know how to choose tasteful colors for your work to be valuable. My reading is the opposite of your reading.
The part which was more problematic for me was the correlation between having a comp-sci degree and working backend, it felt like a subtle form of gate keeping i.e. you could interpret this as: haven't got a degree? you will be working frontend/ui.
I will still never understand why I was asked to code an implementation of a binary tree for an interview once, something that is usually left to those who implement standard libraries or similar low-level needs, for a job that was basically writing user interfaces.
A binary tree is just a tree data structure where each node has at most two children. And that's dead simple and every programmer should be able to implement that when aksed. Note that it doesn't even have the search (i.e. sorted) requirement and much less the self-balancing one. You can work your way towards that with the right questions and that would be an acceptable way to structure an interview. Start with something simple everyone should be able to do and then work your way up the requirements and discuss potential problems/pitfalls and their solutions.
Nobody talked about implementing a specific variant of a red-black tree when the general topic of self-balancing binary search tree hasn't even be mentioned.
It is dead simple. What you're describing (two children, no constraints) is basically a cons cell. Yet I suspect that most people don't use cons cells as binary trees. Which is probably why I didn't understand that apparently rather nebulous topic as being this generic.
To add to that, a binary tree is often not really "implemented" but simply encoded in an array. That is, the children of any given node i are 2i and 2i+1 in the array.
So depending on the actual exercise/interview question you might able to just show that, especially of you have to use pseudo code.
Correct me if I'm wrong, but isn't this exactly the case where you don't want an unbalanced tree? A degenerate tree is bad enough; a degenerate tree implemented as an array in this way is catastrophic. So it seems to me that if you're refining the topic further in some sort of talk, the topic of balancing should precede the topic of physically representing the tree as an array.
It all depends on the specific topic. If I just want to store my data in binary tree structure without the possibility of deletes and just with an insertion-order requirement, then the tree is automatically balanced and using an array like that is totally fine. The use-cases are probably rare (I can't think of a situation from the top of my head), but binary tree doesn't have to mean search tree.
The self-balancing part is usually added after the search part as balance is trivial/unimportant when the order/position of the elements has no constraints.
> The use-cases are probably rare (I can't think of a situation from the top of my head)
I imagine there's quite a lot of them. Things like VP-trees and such get often built statically and could be implemented in a similar way. Not sure if the simple (2i, 2i+1) schema is optimal for this but perhaps it could be adapted.
This is another point in which this can be a good interviewing question. For certain jobs you don't want someone who just forges ahead with an unclear specification.
That said, for most dev jobs implementing a binary tree isn't necessary knowledge. But if I need a dev who can get their hands dirty I would be weary of hiring someone who can't take a decent stab at it during an interview.
This is an arrogant comment. You are right that the general algorithm of a binary tree is not complicated. But coding it under pressure, for anyone who is not writing that kind of code on a daily basis, which is most people, and especially for a job where you’re not required to write that kind of code, is ridiculous.
If you audition for the New York City ballet, they don’t expect you to ice skate.
I have a computer science degree from 10 years ago and I couldn't tell you how to implement one. I vaguely know the general principles but I'd have to sit down and start from scratch on how to implement one.
Why? The only time I knowingly used them is with database indices. For everything else I do, they are an unnecessary implementation detail. It doesn't matter what Ruby's Set uses.
People who make this comment, I generally don't believe you, or don't believe you have done any programming. If I put a gun to your head and give you an hour, you can't make a node that points to two other nodes of the same type?
If you really can't, you should get familiar with how data can point to other data, it is fundamental and done all the time.
Well, the requirements I faced were to implement the tree such that nodes were inserted with the standard ordering you would expect. Walking the tree to find the best place to insert them, etc.
> If I put a gun to your head and give you an hour, you can't make a node that points to two other nodes of the same type?
It's easy to make an imbalanced binary tree.
I once decided that building a binary tree based only on a friend's explanation of how they worked; it seemed like it would be a good learning exercise for myself (I don't have a CS degree). It was like 10 minutes of coding to get it minimally working & tested, but then I had to work out all of the different rotations to make it a balanced binary tree (I don't know that an imbalanced binary tree is ever actually useful, as it could just work out to be a slower linked-list in the worst case scenario).
I did it once, took me a little while, but I don't know that I could do it again without having to reexamine all of the different cases that result in imbalance and the rotations that respond to each.
There are loads of simple tasks, there should be a bit more reasonaing as to why this is a good filter relevant to the job's day to day.
For example I think writing some logic to filter datapoints or timespans is much better and more practical. But it takes more effort than just opening and pocking binary tree.
The fact that you didn't even think about asking if it's balanced or not, and just went into declaring stupid everyone that considers the problem hard is annoying.
I wouldn't want to have an interview with you, or you selecting people for a team I'm on.
Because sometimes the requirement is that the engineer being hired is able to come up with efficient algorithms. You don't have to be able to create the textbook binary tree algorithms, but you should be able to derive a reasonably performant algorithm with simple data structures if it matters to the position you're being hired for.
> requirement is that the engineer being hired is able to come up with efficient algorithms.
Coding binary tree is not the way to discover that.
Presenting a task and observing people solving it and explaining computation complexity behind their solution and the thinking behind their solution is how you do that.
It's a fine way of doing it, although certainly it shouldn't be the only algorithms question.
It's a simple data structures / algorithms question. Can you think in terms of how data is stored in memory? Do you understand the tradeoffs of storing data in one form or another? Can you work with recursive algorithms? Given a challenge of having to think about data in a form you haven't recently, can you wrap your mind around the characteristics of that data structure?
If you're getting asked all this for a ReactJS UI building job, then the interviewer is probably off-base. But otherwise, make sure you're applying for jobs you're suited for. If you don't want a job where coming up with efficient algorithms is important, that's fine - there's plenty of others out there.
> Can you think in terms of how data is stored in memory?
If you're using a modern OS, you don't know this. The kernel remaps everything and pretends to give you a linear chunk of memory. You don't know what's in the various caches and what's been paged out of ram completely.
Everything is built on layers of abstraction. C is an abstraction layer that fits poorly into what modern CPUs and operating systems actually do, but everything is programmed to cater to C's assumptions so people don't need to rewrite their programs.
Higher languages don't necessarily use the data structure you think they do and they can switch the implementation details based on the size.
"Simple data structures" are only simple because you pick one specific layer of abstraction to work with.
That's not an excuse for not being able to reason about data structures if your job requires working with data structures and algorithms. The logical extension of your argument is "put everything in a linked list and let the OS work it out."
If I give a candidate a truly real world difficult problem with an hour or even a day to solve it, some people who are not so smart will get the correct answer by luck, some people who are brilliant will start off with a bad approach and not get there.
If I ask if you can do something basic binary tree things, I will also not know if you are brilliant, but I can at least rule out if you are totally clueless or not.
Well hopefully there's some layering in the question if it matters to the job. If you have an engineering position in which a high degree of CS competence is required and the candidate can't even make it past the initial layer of "regurgitating binary tree operations" then you have the wrong candidate.
> If you have an engineering position in which a high degree of CS competence is required and the candidate can't even make it past the initial layer of "regurgitating binary tree operations" then you have the wrong candidate.
Or you have a candidate who understands binary trees, can implement it but can't regurgitate binary tree operations from the top of their memory.
For me the point would be that a no-frills binary tree is simple enough that any dev qualified for such a position should be able to figure out what needs to be done from the specification, without needing to regurgitate anything.
If they can't recall how balancing is done that would be fine with me, as long as they mention it.
Exactly. And that's the reason you'd probably layer the question with some more complex asks after the initial answer. If somebody has memorized the algorithm but then can't explain further, it's obvious they've "regurgitated" without really having a deep understanding of the problem being solved with the data structure. By contrast, a good candidate may not be able to immediately repeat the algorithm, but can reason their way there pretty quickly.
> By contrast, a good candidate may not be able to immediately repeat the algorithm, but can reason their way there pretty quickly.
I know dozens of good candidates who can't get there pretty quickly but are really good. I also know dozens of good candidates who can get there pretty quickly but won't be good enough on the job.
Maybe they'd be really good at other engineer positions, but if they can't get to a binary tree algorithm reasonably fast I wouldn't hire them for backend positions that require CS reasoning.
> Maybe they'd be really good at other engineer positions, but if they can't get to a binary tree algorithm reasonably fast I wouldn't hire them for backend positions that require CS reasoning.
You not hiring for backend positions doesn't mean they are not good at the positions. It just means they aren't good at your signals.
For me it's a strong signal, but it shouldn't be the only signal. And I do think it's good to have a diverse team.
Some of my coworkers are not nearly as strong as me when it comes to "CS details" like data structures or programming languages, but they have other attributes which complement me and which are good for the team.
Having a bunch of me's (or any of the others) would be a disaster.
find = lambda k, t: t if t is None or k == t.k else find(k, t.left) if k < t.k else find(k, t.right)
I mean, figuring out how to rebalance a red-black tree after insertion is more demanding, and you could legitimately quibble with my golfing here, but I feel like the reasoning required to write that function (given the tree definition) is just not that advanced. It's like the FizzBuzz of pointers.
I asked this once as a take home for an interview. First, I copied the first hit for “implement binary tree in C” from google. I made sure it compiled and then deleted the implementation of the insert method. That was the take home. I wanted to know that the applicant could reimplement this one missing function and compile the project and not get immediately stuck.
Because it is the 2nd easiest possible data structure to implement, if you can't do that it means you don't grasp how data can point to other data and how you can leverage recursion to make hard problems easy to solve. Perhaps they were hiring for someone that they needed to be capable of more than just trivial CRUD wiring. Even when you aren't writing core libraries these things come up constantly, sometimes at higher levels when designing systems and how they interact even.
I implore you get to a state where this isn't hard for you, it will make you capable of more.
That was probably a question to determine basic competency in coding, simple data structures and simple algorithms. I.e. if he can do this, that means he's at least minimally competent. if he can't, then we don't have to waste any more time.
> not every programmer is dealing with performance issues as part of their job. Some has also specialized in writing GUIs and are more concerned with the user experience and making the user interfaces pretty.
This is a really important point that we don’t talk about enough, because everyone gets uncomfortable and feels judged. But I can’t help myself, and I think there’s a lot more to dissect here.
I think programming is slowly bifurcating. We’re growing into two disciplines.
One discipline requires deep systems knowledge. A CS degree is recommended and if you can’t reverse a binary tree on a white board then you won’t be able to read our code base. Developers in this camp work on rendering engines, Vulcan drivers and schedulers. They spend lunch telling stories about that one time we wrote a game engine in J. Or arguing about type systems.
The second group makes most of the software (measured by lines of code). And almost all the software humans actually interact with. They spend their time working around weird safari quirks, arguing about electron, and enjoying the sense of ease and mastery that comes from staying in a single ecosystem for a decade. Getting better at their job looks like getting better at talking to users and customers, making on point estimates and making sure the team releases on time.
The part that gets all of us in trouble is that the first skill set is (arguably) harder - and thus much more prestigious. Everyone wants to be seen as an electrical engineer, not an electrician. So people write “software engineer” on their resumes. And then cry foul when they get assessed as a member of that first group. “I’ve never needed to know about red black trees in my career. It’s stupid for anyone to ask about it in an interview. Everyone just crams leetcode. JQuery isn’t flashy but it works and I can solve real problems with it. Hire me you fools, and I’ll show you!”. They’re sort of right. We’re just collectively confused about what sort of job they’re being hired for.
I think this confusion hurts everyone. I once wore my fancy engineering hat at a normal consulting gig. I coded up a complex solution to a tricky problem they had and called it a day. After I left, nobody on the team could understand or maintain some of the code I wrote. That caused huge problems for my client after I left. I think they might have regretted bringing me on board at all.
But we can’t even talk about this because we don’t have the language. We treat application developers like they’re just junior, in training versions of deep CS systems engineers. But practicing building kick-arse iOS apps doesn’t make you magically good at data structures. And vice versa - doing performance work doesn’t make you good at working with clients. (Ask me how I know). The majority of professional programmers do plumbing / user facing software. We need to destigmatise this stuff so people can be hired explicitly for these roles, based on the right set of skills.
The best programmers are better at talking to users and customers, because that is the harder skill. Programming isn’t a solo activity - at the very least the programmer is working with their past and future selves. Which of Linus’s projects have had the bigger impact - Linux or git?
> The best programmers are better at talking to users and customers, because that is the harder skill.
Uh, what? Most humans find social skills much easier to learn than crunchy engineering skills. Good social skills are way more common in society than, say, deep knowledge of algorithms and data structures.
But it’s not a competition. As you say, programming is rarely a solo activity. Our projects can fail due to a lack of teamwork. Or a lack of technical skill. Or a lack of empathy with our users. Or a lack of leadership. Or a dozen other things.
We need (and need to value, depending on the project) all of this stuff.
Being able to work with another skilled programmer (be that yourself in the future or someone you're working on a kernel with) is not the same kind of skillset as being able to talk to an end user about what the pain points are in their day to day work and trying to actually sympathise with them and understand what problems they're encountering without making the rookie mistake of taking everything the user said at face value and designing a system around it.
These are disparate skillsets and both of them are difficult to master. Claiming one is harder than the other is really quite shallow. Some people are going to naturally have an easier time picking up one skillset while other will have a naturally easier time picking up the other skillset. I think rather than trying to pretend like people should stop focusing on hard skills and start focusing on soft skills, we could realise for a moment that it does nobody any good to take people who are good at one task and insist that they must retrain for another when there are already plenty of people who are good at the other task.
It would make more sense to get these groups of people to learn just enough of the opposite person's skillset so that they can effectively communicate and as a result form a well functioning two-sided system where one group of people talks to users and analyses their requirements and the other group of people actually works off of these more concrete better defined requirements to actually implement the software.
If you think that "soft skills" are harder than "hard skills" it's probably because they're harder for you, that doesn't mean they're harder for everyone, and instead of pretending like this is some kind of major flaw in your own professional development, you could treat it as a gap which could be filled by people who have the opposite view of difficulty.
>I think programming is slowly bifurcating. We’re growing into two disciplines.
It has always been like this. The distinction used to be: base software developer vs applications developer.
Base software developers went to work on operating systems, databases, web servers, etc.
Application developers worked on business software.
And most decent business software development shops kept a couple of devs with decent knowledge of base development just to go for when the needs got technical.
The author is talking about hiring interviews like it’s some kind of multiple choice test. Interviews should be more like essays. Of course it is not that important that the interviewee knows the exact definition of ACID but they should certainly know what it is about. Knowing about ACID and the problem space it comes from enables people to think about problems that let to the ACID concept as a solution to said problems. It shows that the interviewee might know enough to look things up and might be capable of selecting the right answer in the reference material/search engine results.
Knowing the exact definition of ACID and nothing else means that somebody can memorize things which is useful but not useful on its own.
You want people that can find a solution and have an somewhat informed opinion about it, not people who can just memorize things.
So asking about ACID is a fine thing, but expecting a text book answer is silly.
Edit: ivanhoe said it better in their post and I changed my phrasing to be less inflammatory. (My point of different hiring mind sets on different continents is not the point.)
My take is that too many people are telling others how to interview. It is their money, their company, their job, their life. Let them do it the way they want as long as they follow the legal requirements.
If you think you have a better way, tell us about it without passing judgement on others. Tell us why it works, how great it is, give us some qualitative and quantitative data. Most of all, enjoy your competitive advantage. But for god's sake, quit telling other people they are doing it wrong.
While I don’t disagree, the biggest problem I think is that some or even many interviewers just don’t care or not motivated enough to care whether the candidate can indeed be a great help to the company or the project and approach their interviews in a shallow and formal way.
Interviews aren't even like school exams at all. School exams are designed to try to ensure that you learn a certain set of material that was covered in a class. If you missed a bunch of it, that's bad. If at the same time you did independent research in a topic that wasn't covered in the class, it's not the business of an exam to discover that.
School exams are supposed to be fair, in the sense that they only cover the topics covered in class, and only to the level of detail and difficulty covered in the class. They test for weaknesses relative to that standard. When a school exam asks a question, it's because you're supposed to know the answer.
If someone takes the school exam definition of fairness into job interviews, they are going to feel terrified and aggrieved. Interviewers talk to candidates from vastly diverse backgrounds. Other candidates didn't go to your school, maybe didn't go to school in the same country as you, maybe didn't go to college at all. There is no fair, agreed-upon set of standard knowledge. If interviewers restrict themselves to "standard" topics, they will undervalue a lot of people who have unique backgrounds and experience. Not only that, but job interviews can't just measure a candidate's weaknesses against a set standard. It isn't even really a matter of "strengths" and "weaknesses" but wanting to discover a candidate's full capabilities. You don't want to miss a candidate's depth of knowledge in an area simply because you failed to ask any harder questions that would have revealed it. When an interviewer asks a surprising question, it's often not because you're supposed to know but because you might know and they don't want to undervalue you because they didn't ask.
The fundamentals are important. In my field of engineering (not software) ignorance of the fundamentals means not knowing how to properly run studies, not being able to accurately make recommendations, not understanding proper cause and effect, not knowing the relevancy of the various settings of the industry standard software...etc etc. It's really hard to find that in new graduates and there is virtually no decent training material. It typically takes 5+ years for just minimum competency, but over a decade before someone begins seeing how everything fits together. Is that common elsewhere?
Power grid operations. Things are enormously complex (business rules, standards, software stacks, processes, massive data sets, and of course the fundamentals).
Knowing fundamentals is important. It is simply part of the professional culture, or part of what they sometimes call a degree of professional “maturity.” A programmer who is not aware of the fundamentals will always find a way to write bad code.
Yes but there is a catch. I've seen some shit from some fundamental lover programmers. The fundamentals are important but not essential to every job, there is hardly a role they can't improve, but they're not guaranteed to give good outcomes. These days is more important to know how to compose high level components in a simple way than optimizing specific algorithms you can learn one from the other tho.
I love these "higher order" questions. Every thinking person should have their doubts or opinions about what is thought of as common knowledge, "best practices," etc., be aware of their limitations, know about alternatives; and, in general, despise any sort of dogmatism - especially in fast developing areas such as programming.
It is... but the book knowledge version of fundamentals and their actual implementation are completely different when you start asking questions.
Even more so when those "fundamentals" are implementation details you don't think about much in your day to day job. Actively recalling knowledge that has become part of your intuition and understanding of the world is hard.
I know enough about many things to do basic implementations safely; everything else is paged out to google. The finer details of database transations? Usually not my job, but I used to know more about them and could relearn that information quickly if needed.
The Feynman principle applies - it is important to know the thing and not as important to know it's name. In an interview, find out if the person knows them thing or can work it out.
> it is important to know the thing and not as important to know it's name
Yes and no. For years I did little computer programming projects on my own. Built some neat things, for my purposes at least. But because I didn't have the words to describe things I wasn't able to build a concrete image in my head. When I finally took a CS course and had the words to describe concepts I'd used for so long it was like a switch had been flipped, allowing me to think about problems in a much better way. I knew what many things were but because I didn't have the concepts mapped to concrete terms they remained fuzzy.
Exactly. If someone does know what ACID compliance is that's great. We share a common language and that makes things easier. But if they don't, no big deal, but they should still know the concepts. There are people out there who don't know the difference between mongo and postgres. That's lacking the fundamentals.
> So I had learned already that names don't constitute knowledge... That's caused me a certain trouble since because I refuse to learn the name of anything... What [my father] forgot to tell me was that the knowing the names of things is useful if you wanna talk to somebody else.
Yes indeed. The names are relevant for communication (i.e. to become a part of a "discourse community" [1]), but are not indicative of one's competence at a topic. They kind of go hand in hand often due to arranged education, but that's not essential.
Names also come handy when you need to search for something and search engines are not powerful at taking your description of what you're searching for.
It's been sort of 18 years since I started using another heuristic - if you can google for something, don't bother requiring a candidate to know it. Not an exact principle, but somewhat like that .. in that I'm more curious about how a person thinks in an interview than what he/she knows.
This change in me was triggered when a candidate came in for a dev interview and we were doing the usual whiteboard problem thing back then. Someone on the panel asked "what would you do if you faced this problem on the job" and he said "I'd google for the solution". I recall not being able to make up my mind about whether to hire him right there or send him away :) .. but that was something I hadn't realized back then so the issue was not with him but with me. The lesson stayed.
So .. on the topic, if the candidate didn't know ACID .. an opportunity to google followed by a detailed discussion on it would work for me.
It actually might be more informative. If a candidate, that had never heard the term "ACID", was able to Google it and talk intelligently about it I would be most impressed and it would probably earn them a next step. In fact, that might be a good practice to try out. Find a related term that the candidate wasn't familiar with and have them research it. Give them X minutes to reason about the term and then have a discussion about it. This is a thing we all do, every day. You can't possibly know everything so you must know how to find out about new things and how to think about them.
This is basically because interviewers need to properly advertise the level of depth they want in a role. It's unfair to ask for each engineer to be in the top 5% of engineers everywhere -- not every engineer can be -- but there are roles that don't require such depth.
But interviewing in general is tough (I'd go so far as to say it's a shitshow) and there's lies on both ends -- companies and managers and folks on the inside just want the best of the best -- and candidates just want a chance to get in the door -- the incentive to embellish is too high.
Interviews are creative fiction: the applicant pretends to be competent and the employer pretends not to be a toxic workplace environment. I’m increasingly convinced that this is not a solvable problem.
The interview style that requires lots of prep studying and leetcode memorization probably does produce the desired result. Workers that will spend non-work time on arbitrary company activities of questionable value.
I've found bad interview questions fall into one of three camps:
1. The interviewers aren't subject matter experts themselves but are hiring managers, so they've taken their questions from someone else.
2. The company gets far more good candidates apply than they can hire, so the process is less about removing bad candidates and more about having some arbitrary way of whittling down good candidates.
3. The interviewer is trying to prove how clever they are or the company is. Sometimes for their own ego but more often because they think it might make the company more attractive to work at.
If there is a theoretical question I'd normally Google / DDG instead of memorise then I'd usually be honest in the interview and say "it's something like xyz but in truth this is something I double check these days as, pragmatically speaking, it is not a problem that comes up frequently. However it is something I have done before [cite example]".
If the questions is asked because of reason 1 or 3 from above, then usually they're happy with that answer. If they ask it because of 2, well as much as I might disagree with their interview process, I'm still not the candidate they're looking for. So we still part of good terms.
In an interview, they gave me access to their codebase and asked me to fix an actual (small) bug. The interviewer showed me where the codebase was, how to replicate the bug, all of that took less than 10 mins. Then he left, told me to give a shout once I am done. Took me about half hour to do it, another 10 mins to explain what I did to fix the bug.
There were no other questions - no white board stuff, no algorithm questions, no stupid questions like "where do you see yourself in 5 years?" etc. I still remember the interview nearly a decade later.
Obviously this won't work for all jobs, but this will work surprisingly well for a large number of jobs, because most jobs are ordinary. One doesn't need to know complex algorithms to do these
I'm not OP, but I imagine the concern is doing work for the company (i.e. fixing a bug in their codebase) without getting paid at least minimum wage for it.
Ah, I assumed it was just a realistic looking exercise. If they actually used code written by the applicant, I can definitely see all kinds of problems!
If I were designing this question I would probably use a real bug that we recently fixed. We'd already have a sense for how hard of a problem it was, and there's no worry about unpaid work.
I did something similar and loved it but a friend pointed out That he will fall over with an anxiety attack in such a situation. I always thought the best case would be to offer a choice to the candidate among 2 or 3 different ways to do the interview.
>I did something similar and loved it but a friend pointed out That he will fall over with an anxiety attack in such a situation.
I might, too, but I frankly might do the same to an even more extreme degree for a particularly challenging real-time 1-on-1 interview question from someone staring at me.
All things considered, I think I'd prefer being left alone in a room where I don't feel them over my shoulder while constantly wondering what they might be thinking with each passing second. I can take a few minutes to relax a little, look over the code, and try to figure it out.
I don't understand why this would give someone anxiety attack - it would suck if the interviewer sat next to me and watched my every keystroke, it wasn't that in my case. He just set me up and left to do his job. I was on my own, until I finished and called him over to show the bugfix.
I have anxiety issues when an interviewer asks me some obscure syntax option for grep command or some b-tree stuff that I learned in college decade ago (and never used in real life).
I don't understand why that grep stuff would give you anxiety. It is a dumb trivia question, just say you don't know.
But I don't understand because I am not you. Different things give different people anxiety. Anxiety isn't something you reason your way into or out of.
My entire "interview" at Netflix was me helping their top database people solve a severe performance problem in their operational systems. In the two hours I was there, I diagnosed the root cause and showed them how to fix it doing something they did not know was possible. Best and easiest job interview I've ever had.
They tested it after I left (a >100x improvement apparently) and called me the next day to ask when I could start.
4. Interviewer did not have time set aside to prepare for the interview, and sort of has to wing it. I was surprised to realize how often this happens.
Author is not completely wrong, but I don't like this attitude of celebrating ignorance. I agree we shouldn't judge others who happen not to know what we know, but IMO it's likely that knowing this concept at a deconstructed level will be helpful to most devs at some point in the future. The ones who don't know it won't even know it would have helped them, because that's how knowledge works. And that's why it's important to celebrate knowledge, not ignorance.
In my experience there is not much to know about the ACD parts of ACID, because they're just guaranties that are obvious to want and which will naturally make your life easier.
The I (Isolation) part is the one you'll need to understand whenever you scale up to a point where concurrency starts to become non trivial in your product, mainly because there are multiple levels of isolation, which each provides different guaranties, and you'll need to understand what level your database uses and what it means for your application.
As for the overall point of the article, it's just a question of scale. The more you scale the more you need to cut through the abstraction to understand what the hell is going on. Hire accordingly.
They may be obvious to you, but you'd be surprised at how many people out there are doing stuff without optimistic concurrency control, locking or transactions. I mean Mongo only added transactions for updating multiple documents relatively recently. Plus from what I remember they originally missed out the "D" due to only writing to disk intermittently.
322 comments
[ 2.9 ms ] story [ 276 ms ] threadI sort of agree, but anyone typing out code at 120WPM isn't programming, they're writing. Programming is an intellectual activity, not a physical one so learning to touch type is worthwhile but might only gain you a few minutes a day of productivity over someone doing 40WPM in a home-grown hunt-and-peck style of typing?
Everything else being equal, typing speed will help you try out stuff faster.
Of course your senior engineer that wrote the thing will be more productive than your intern that writes 1000 WPM.
I am pretty sure that people who didn't learn to type properly have to switch their whole mind frame from the content into the typing, back and forth.
Some people even speak outloud, or at least move their lips according to, whatever they are typing.
Language might make a difference there. Writing C# with a QWERTY keyboard, you still have to interrupt a normal typing flow for your parentheses, braces, etc.
I suppose it's helpful if you're writing long comments, but for the most part, I rarely hit sustained bursts of my peak typing speed when writing code.
To be controversial, a language that rewards typing speed seems to me like a failure of tooling, why am I manipulating text like a caveman (ok.. ok.. Mesopotamian) rather than with the symbols that the language actually deals with.
It helps a lot with most programming languages. Snippets for if, while, and for can also help with the {}.
Imagine that you want to correct a block of code or a whole paragraph. If you don't touch type, maybe you will think twice before rewriting it.
Besides, only Java can make any programming task I/O bound ;p
I might think for a couple minutes and then come up with ten lines of code, and once they're in my head I want them in my editor now before I lose track of them.
i developed a pretty fast way of typing naturally, using 7 or 8 fingers.
unfortunately it caused wrist strain.
i could not use split ergonomic keyboards because some keys where "mapped" to the wrong hand for that.
i also could not simply learn "proper" touch typing on a qwert layout because my typing habit was too strong.
in the end i learned touch typing on dvorak to fix my wrist issue. maybe after 10 years i'll have unlearned qwert typing enough to go back? (not sure i would, i do prefer dvorak, it just feels silly to type at snail speed on others ppls keyboards).
Regarding speed - it didnt prevent to be TOP1 at university in typing contest :)
My naturally acquired typing style involves me holding my hands hands in just about the same angle as those "ergonomic" keyboards, but just using a regular keyboard. Fingers rest on LShift, A, W, D, and K, O, P, Ø with both thumbs on Space.
Should be noted that's on a "105-key" Nordic keyboard with US layout when not writing Norwegian, so the Ø becomes '.
For me this has worked quite fine for over 20 years. I haven't bothered measuring my typing speed because it's never the limiting factor.
So I simply asked her to teach me the proper technique and now, since decades already, I'm touch-typing the proper way.
Having said that: you can be very fast using improper touch-typing technique.
However, it might be a good shortcut for people learning. I was taught at a youngish age to attempt to touch type (box over the keyboard and a tutor program) I fucking hated it, and I was slow, it was dull and painful.
However, Then I had a need for typing, so I got faster and faster, and the initial structure that I got from the rigid teaching, allowed me to learn faster than my peers.
People can practice a skill for a long time, but if they don't focus on improving, they tend to just harden their existing habits.
Any activity on a computer is aided by reducing the friction involved in getting what is in your brain into the computer and from the computer back into your brain.
Sure, you can program with punch cards and no display. People did it for years. But having three monitors and a quality keyboard that you’ve mastered will be much more efficient.
Need to use the arrow keys? Outside the touch typing range. Mouse? Outside touch typing. Function keys? Numpad? Touchbar (blech)?
I have never felt limited by typing speed. Informal touch typing is good enough. Quite often it's the computer or network that can't keep up, anyway.
But I have encountered one IT professional (not a programmer) who despite being in the industry for 20+ years, could only manage about 15WPM, if that. His style seemed to include legitimate hunting for the next key, and only two fingers.
At that speed it was starting to become problematic: It was painfully slow to watch him do anything or help him with a problem.
Not an absolute deal breaker, as long as such a person is good at their job and is efficient with what they type (taking advantage of copy/pasting or scripting), it's workable.
If I can touch-type at 120WPM, then I can churn out a great deal more documentation per unit time than another engineer that can only type at 40WPM. It's less burdensome for me to document well, to write expositional documents and to communicate regularly than my colleague that needs to budget their typing more carefully.
While I might not be bottlenecked when working on a tricky piece of code at 40WPM, I'll likely be bottlenecked elsewhere often.
The keyboard is a tool.
To me, it feels disrespectful for a person to expect these high wages when they don't even make the moderate effort involved in learning to use this tool efficiently.
In my experience, this usually signals that a person isn't actually interested in continuously improving, feels overly entitled and they tend to be a nightmare to work with.
Most places I've worked require substantial pair programming or at least working through problems together on a regular basis.
If you can't type, you're going to waste a lot of people's time and they may be too polite to say anything, but it's frustrating for everyone around you.
A carpenter who claims to be experienced, but can't work a hammer efficiently would be ridiculed on a job site.
I'm not proposing we ridicule anyone, but being unwilling to type and claiming to be competent at a job that requires an enormous amount of typing does have a ridiculous look (to me).
Of course, I have worked with people working through a variety of disabilities, including dyslexia that make touch typing basically impossible. I'm not talking about them here.
I'm talking about people who just refuse to make the effort for whatever reason. It's a bad look.
PSA for newer players: Regardless of how you may feel about this subject, failing to demonstrate at least moderate typing skill in an interview is often a hard pass unless there's some explanation requiring an ADA accomodation.
>A carpenter who claims to be experienced, but can't work a hammer efficiently would be ridiculed on a job site.
It's just not remotely the same. How quickly and accurately a carpenter can hammer has a large impact on the quality and speed of the work, while typing should never be the bottleneck for a developer
Yes, but one of many.
> If you can't type, you're going to waste a lot of people's time and they may be too polite to say anything, but it's frustrating for everyone around you.
I don't think in my career I have ever been alerted to anyone's typing speed being slow.
Yes, I've heard endless noise about "the perfect keyboard" and the ins and outs of spring types and junk. But I've never noticed the output of someone because they type slowly.
And to be honest, I really couldn't care. Can they code? can they keep to a deadline? are they nice to work with?
if the answer to all three is yes, then I couldn't give a shit about how they type.
> but can't work a hammer efficiently would be ridiculed on a job site.
Only if they need to use a hammer effectively though. If they are laying veneer, carving, using a screwdriver like a sensible person, then a hammer is only rarely needed.
This all smacks of gatekeeping to me.
I do, when I'm pairing with someone, or in any synchronous meeting environment. It may not effect the 'output' of delivered code (for a developer) but it can hinder their ability to pair effectively.
I agree that some (maybe lots of?) people feel differently about this and of course, that's okay too.
To be clear, I'm not saying anyone needs to type 120 WPM, I'm talking about a single finger on each hand, eyes down, hunting for the next key to press.
Over the 20+ years of my career, I've been exposed to hundreds, maybe thousands of different folks doing different kinds of computer work.
And yes, I agree that there are likely people who can kind of get the job done without being able to operate a keyboard efficiently. I also agree, that it helps a lot if they're nice to work with.
But I have not yet met a single hacker who has demonstrable and deep knowledge of computers and the software that runs them who cannot type.
I'm sure they must exist, and it wouldn't surprise me if Donald Knuth himself dropped into this thread to let me know that he hunts and pecks!
I'm just saying, that I have not personally met or observed someone operating at the highest levels of this field who cannot type with a reasonable speed.
[update] Almost forgot about Gatekeeping, Yes!
Gatekeeping is an essential part of interviewing. In fact, that's the entire point. We interview people for roles with the specific intent of bringing on people who we believe (or at least hope) have higher competence and closing the gate to people who have lower competence.
It's incredibly difficult work and it's not perfect and we have a lot of work to do, to make it better and more inclusive, but expecting competence in basic, role-relevant skills is unambiguously reasonable IMO.
I most enjoy working with people who care deeply about their work, and in my experience, those people are often (not always), people who also care about being efficient. In fact, if you're writing code at 120 wpm, maybe you've found a technique worth sharing and teaching! :-)
People with mild to moderate dyslexia (in my experience) and people with injuries are generally able to demonstrate competence with the keyboard and computer operation.
I have dealt with moderate to severe RSI, and I have worked with a relatively large number of people dealing with RSI over the years and in my experience injuries don't mask operational competence. These behaviors look different from, "where was that key again?"
Dyslexia often surfaces in a variety of early communications and over the years, working with some especially brilliant people challenged with dyslexia, I've learned to assume (early on) that typos are not a signal for carelessness, but potential indicators of dyslexia.
I'm aware dyslexia is a spectrum, and at the extreme it can be fully debilitating for some activities. I haven't had much exposure to that cohort, but I would be happy to make reasonable accommodations and certainly would not consider it disqualifying.
Thanks for bringing this up, I think it's helpful to be able to clarify these details!
In the modern workplace there is often quite a bit of communication via a text chat system. A chat where everyone types at 120 WPM is about as fast as the low end of normal conversation speed (120-150 WPM). A chat with several people only doing 40 WPM is much less pleasant.
Email doesn't have the quick back and forth of chat, so slow typing usually won't be directly noticeable to your colleagues, but it can take up a lot of your time.
Someone else already mentioned documentation.
I think the best analogue is blindness: it dramatically reduces the bandwidth of the information channel from the computer to you, but there are highly productive blind programmers. Hunt-and-peck programmers are in a similar situation.
I've been railing about this for years. Please stop calling it a REST endpoint. In 95% of the cases it is a custom HTTP endpoint and that's it.
I try to just call these things an "http API" when talking about it myself, just to mean an endpoint that is intended for consumption by other applications, not by humans, but I'm not gonna fight other people using language the way it works in practice, where words just mean whatever enough people agree they mean, regardless of what the very first user of the word meant.
Dojo’s approach seem to be one step beyond, but it requires even longer commitment to the employee and an asymmetrical market opportunity and economic chance. Definitely will help people and become a win win.
Web work is exploitative because... the workflow is undocumented? If it pays well, who cares?
The issue is that training/skilling juniors or fresh university graduates costs you money and slows you down (as you need to assign senior engineers/mentors to assist them). It is not unlikely that a newly joined junior employees only costs you money and decreases throughput/value for your company in the first 6-12 months, but you still pay them a (decent) salary. So once you got them skilled, you as a company would probably need to raise the salary to market/senior level to keep them - given the mass amount of recruiter approaches we in IT get. In that case, you could just start off hiring seniors only in the first place.
This is an issue I see everywhere, nobody wants to hire freshly graduated and everybody is only shopping around for seniors.
Seems like paying at least market rates once they are trained up would solve that problem. Once someone develops a skill, a company can't expect to keep them at intern or junior salary levels.
These days, that’s equivalent to saying “don’t ask questions about what the day to day job requires one to know”.
Sites like stackoverflow contain answers to problems that people encounter in their day to day jobs, and if you eliminate those as potential questions, then what should you ask? And what is the value in asking some obscure question, other than making the interviewer feel smart and special?
"I'm emitting 10TB of data a day with this schema {X} - it's produced in Avro. Currently we upload it into S3. But, instead, I want to stream into Apache Druid, but also want to make it available for ad-hoc analysis by BAs.
What approach would you take?"
Can't exactly google an answer to that.
The keywords there were "streaming" and "also ad-hoc analysis".
Answering those questions requires no knowledge of Avro or Druid, but rather, how to build streaming pipelines, and also make the data available for a query engine.
But that said, if you're in data engineering and are unaware of what Avro is, you're probably not at the level of knowledge I'd expect, and I'm happy to filter you out.
Likewise, if you're unfamiliar with S3, why are you even applying for a data engineering role?
I believe that "something else" usually indicates some previous data engineering work though, so it's an egg and chicken problem.
... you don't need to be in data engineering to use S3.
And can you be familiar with it? Read the docs, write some code.
BTW from my experience we are streaming a lot of data (similar to 10TB but definitely more) produced in JSON then converted to Avro using Spark and finally loaded into Vertica and the analysis pipeline. I have never used Druid but from Googling it is a column based data store which I guess is not too different from Vertica. Is there any specific difficulty you think would be encountered with Druid? Because the wording seems to indicate that Druid is not particularly good with ad-hoc analysis.
Bingo ;)
I was trying to phrase it like I've heard people do so in interviews.
And tbh, a candidate saying "I'm not familiar with Druid, but it sounds like you can't use it do ad-hoc queries with it? Can you tell me more?" would be a very good sign.
This is kind of thinking is why people hire all these leetcode jockeys who can't write working software, and won't stay beyond a few months anyway.
But for God's sake please Google before asking any general question. Some questions are literally within the first 5 results of the first page.
Maybe I shouldn't blame them though, sometimes I did that too :/
I agree that you don't need to know anything about von Neumann Architecture and the likes in order to be a Web Dev.
But I wouldn't want to hire someone who doesn't have basic knowledge about data structures and runtime complexity. It's just too easy to have something work fine in development, but choke in production due to accidentally quadratic behaviour.
At some points either teams converged to people without basic knowledge of data structures or they rushed with something that works (when N in O(N) was small) then scaled up. They get used to lower performance slowly like boiled frog.
Then someone is tasked with optimization, but instead of attacking data structure at hand, usually those people tried to optimize pieces of the puzzle for smaller gain, then they ended up with small hacks to shave some execution time, with unmaintainable code which lead to serious bugs.
While academic algorithm astronauts write papers which include Greek math symbols and can be proven correct, most people do... something else.
Optimising the something else would be a huge benefit to the industry. But no one even looks at it. It's just a thing that happens, and there's no research into fixing it or making it better.
edit: the features I refer to are stuff like foreign keys, cascades, dynamic fields, complex joins, etc. Once your stuff is spread out you loose some of these by definition and if you're still sticking to a single DB with massive stores you're not getting much humpf out of it anymore.
I 100% agree there are way too many fools that think their DB is big and try non rdbms too early and waste too much time. But I still think their lack of recent popularity is due to their own limitations.
But 99% of projects do not come even close to the scale where you need that.
There are use cases that aren’t best suited to an RDBMS at any scale, like managing a collection of denormalized documents, or a graph. But in my experience, considering these factors isn’t what’s motivating adoption of Mongo, or Dynamo. Your key lookups will always be fast (until you get a hot shard), but when you cram a highly relational set into a document db, eventually you’re forced to realize that there’s a lot of mutations and access patterns that you simply can’t implement, because you chose the wrong tool for the job.
Either that or you’ll decide that it’s a good idea to implement your own concurrency control mechanism on top of your document database. Which I’ve personally seen at least 3 people try and do.
- interviewer asks candidate to determine the complexity of some algorithms
- candidate is nervous, has a minor brain-fart and flubs a couple of them
- interviewer evaluates candidate as lacking basic CS knowledge you learn in 1st year of any CS course
Ok in this case we're talking about big-O, but it could really be anything.
By my experience, some companies need a competent body in the seat ASAP, and some are looking for world-class engineers.
Think Sears vs Amazon-- which company would know what gold was if they fell into a pile of it?
Also I don't think a company deserves to fail if they've got a bit too much faith in a slightly flawed recruitment process. The entire point of us sharing posts like this on here and discussing them is to learn and even help each other, isn't it?
I think O notation represents the upper bound, so a complexity of N can also be said as O(N^2) or O(N^3), but that isn't particularly useful.
In other words, if f(x)/g(x) < inf when x->inf, then f(x) = O(g(x)).
Then there's omega-notation (lower bound, f(x)/g(x) > 0) and theta-notation (both lower and upper bound, 0 < f(x)/g(x) < inf).
But if I give a code sample that has three nested for loops over some array and doing string comparisons, the candidate should at least have a vague idea that this might be problematic and could better be solved using sets or maps.
If you don’t know that, for example, debiting one account and crediting another needs to be done inside a transaction - or the converse, what you need to do if you don’t have access to transactions - then you shouldn’t be anywhere near a database.
Fundamentals are actually important. You don’t have to know everything, but you should know the fundamentals of the field you work in.
Such as "I" - what is it again? "Isolation"? "Transactions are independent". Well--yes.
...I think that was the entire point of their post.
> I guess we would be much happier if we dared to admit that one man’s fundamentals are another man’s trivia
And I would guess that many developers working with relational databases don't actually understand the more subtle differences between transaction isolation levels. I think asking backend developers to know the basics of transactions is reasonable. But e.g. knowing the exact difference between READ COMMITTED, REPEATABLE READ and SERIALIZABLE (and what your specific database actually implements there) might not be necessary in many cases. Though it certainly is good to be able to spot when you're about to enter territory where these differences might matter.
I agree that a backend developer working with databases should know what transaction are. But I don't really need the definition of ACID, and you can really push this question into details that are not relevant for all developers.
The point of jargon is to allow concise communication between experts. Knowing the jargon is part of the job.
On the other hand, if you are actually able to design for all those complex points you may still not have one short definition to quote available, which would not speak against you at all. Not having a short definition for ACID to quote may happen to both real experts as well as people who really don't know anything. Because for real experts that definition is too simplified to be of any use.
I saw similar issues a lot in school and in academia, when having to answer some test questions: When you actually know a lot more you may actually get many answers wrong, because the one asking used a much simplified model (and therefore more inaccurate than yours) as basis for creating their question. For example in biology, there are some key tenets, but when you go deeper you learn about all the additional quirks and exceptions. Using the simplified tenet an answer to a question "can this or that happen" would be never, but for someone who knows a lot more that answer would be wrong. As an expert, you no longer use that simplified model you learned early in school.
Example blog post showing the world is more complex than the simple definition: http://www.bailis.org/blog/when-is-acid-acid-rarely/
That was the point of the article, as shown by this example quote:
> How many similarly vague “fundamentals” we have floating around us? Plenty: RESTful, MVC, SOLID are the first things that come to my mind. If you start a discussion about the meaning behind those terms, you will find yourself in the middle of a food fight where everyone will have a slightly different opinion and no consensus will ever be reached.
All I have top do is read HN whenever such a topic comes up. It usually ends up with a lot of back and forth of people who are all experts but have a slightly different angle, and the initially clear definition falls apart when you read the discussions.
Well, in the absence of evidence in the contrary, they probably did.
Also, things may be fundamental, but not everything is built up from fundamentals. Maybe math is the best place to look at this. When you go to school, you just do it. Number lines, equations, calculus, matrices. Even through the engineering course I did, which had all sorts of advanced topics, nobody mentioned ZFC or Peano or Cantor or anything like that. And while it's true that if you dig under the stuff that you get taught there's a whole interesting world, it doesn't seem to stop anyone from using math.
With coding, a lot of it is having the right attitude. Something's broken, I'm going to try to understand why and what other people have done to fix similar issues. There's a resourcefulness that beats classroom teaching in terms of usefulness. You want that person who is capable of understanding everything that might happen, what he needs to do to fix it, etc. That person is never going to know or have heard of everything to start with, but they can deal with things when they happen.
But of course it's easier to test whether someone has heard of this or that thing. Perhaps this is why senior devs find it easier to get a job than junior ones, employers are more likely to think that they can work through issues.
My exact response after reading the first quote. It is indeed fundamental, and something I am mindful of with every database query I execute. But I haven’t thought about the exact definition and its mnemonic in over a decade.
Not being able to give any explanation or sign that you've at least heard of it is a completely different story from not being able to recall what the letter C stands for (consistency IIRC?).
As the saying goes: If you think you understand something, and you can't explain it to someone else, that you probably don't really understand it.
Drop P and you'll end up with both consistency and availability, right? Right?
("This is a trade-off we are ready to accept")
This "almost always" works.
Put quite many people say "you cannot have all three of Conistency, Available, Partition Tolerance", which sounds like you can keep any two / drop one at will.
But the right starting point is: "Partitions will happen. When they do, do you drop A or C?"
Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?
I myself had a similar fuckup with MongoDB a decade ago, because I used it "because it's fast", but failed to read the small letters, and wasn't aware that it achieves that speed by buffering the data for async save, and so has no guarantees it will actually be saved at all. So I lost a lot of data when traffic suddenly jumped, and since those were affiliate clicks lots of people got very pissed about not getting paid, and it almost ruined both my client's business as well as my own, as it was our core client.
Lack of understanding of (or even caring to know about) DB internals is also why like 90% of projects I've seen has wrong indices, or often no even a single index set at all. Because, hey, it's someone else's job to think about it, but in reality budgets are limited and team don't have a dedicated DBA or even a capable devops person to notice it, and it just ends up in the production like that.
So, in summary: no you don't need to know every in-and-out of every technology out there, no one can do that, but there's a valid requirement that one has to be familiar to those that they actually use - at least those that can seriously bite one's ass and cost a lot of money. And all DB related stuff is very much like that, that is probably the most critical part of the whole system, so no, it's definitely not a black box.
Same thing here, our legacy codebase avoided using transactions, and to this day, after migrating to a new codebase, we still get reports from clients here and there about errors which stem from inconsistent data
I suspect the author of this article would have no objections to probing if a candidate understands transactions and indices.
I mean, I learned ACID's definition, nodded because it made sense, realised that MySQL's default (at the time) MyISAM storage engine wasn't ACID, it flat out failed at C, so encouraged my LAMP stack using friends to switch to InnoDB, then switched to Postgres myself.
Likewise, I use a lot of distributed systems, but I don't think in terms of the CAP theorem on the regular.
Rather I just think about all the fun ways each distributed system can die in, and while ZK is CP (although I'm dubious about the P), and Kafka is CA, the failure mode is roughly the same - you lose enough cluster members and shit breaks.
I know that my k-safety (k being how many nodes I can lose and still have a system) for ZK is N - (N +1)/2 (assuming truncating int division), and as for Kafka, well... it really depends on the failure.
But the usual conservative design for a Kafka cluster is n * 3 replicas on a 3 AZ stretch cluster with min.insync.replicas set to num.replicas - 1,leading to a k safety of 1.
I'm a fan of the 2.5 stretch cluster, if I have to do a stretch cluster, I generally prefer separate clusters replicating.
So for a 2.5, you'd go 2 AZs with M brokers and N ZK nodes, where M % 2 == 0 && N % 2 == 0, with one ZK in the third AZ as the tiebreaker.
You can now lose an entire AZ, and still have a quorum, while paying 33% less inter-AZ traffic from your Kafka clients.
Basically, theory underpins all my work, but the theory and the reality are quite different things.
When I read that sentence, I gasped and thought the same thing.
I've spent a career writing thousands of lines of application code that would have never been needed if the database had been designed properly. But it wasn't because the deployer didn't think they needed "deep knowledge" that day.
We stand on the shoulders of giants who figured out the fundamentals just for us.
The worse the violation (different data types in the same column, ugh!), the more application code (10X) will be needed.
The more code written, the more more maintenance (another 10X) needed.
And none of it ever gets replaced, just migrated to something else years later (if we're still in business).
ASIDE: As for the interview question, I would never ask what "ACID" meant. I'm more interested in understanding than recall. I would just give the candidate a poorly designed database and ask what they would do to fix it.
If you are trying to hire, expecting every junior dev to know all this stuff means you'll skip a lot of candidates who can contribute a lot to your business.
They shouldn't be trusted with writing things from scratch. But if your codebase is mature enough, getting db connections, making endpoints atomic and stuff like that already should have a convention so your junior engineer wouldn't have to do it on their own.
And then there's the time that I rewrote a small database application, but couldn't really reduce the byzantine schema, as that's just how our business works. However, I did reduce the main query from 727 lines to 93, and made it run twice as fast. And then, of course, the project was canceled because of politics, and the group still struggles with their terrible application, 3 years later.
I should really write a book about terrible management in Fortune 250's.
I think there is a big difference between not knowing something off the top of your head and being completely unaware of it.
If you have awareness of something, you can make decisions based on that awareness without necessarily having a deep understanding. When you have no awareness is when the big issues arise.
For example, with your MongoDB fuckup if you had even a vague idea about the underlying behaviour you might have been able to consider the failure scenario ahead of time and prevent it from occurring. A deep understanding of how MongoDB works probably wasn't required to prevent this incident.
Same for indexes. If you're aware of them, you can trigger the thought process of "this query is pretty slow, maybe I should look into adding an index because I know that speeds things up". You don't necessarily have to understand how a B-tree works for an index to be effective.
The next level is being aware that the “I” doesn't quite apply in most databases with their default transaction settings, true isolation/serialisability being sacrificed for performance, though the distinction rarely matters is can be significant.
> You don't necessarily have to understand how a B-tree works for an index to be effective.
How a b-tree operates can be safely kept as a black-box thing, the important level of detail that many miss in my experience is how compound indexes can (and more importantly, can't) be used. Others include that in most databases defining a foreign key relationship does not create a supporting index by default, and that careless use of functions in joining & filtering clauses often blocks index use. If you have a DB specialist on the team there is probably no need for other devs to be intimately aware of even these details, but they should at least be aware enough of their existence than they know to ask the local expert (or look things up elsewhere) to check their working when the issues might affect what they are working on.
As a bit of a local DB specialist who has met some bad ones, I should point out that it is important that we, and other local “exports”, be approachable so the other devs feel they can run things by us for verification/improvement.
People only need to know about ACID if they are arguing about whether to use a database that doesn't implement it, or switching between databases some of which do and some of which don't provide strong ACID guarantees.
You may have been much better off, both performance and correctness, to relax MySQL's durability from flushing on every write, to flushing/committing every 1 second (using innodb_flush_log_at_trx_commit). Doing so improves performance by multiple orders of magnitude (it can be the order of 10-100x) to the point that your server is no longer sitting there queuing up queries for writes to complete. The database loses some guarantees about durability but the system as a whole gains better durability.
I could make similar arguments about the other properties as well, the point is that a competent person should actually know this stuff, understand it, not just memorize an acronym and boil the entire issue down to whether a database has ACID or not. It's not a binary matter, it's more of a continuum where each property can be tuned to various degrees. MySQL and Postgres both offer many options to tune the various ACID guarantees from having complete conformance to relaxing them with various upper bounds. Relaxing guarantees about a local property of the system can improve the guarantees of system as a whole.
On my current work database is riddled with indices over various status columns. Those indices are not selective and basically useless. Worse of all: it's not immediately obvious that those indices are bad, because inserts/updates are fast enough, but they slow system down as they accumulate and there's no easy way back.
In my experience the most reliable and long living projects treat the DB as its own separate product, whose interface to data mutation is via stored procs. The db can then be sanely designed with good data integrity rules (foreign keys, unique indexes and data column constraints).
Then external callers use the stored procs as their API, and then mostly don't have to worry about the internals. You can easily create a web based client, REST calls or a native GUI app that all work directly off the same db.
However they do have to worry about : -connection timeouts -query timeouts -deadlocks
But that's the same for all distributed systems.
I think in many cases, it's that even just having a vague awareness of something prompts you to double-check more often or to do something about it when you aren't really sure.
As an example, I find myself reading about the pros and cons of various PostgreSQL index types over and over again. It isn't really all that important for me to retain every detail about them for immediate recall but knowing that there's enough of a difference motivates me to read the documentation or run a quick web search before shooting myself in the foot.
I tend to think that testing people on this kind of deep knowledge isn't really helpful nor is it indicative of how developers really operate. What you need to test them on is what steps they would take when they aren't really sure and how they learn from it.
But… I’d like to say this is the kind of problem these quick-boot coding schools fall short on. Sure one can learn programming language X, but that doesn’t mean one has any sort of fundamentals in algorithms, data structures or general database technology.
This is going to require being conversant with the terms in that ACID acronym.
> A deep understanding of how MongoDB works probably wasn't required to prevent this incident.
That deep understanding is exactly what was required. If you don't understand how Mongo works, you don't understand the ways it can fail to do what you think it will do.
> If you're aware of them, you can trigger the thought process of "this query is pretty slow, maybe I should look into adding an index because I know that speeds things up".
So why not add an index for everything ahead of time, then every query will be fast, right?
Or maybe you need to have a sense of the cost of adding an index, then. Which may require knowing the different kinds of indexes and what the speed/cost tradeoffs are.
I think in all these cases, not every developer has to understand these things, but if you don't have anyone in your organization that understands them and is thinking about them, things can blow up for you at unexpected times that make your customers very unhappy. And then you might need to hire someone who understands them to fix the mess.
I don't remember what each letter of acronym means either. But if I was interviewee in this situation, I'd just start talking about transactions from user perspective -- how they either completeley pass or completely fail, and how even with power failure, you are not going to see partial data.
If I were interviewer, I'd accept this answer. No one cares about hearing exact words (0), "user will never see only a part of transaction" is good enough even if person never said the word "atomicity"
(0) According to some sources, Google runs scripted interviews where interviewer is non-technical, and grades answers against a checklist.
• the transaction is the minimal (atomic) unit, and must never be split up;
• all transactions (and everything else too) must map valid states to valid states;
• parallel database transactions should be viewed as a directed asynchronous graph over valid states (like Git)
• store stuff on the hard drive rather than RAM (i.e. the hard part of making databases; filesystems are horrible!)
I have fairly deep understanding of these things; I just didn't know the cool acronym mnemonic. (Database engine devs don't tend to use it in their blog posts about the internals of SQLite or Postgres.) “If they didn't learn how I learnt, they don't know anything” is one of my pet peeves.
Understanding the way your storage works is crucial, yes. But you ensure people understand it not by asking "Explain ACID to me" but by asking to design a certain system and observing them figuring out requirements and asking questions. You then ask follow up question about consistency and reliability, you probably even can ask about benefits of using transactions but I wouldn't because there's plenty of databases that can be used reliably that don't have transactions.
Life is more complex than ACID, that's it. Investigate if person understands life, not that they were able to memorise bunch of magic acronyms.
For the life of me I don't remember what SOLID is and I totally fail to parse any written explanation of liskov substitution principle but whenever I see code that violates those I immediately go "please don't do this, that'll cause problems in the future".
Off the top of my head, "all child classes should be usable interchangeably when dealing with an interface/base class". I guess that's too imprecise to qualify as "written explanation"?
Yet the GP says exactly the same thing as you said. He just ignored that the property belongs to the base class.
Given how some people study for these interviews, you might be better off going deep either way, to avoid getting rehearsed answers on these trivia questions.
Who has time for that? Interviewers design interview for themselves, not for the interviewees. It's basically a trivia game at this point of time. A prime example of "Everything is obvious once you know the answer" syndrome.
Often you're time-limited and can't do actual code testing, and acronyms and buzzwords can be useful to do the initial filtering quickly. Of course, not expecting for candidates to reproduce what each letter means, but to show they've heard of it and have some idea what it's about.
> For the life of me I don't remember what SOLID is and I totally fail to parse any written explanation of liskov substitution principle but whenever I see code that violates those I immediately go "please don't do this, that'll cause problems in the future".
In my book this is a perfectly valid answer to that question, actually far better than just citing the book definition blindly. Interviews are all about trying to read between the lines, and often the whole impression of competence that candidate leaves to you is far more important than the actual answers.
There's a difference between knowing the internal implementation, and the pros/cons of different usage patterns. What you're talking about is having mastery over an interface. This is distinct from knowing the internal implementation.
As someone who used to work as a hardware engineer, I can assure you that there's a tremendous amount of hardware internal-implementation-details that you're ignorant of. But that doesn't get in the way of you being you a great developer, as long as you spend some time understanding the interface and the recommended ways to use the hardware black-box. The same applies to what the OP is describing.
My statement is that it was a terrible example, but the core principle stands. Yes, we should all definitely understand how the storage layer works (and anything else that could corrupt data) but no one needs to have the laundry list of acronyms memorized.
I think the article also touches on the fact that when you look up the definition of ACID you get a rough description of the acronym and then a description of how it is mostly a marketing term and all the ways real systems can deviate from it. So really the deep knowledge is not so much what ACID means as what it doesn’t.
The article is basically complaining that knowing what ACID stands for is not really a fundamental and that knowing how to use a database is. So I don’t think you really disagree with the OP much either.
If you’re implementing interactions with a fairly new database (let’s face it. 10 years ago MongoDB was basically experimental) then it is indeed incumbent on you to understand in greater details what you’re doing.
But if you’re building a CRUD web app, that’s maybe only used internally by your company and backed by SQL Server you probably have to never give it a thought.
Either way, lack of knowledge of a particular acronym, one which is described as basically meaningless in texts that describe it, is not a deal breaker of any sort. And it doesn’t apply just to ACID but most acronyms in tech today, such as REST/SOLID, etc like the author points out.
Until you get complaints that the hour-long reconciliation transaction you run in the morning is locking out all the users, or there's an API a customer hits that's erroring out 1% of the time because Sybcoughuh, Wonderful Microsoft SQL Server has detected a deadlock, and this is causing us to lose customer orders, or...
No, you really need to know what ACID means, and what tradeoffs are made to balance ACID with other considerations in your chosen database implementation. Lumping ACID in with vague heuristics like SOLID on the basis that they're both acronyms is just plain ignorance.
Sure, you're right, but I'd argue that transactions are is hardly "database internals". Transactions are day 2 of "using a database 101".
You don't have to know what ACID means or how transactions are implemented to use them correctly.
I'd rather hire a candidate that knows how to use transactions but has no idea what ACID is than one who knows what ACID means and how it's implemented but doesn't know when or why they should use transactions in real life. (Of course the second case is rare, but this is just an example).
> you don't know what you don't know
aka knowledge is what you have after you forgot everything you learned.
> But, that’s only because you are a noob and you don’t deal with the things that I have deal with.
> Someone on the internet
In conclusion: not all software problems require the same implementation.
I don’t run a DB for some deterministic applications I have that can regenerate the same outputs such I don’t need to store it all. I could, in theory, regenerate an entire data set with a seed. Maybe app developers are ignorant and lazy for not designing systems that do that!
What does your one story have to do with “software engineering” as a field?
Sounds like people are conflating problem solving and personal career demands.
Understanding why we should use database transactions is not the same, to my mind, as understanding database internals. When the author said they can treat the database as a "black box", my thinking was the they didn't necessarily need knowledge of the database's implementation. People can understand how to effectively work with a database in the general sense (i.e. use transactions) without a deep knowledge of every implementation.
When we start talking about MongoDB, I think that's really interesting because it is so different from our traditional, SQL-first databases like PostgreSQL, Oracle, etc. Our understanding of SQL-first databases isn't all that useful when we try to apply it to MongoDB.
At that point I think we are in agreement: it's unreasonable to think we can treat a MongoDB database as a black box if our base knowledge is only SQL and SQL-first databases. We probably do need someone who has already dug in and dealt with the implementation, how MongoDB really gets things done.
They invested incredible amount of time in learning technology and scaling their application but forgot about need to learn fundamentals -- data structures and efficiency.
Fast forward 1,5 years, the application as I left it ran on a single server using about 10% of its capacity. A second server is just a hot standby backup.
The service went literally from being able to process tens of transactions to a hundred thousand transactions per second on a single node.
More than that, we threw away most of the exotic technology that was used there -- greatly improving team productivity. The implementation is simpler than ever with layers upon layers of microservices replaced with regular method calls and a lot of infrastructure basically removed without a need to replace it with anything.
People who do not learn from history (fundamentals) are doomed to repeat the same mistakes.
I still remember some interviews within the last decade where single-core machines with a few GB of RAM were a default assumption for whiteboard designs, or that spinning disks were the default.
But in general I think that the problem isn't that people are not "updating" their understanding. Even 10 years ago it wasn't a huge problem getting hundreds of thousands of transactions per second on a single modest machine.
The problem rather is people relying on more and more layers of abstractions for vary small gains.
Example: I get that Python is a nice language (for somebody that does not know Lisp). But is it worth it to choose Python for a little bit improvement in productivity for a problem that requires a lot of throughput, to then suffer performance issues, to then spend many times more effort on trying to improve performance? I don't think so.
Or more in my space: is Spring Data (Java) worth the very incremental productivity improvements if it completely destroys your application performance? The application I described in my parent post used Spring Data MongoDB which kinda means it was fetching entities one by one which is extremely costly.
By replacing it with bare MongoDB reactive driver and ensuring data is being streamed in large batches (why read one user data if you can read 10k at a time) and getting rid of costly aggregations in favour of application side processing we have improved throughput by many orders of magnitude WHILE reducing load on MongoDB.
Granted, there is a little bit of additional complexity (on the order of 10% more of application code) but just the performance improvements mean that the team can breathe and focus on other problems like modelling the domain correctly.
I would consider transactions part of the api. I use them all the time in postgres and I'd hardly call myself an expert in the low level details
The author's casual arrogance about "pixel pushers" only caring about "hotdog stand colors" is just as damaging to the profession -- as it infects the hardest part of getting things right most of the time: teamwork.
I care far, far more about data-structures, eventual-consistency, data-design, and types than CSS and divs.
These days getting the design to look nice is my two-hour reward for nine days of data-stitching hell, writing backend-for-frontend lambda functions, and negotiating API contracts that actually support the product I've been asked to build.
We need to stop assuming that just because a programmer works on client code, that they are in any way less capable that a programmer that works on server code. My point stands: It's absolutely toxic to collaboration, which is the key to success.
Nobody talked about implementing a specific variant of a red-black tree when the general topic of self-balancing binary search tree hasn't even be mentioned.
So depending on the actual exercise/interview question you might able to just show that, especially of you have to use pseudo code.
The self-balancing part is usually added after the search part as balance is trivial/unimportant when the order/position of the elements has no constraints.
I imagine there's quite a lot of them. Things like VP-trees and such get often built statically and could be implemented in a similar way. Not sure if the simple (2i, 2i+1) schema is optimal for this but perhaps it could be adapted.
That said, for most dev jobs implementing a binary tree isn't necessary knowledge. But if I need a dev who can get their hands dirty I would be weary of hiring someone who can't take a decent stab at it during an interview.
If you audition for the New York City ballet, they don’t expect you to ice skate.
Why? The only time I knowingly used them is with database indices. For everything else I do, they are an unnecessary implementation detail. It doesn't matter what Ruby's Set uses.
If you really can't, you should get familiar with how data can point to other data, it is fundamental and done all the time.
If the current node is a leaf, insert left if smaller and right if bigger.
If not a leaf nods, run the method recursively on the left it it's smaller, on the right if it's bigger.
If they asked you to write a self-balancing binary tree it's one thing, but there's nothing tricky about writing a normal ordered binary tree.
It's easy to make an imbalanced binary tree.
I once decided that building a binary tree based only on a friend's explanation of how they worked; it seemed like it would be a good learning exercise for myself (I don't have a CS degree). It was like 10 minutes of coding to get it minimally working & tested, but then I had to work out all of the different rotations to make it a balanced binary tree (I don't know that an imbalanced binary tree is ever actually useful, as it could just work out to be a slower linked-list in the worst case scenario).
I did it once, took me a little while, but I don't know that I could do it again without having to reexamine all of the different cases that result in imbalance and the rotations that respond to each.
For example I think writing some logic to filter datapoints or timespans is much better and more practical. But it takes more effort than just opening and pocking binary tree.
I wouldn't want to have an interview with you, or you selecting people for a team I'm on.
People in general do not communicate on the cleanest ambiguosity-free way;
I’ve interviewed a _lot_ of people. The people who gripe the loudest about algorithms haven’t had a chance just from answers to situational questions.
Coding binary tree is not the way to discover that.
Presenting a task and observing people solving it and explaining computation complexity behind their solution and the thinking behind their solution is how you do that.
It's a simple data structures / algorithms question. Can you think in terms of how data is stored in memory? Do you understand the tradeoffs of storing data in one form or another? Can you work with recursive algorithms? Given a challenge of having to think about data in a form you haven't recently, can you wrap your mind around the characteristics of that data structure?
If you're getting asked all this for a ReactJS UI building job, then the interviewer is probably off-base. But otherwise, make sure you're applying for jobs you're suited for. If you don't want a job where coming up with efficient algorithms is important, that's fine - there's plenty of others out there.
If you're using a modern OS, you don't know this. The kernel remaps everything and pretends to give you a linear chunk of memory. You don't know what's in the various caches and what's been paged out of ram completely.
Everything is built on layers of abstraction. C is an abstraction layer that fits poorly into what modern CPUs and operating systems actually do, but everything is programmed to cater to C's assumptions so people don't need to rewrite their programs.
Higher languages don't necessarily use the data structure you think they do and they can switch the implementation details based on the size.
"Simple data structures" are only simple because you pick one specific layer of abstraction to work with.
If I ask if you can do something basic binary tree things, I will also not know if you are brilliant, but I can at least rule out if you are totally clueless or not.
Or you have a candidate who understands binary trees, can implement it but can't regurgitate binary tree operations from the top of their memory.
If they can't recall how balancing is done that would be fine with me, as long as they mention it.
I know dozens of good candidates who can't get there pretty quickly but are really good. I also know dozens of good candidates who can get there pretty quickly but won't be good enough on the job.
You not hiring for backend positions doesn't mean they are not good at the positions. It just means they aren't good at your signals.
Some of my coworkers are not nearly as strong as me when it comes to "CS details" like data structures or programming languages, but they have other attributes which complement me and which are good for the team.
Having a bunch of me's (or any of the others) would be a disaster.
I implore you get to a state where this isn't hard for you, it will make you capable of more.
This is a really important point that we don’t talk about enough, because everyone gets uncomfortable and feels judged. But I can’t help myself, and I think there’s a lot more to dissect here.
I think programming is slowly bifurcating. We’re growing into two disciplines.
One discipline requires deep systems knowledge. A CS degree is recommended and if you can’t reverse a binary tree on a white board then you won’t be able to read our code base. Developers in this camp work on rendering engines, Vulcan drivers and schedulers. They spend lunch telling stories about that one time we wrote a game engine in J. Or arguing about type systems.
The second group makes most of the software (measured by lines of code). And almost all the software humans actually interact with. They spend their time working around weird safari quirks, arguing about electron, and enjoying the sense of ease and mastery that comes from staying in a single ecosystem for a decade. Getting better at their job looks like getting better at talking to users and customers, making on point estimates and making sure the team releases on time.
The part that gets all of us in trouble is that the first skill set is (arguably) harder - and thus much more prestigious. Everyone wants to be seen as an electrical engineer, not an electrician. So people write “software engineer” on their resumes. And then cry foul when they get assessed as a member of that first group. “I’ve never needed to know about red black trees in my career. It’s stupid for anyone to ask about it in an interview. Everyone just crams leetcode. JQuery isn’t flashy but it works and I can solve real problems with it. Hire me you fools, and I’ll show you!”. They’re sort of right. We’re just collectively confused about what sort of job they’re being hired for.
I think this confusion hurts everyone. I once wore my fancy engineering hat at a normal consulting gig. I coded up a complex solution to a tricky problem they had and called it a day. After I left, nobody on the team could understand or maintain some of the code I wrote. That caused huge problems for my client after I left. I think they might have regretted bringing me on board at all.
But we can’t even talk about this because we don’t have the language. We treat application developers like they’re just junior, in training versions of deep CS systems engineers. But practicing building kick-arse iOS apps doesn’t make you magically good at data structures. And vice versa - doing performance work doesn’t make you good at working with clients. (Ask me how I know). The majority of professional programmers do plumbing / user facing software. We need to destigmatise this stuff so people can be hired explicitly for these roles, based on the right set of skills.
Uh, what? Most humans find social skills much easier to learn than crunchy engineering skills. Good social skills are way more common in society than, say, deep knowledge of algorithms and data structures.
But it’s not a competition. As you say, programming is rarely a solo activity. Our projects can fail due to a lack of teamwork. Or a lack of technical skill. Or a lack of empathy with our users. Or a lack of leadership. Or a dozen other things.
We need (and need to value, depending on the project) all of this stuff.
These are disparate skillsets and both of them are difficult to master. Claiming one is harder than the other is really quite shallow. Some people are going to naturally have an easier time picking up one skillset while other will have a naturally easier time picking up the other skillset. I think rather than trying to pretend like people should stop focusing on hard skills and start focusing on soft skills, we could realise for a moment that it does nobody any good to take people who are good at one task and insist that they must retrain for another when there are already plenty of people who are good at the other task.
It would make more sense to get these groups of people to learn just enough of the opposite person's skillset so that they can effectively communicate and as a result form a well functioning two-sided system where one group of people talks to users and analyses their requirements and the other group of people actually works off of these more concrete better defined requirements to actually implement the software.
If you think that "soft skills" are harder than "hard skills" it's probably because they're harder for you, that doesn't mean they're harder for everyone, and instead of pretending like this is some kind of major flaw in your own professional development, you could treat it as a gap which could be filled by people who have the opposite view of difficulty.
It has always been like this. The distinction used to be: base software developer vs applications developer.
Base software developers went to work on operating systems, databases, web servers, etc.
Application developers worked on business software.
And most decent business software development shops kept a couple of devs with decent knowledge of base development just to go for when the needs got technical.
So asking about ACID is a fine thing, but expecting a text book answer is silly.
Edit: ivanhoe said it better in their post and I changed my phrasing to be less inflammatory. (My point of different hiring mind sets on different continents is not the point.)
If you think you have a better way, tell us about it without passing judgement on others. Tell us why it works, how great it is, give us some qualitative and quantitative data. Most of all, enjoy your competitive advantage. But for god's sake, quit telling other people they are doing it wrong.
School exams are supposed to be fair, in the sense that they only cover the topics covered in class, and only to the level of detail and difficulty covered in the class. They test for weaknesses relative to that standard. When a school exam asks a question, it's because you're supposed to know the answer.
If someone takes the school exam definition of fairness into job interviews, they are going to feel terrified and aggrieved. Interviewers talk to candidates from vastly diverse backgrounds. Other candidates didn't go to your school, maybe didn't go to school in the same country as you, maybe didn't go to college at all. There is no fair, agreed-upon set of standard knowledge. If interviewers restrict themselves to "standard" topics, they will undervalue a lot of people who have unique backgrounds and experience. Not only that, but job interviews can't just measure a candidate's weaknesses against a set standard. It isn't even really a matter of "strengths" and "weaknesses" but wanting to discover a candidate's full capabilities. You don't want to miss a candidate's depth of knowledge in an area simply because you failed to ask any harder questions that would have revealed it. When an interviewer asks a surprising question, it's often not because you're supposed to know but because you might know and they don't want to undervalue you because they didn't ask.
Yeah, maybe this is all fields, which is a shame.
Even more so when those "fundamentals" are implementation details you don't think about much in your day to day job. Actively recalling knowledge that has become part of your intuition and understanding of the world is hard.
I know enough about many things to do basic implementations safely; everything else is paged out to google. The finer details of database transations? Usually not my job, but I used to know more about them and could relearn that information quickly if needed.
Yes and no. For years I did little computer programming projects on my own. Built some neat things, for my purposes at least. But because I didn't have the words to describe things I wasn't able to build a concrete image in my head. When I finally took a CS course and had the words to describe concepts I'd used for so long it was like a switch had been flipped, allowing me to think about problems in a much better way. I knew what many things were but because I didn't have the concepts mapped to concrete terms they remained fuzzy.
> So I had learned already that names don't constitute knowledge... That's caused me a certain trouble since because I refuse to learn the name of anything... What [my father] forgot to tell me was that the knowing the names of things is useful if you wanna talk to somebody else.
I think an interview might count :)
[1] https://www.youtube.com/watch?v=ga_7j72CVlc
Names also come handy when you need to search for something and search engines are not powerful at taking your description of what you're searching for.
[1] https://en.wikipedia.org/wiki/Discourse_community
This change in me was triggered when a candidate came in for a dev interview and we were doing the usual whiteboard problem thing back then. Someone on the panel asked "what would you do if you faced this problem on the job" and he said "I'd google for the solution". I recall not being able to make up my mind about whether to hire him right there or send him away :) .. but that was something I hadn't realized back then so the issue was not with him but with me. The lesson stayed.
So .. on the topic, if the candidate didn't know ACID .. an opportunity to google followed by a detailed discussion on it would work for me.
But interviewing in general is tough (I'd go so far as to say it's a shitshow) and there's lies on both ends -- companies and managers and folks on the inside just want the best of the best -- and candidates just want a chance to get in the door -- the incentive to embellish is too high.
1. The interviewers aren't subject matter experts themselves but are hiring managers, so they've taken their questions from someone else.
2. The company gets far more good candidates apply than they can hire, so the process is less about removing bad candidates and more about having some arbitrary way of whittling down good candidates.
3. The interviewer is trying to prove how clever they are or the company is. Sometimes for their own ego but more often because they think it might make the company more attractive to work at.
If there is a theoretical question I'd normally Google / DDG instead of memorise then I'd usually be honest in the interview and say "it's something like xyz but in truth this is something I double check these days as, pragmatically speaking, it is not a problem that comes up frequently. However it is something I have done before [cite example]".
If the questions is asked because of reason 1 or 3 from above, then usually they're happy with that answer. If they ask it because of 2, well as much as I might disagree with their interview process, I'm still not the candidate they're looking for. So we still part of good terms.
There were no other questions - no white board stuff, no algorithm questions, no stupid questions like "where do you see yourself in 5 years?" etc. I still remember the interview nearly a decade later.
Obviously this won't work for all jobs, but this will work surprisingly well for a large number of jobs, because most jobs are ordinary. One doesn't need to know complex algorithms to do these
I might, too, but I frankly might do the same to an even more extreme degree for a particularly challenging real-time 1-on-1 interview question from someone staring at me.
All things considered, I think I'd prefer being left alone in a room where I don't feel them over my shoulder while constantly wondering what they might be thinking with each passing second. I can take a few minutes to relax a little, look over the code, and try to figure it out.
I have anxiety issues when an interviewer asks me some obscure syntax option for grep command or some b-tree stuff that I learned in college decade ago (and never used in real life).
What can you do to understand why this happens?
But I don't understand because I am not you. Different things give different people anxiety. Anxiety isn't something you reason your way into or out of.
They tested it after I left (a >100x improvement apparently) and called me the next day to ask when I could start.
4. Interviewer did not have time set aside to prepare for the interview, and sort of has to wing it. I was surprised to realize how often this happens.
The I (Isolation) part is the one you'll need to understand whenever you scale up to a point where concurrency starts to become non trivial in your product, mainly because there are multiple levels of isolation, which each provides different guaranties, and you'll need to understand what level your database uses and what it means for your application.
As for the overall point of the article, it's just a question of scale. The more you scale the more you need to cut through the abstraction to understand what the hell is going on. Hire accordingly.