That's only if the challenge is completely new. Sure, if you're researching something novel in an area you've not worked on, the information you have is likely very far from complete. But if you're writing a backend server for a new POC for the umpteenth time in your career, you're likely to be able to do it fast, well, and be able to estimate how long it's gonna take. The reason being that the unknowns are slim to none.
This reminds me of a talk of Linus Torvalds. From this perspective, mastery can be seen as an acquired taste that came after doing a large volume of work.
Pai Mei mustache, sits on a rock meditating and dispensing advice to juniors, quotes Martin Fowler by heart, seems to never touch a keyboard but every commit will replace entire module(s) flawlessly, demonstrating high level architectural knowledge that lies on the far side of the organisation's learning curve.
Every module Pai Mei implements fits on a single page of his 80-column terminal, including comments. He manages this brevity not with obscure one-liners, but through algorithmic simplicity. Every module is precisely as complex as it needs to be to accomplish its purpose, no more, no less.
He never said the commit will be big. He just said the commit will "replace entire module(s) flawlessly" which you seem to acknowledge can be sometimes done with as little as a single character.
Knowing that the correct answer to almost every question is "it depends".
Knowing how to somewhat-less-inefficiently produce larger-scale software that adequately meets the need. (Note that I did not say "efficiently", and I did not say "bug-free".) This takes at least some skill in dealing with people, not just computers.
I'm somewhat doubtful that 'mastering' Software Engineering is possible.
You wouldn't ask what it looks like to master the practice of medicine today because there are dozens or hundreds of specialities and roles all of which contribute to medicine.
Firmware engineering and 'agile' website development are very different practices with different assumptions, requirements, etc. I wouldn't expect someone to master both web development and video encoder development just like I wouldn't expect a doctor to be a master pancreatic surgeon and develop vaccines.
On the other hand there is a 'journeyman' level of software development and engineering skill that carries over between different development areas. It's definitely not algorithmic interviews though.
> You wouldn't ask what it looks like to master the practice of medicine today because there are dozens or hundreds of specialities and roles all of which contribute to medicine.
This is like saying you can't have a favorite type of rectangle because some rectangles are squares. If you're a master of neurosurgery, a reasonable person would also consider that to be a master of medicine. Just as I would expect a master neurosurgeon to not be a master dermatologist but be able to master dermatology much faster than I could as a non-medical person, I would similarly expect a master C++ developer to master Javascript frontend development much faster than a junior dev, let alone a non-software person.
He is one of 3 people that came to mind immediately reading this post. One of his latest creations, QuickJS, is quite remarkable.
The other two I thought of are Peter Norton and Linus Torvalds.
Fabrice, despite his tremendous contributions to the field, doesn't reach for acclaim and just seems to go his own way.
Peter Norton was a genius in his field. His book on Assembly Language was one of my favorites as a youth. Norton tools were impressive for their time.
Linus, as the namesake for Linux also made a huge impact on the field... and comes off as somewhat of an uncontrolled lunatic.
I don't though think that these three people are the highest level masters in the field. I think they are just the ones who have chosen to do things that have high impact and visibility.
>Linus, as the namesake for Linux also made a huge impact on the field... and comes off as somewhat of an uncontrolled lunatic.
What? Linus is one of the most Rational, Direct, No-Nonsense, Get Things Done Leader there is. Whenever i hear some self-appointed "expert" pontificate on Leadership/Management/etc. i always think of what Linus would say/do in that situation. People only think of him as a "Technical Guy" but his Leadership/Management of something so vital, so huge, so distributed, with so many "employees" and with so few resources, so many stakeholders is what is amazing to me. Nobody comes close.
You really need to take back the "uncontrolled lunatic" phrase.
As an "uncontrolled lunatic" myself, they are rather obvious in the field.
I'll agree that many of the things he says are reasonable. An example is his recent defense of long lines and dismissal of the insane "80 characters per line" peeps.
What I mean specifically by "uncontrolled lunatic" is that he often ignores convention and current opinion and does and says things that get him into trouble. He also has a temper that he often is bad at controlling. I myself have this issue, and I don't see how one could say he does not have issues in this regard.
While it requires some amount of perpendicular thinking to be a master of software, having such an attitude and working with other people in that way will cause too much conflict that is unnecessary.
I think what you are missing in my statement is that I consider calling someone an "uncontrolled lunatic" as a sort of compliment.
I agree also that he is less of a technical person and more of a manager. His skills lie in herding.
On the other hand; Git is a flaming piece of garbage and he is directly responsible for its popularity... so... He did though reasonably tell people he didn't intend for anyone outside of kernel developers to use it. I just wish he corrected its deficiencies. Maybe someday he can apply his "arbitrary character restrictions are silly" logic to Git comments and become less contradictory.
I have heard these arguments before. You cannot separate Linus' so-called "shortcomings" from how they were instrumental in getting him to what he has delivered and where he is today.
Genius cannot be constrained by every single artificial social more and etiquette; we are Humans not Automatons. As long as they do not lead to destruction of product/project/self/others it is merely a character trait which makes the person unique. All successful companies know this and "manage" their most important people accordingly.
I disagree. The notion that "you have to accept the arrogance and obnoxious attitude of high level software engineers because it is inevitable that the type of people who excel at creating software are that way because they don't focus on social skills". Every one of us can work to be more polite and caring of the emotions of others. Especially if you know you suck at it you ought to use effort to be better in that way.
Justifying treating others poorly just enables more people being hurt. We should not reward people for this sort of behavior. Turning a blind eye to it because "they are great software engineers" is not right.
Your comment treads very much into the "for the greater good" mentality where you can justify anything based on the results.
"Unique character trait" my ass. Obnoxious aholes are just that. Obnoxious aholes. They need to fucking stop and be more kind.
You are now reading things which were simply not said nor implied.
It is also quite ironic that while you talk about "being nice/kind" your posts are the ones containing inflammatory phrases like "uncontrolled lunatic", "arrogance and obnoxious attitude", "my ass" and "obnoxious aholes" !
Coming back specifically to Linus, he has never been any of the above. He has been direct and harsh when needed (there is a world of difference between this and the phrases you have used), otherwise he has shown a lot of restraint. For example, he has said that due to the nature of email communication, one has to be direct since a lot of non-verbal and personal nuances cannot be conveyed which is exactly right.
I am sure we can all recollect email chains/discussions/meetings where nothing was ever achieved because people did not want to offend somebody with thin skin in spite of the fact that they were squarely to blame. Being direct and calling them out is not being obnoxious/arrogant but simply moving things along towards a goal. To be nice/kind does not mean everybody has to be mollycoddled and tip-toed around due not wishing to "hurt their feelings". Engineering is a hard science and while we have to follow social mores and etiquettes this should be regulated (i.e. we should know when to break them) so that progress towards the end goal is not derailed.
There is a big difference between what you communicate broadly as a public figure and random comments under a throwaway.
Linus has said many attacking things publicly under a position of authority.
What I say here bears no weight because if what I say is disliked I am simply downvoted and dismissed.
I disagree that being offensive solves anything. I am fine ( as you've pointed out ) with being extremely offensive and attacking the shit out of anyone I see fit, but I don't think it is particularly effective or a quality trait.
Anytime you say things in an offensive way, you are intentionally being hurtful. This is not a good thing, and I don't think is justifiable.
My doing it in no way makes it right, nor invalidates my judgement of those who make similar statements to a broad audience.
Consider the effect of me ranting here. No one is going to go: "Wow I'd better pay attention to what permille42 says he is an expert. He must be attacking people for a good reason."
Nope. Anytime I rant most readers will just immediately dismiss everything I said, mostly because my tone tends to be so aggressive. I don't think such people are wrong to dismiss as such aggressive statements do tend to be from people who are offbase and don't know what they are talking about...
- Should know many tools to the level where they know their strengths and weaknesses. This allows them to select a useful collection of them for any new piece of work.
- Should produce incremental project changes which all either improve the value or reduce the cost.
- Should balance all the improvements to produce useful, working software with a reasonable cost. For example, producing super-secure software with no useful features, featureful software with a bajillion bugs, or bug-free software with a gigantic cost are all bad.
IMO it's about understanding the solution curve in the problem space. Every problem type has a range of possible solutions. Most of these solutions are wrong and a few are correct with different tradeoffs. When you can't do better without making tradeoffs, you're on the curve.
Novice engineers will struggle to get something done. Intermediate engineers can get things done but fail to see the tradeoffs. Expert engineers can identify possible solutions with different approaches.
How do you learn that solution curve and their tradeoffs? Swap jobs and hope you see them all in your lifetime? Read articles and hope you pick everything up correctly?
You can never truly be perfect, you can only approach it. Understanding your limitations is just as important as understanding your capabilities. Depth vs breadth is hard and you only have so much life to go so deep into so many domains.
I once was trying to design a validation framework. I thought it should work this way. The other lead thought it should work that way. So I proposed we write some sample code for these APIs and show it to the team. TDD with pseudocode before any of us knew what TDD was.
Only, I didn’t like having two to chose from. Something told me three would be better. So we paired, I wrote mine, then hers, then just made another one up on the spot. Once we agreed they were complete and implementable, I shopped them to the team.
About 2/3rds preferred the made up one. Including me. So that’s what I implemented.
Definitely this. Nothing like learning from watching your solution hitting roadblocks (performance, extensibility etc) and reassessing your earlier approaches and again coming with a solution to address the newer challenges
You can also dig into the problems that existed when you got to the project, and try to work out how they came to be. Project forensics is a skill set unto itself. And listen to and help people on other projects at your company. You can see how their story arc goes and where it surprises you.
An intelligent man learns from his own mistakes. A wise man learns from the mistakes of others.
OP is giving a problem solving framework, it's not tied to specific solutions or tradeoffs. It's more of a method for thinking through a solution.
In more detail, the steps go like this.
Find as many solutions to a given problem as you can. Bad engineers run with the first solution that comes to mind, letting confirmation bias drive them.
Evaluate each solution for its costs and benefits. Imagine two steps in the future when the solution is implemented. What pains are there?
Search for creative new solutions that create win-win scenarios. That's riding the solution curve.
Given all viable scenarios, compare the costs and benefits against the quality measures for your specific context. Some projects value speed over precision. Some projects value performance over extensibility. Some solutions are easier to change later than others. This is choosing a specific point on the solution curve that best fits your context.
You can apply that method of problem solving to any problem, large or small. You don't need a ton of experience to practice it.
This comment made me realize what my mentor did years ago. He would have the entire team list all possible solutions to a problem, including the obviously bad ones, then have us whittle down the list based on pros and cons of each until we reached consensus on what to do. It was a teaching exercise and I didn’t realize it.
I’ve repeated it that exercise with junior engineers to great effect. Some catch on over time and start intuitively considering the trade offs of a few reasonable solutions to a problem; some don’t.
I never reflected on what he was doing there; thanks.
To add to that, some of the smartest "good" ideas come from "bad" ideas that people rejected out of hand, sometimes from unusual sources. It pays to do brainstorming thoroughly.
The best software engineers know when to solve a problem without using any code at all, like the classic "just do it manually" for a complicated task that is worth more than $x per task.
- Identify and document possible solutions. Think hard about why each solution is good and each decision is bad.
- Within each solution, attempt to find common patterns in that problemspace (when working with Sharded databases, this approach brings these results, etc)
- Try to match your personal and organization requirements against the various patterns you find and the pros/cons you defined previously
During implementation
- Keep a running list of things that seem weird, or things that seem great, or things that turned out to be untrue
- Don't stop implementing, but for each thing you found that wasn't great in step 1, try to find a better way to approach that thing (while moving forward)
Post implementation
- Compile all the notes you made in pre/during into a manageable list of goods/bads.
- Use this to drive A.) iterations of your solution, and B.) future solutions
If you do this enough times, you'll have a pretty decent list of your experiences in a space, and you'll start to notice patterns as you try different things and become exposed to problems, their solutions and their tradeoffs. Eventually you won't even have to look at your previous notes very often, as you'll have built up a pretty decent amount of experience in various problemspaces such that you can predict what goes well/doesn't go well.
Also worth noting, this type of behavior doesn't really have to be applied to software, but will also gain you big points with future employers when interviewing. You'd be surprised to find how many candidates never stopped to reflect on the work they'd done, why it was sub-optimal, or how it could be corrected moving forward. Response like "We did it ____ way because that's how we always did it." Which, from a progress standpoint, is basically a none-answer.
Note: this is obviously just my opinion, and I'm no expert in "Becoming a Master of Stuff", but these are the methods that I have used and seen my peers use (in one form or another) over quite a long time. Sometimes not as obvious (doing these things in your head vs. writing them down), but the shape is always pretty similar. Ultimately I think being reflective is one of the best skills a person can possess (with respect to employment and I guess also relationships). Acting without thinking is reckless, I think.
Additionally, expert engineers can better define the problem space to begin with. Some engineers can get things done and evaluate tradeoffs well, but end up solving the wrong problem.
Expert engineers have the ability to think about a problem in a broader context than the engineering problem at hand.
You're right, except you sell your own answer short when you say "curve". In practice, the set of trade-offs has a much higher dimensionality, and solutions are much less often strictly "wrong" as they are "less optimal" or "out of scope".
Now, you might be thinking "that's not true, if I implement a function to add two numbers, and it returns 1+1=3, that's wrong. So that's a clearly wrong solution." But I argue: is it wrong? What's your context? What are your acceptable trade-offs? In many real-world applications, precise correctness isn't always critical. Sometimes, sacrificing accuracy might be acceptable, and may save on cost and/or complexity. Other times, that's not acceptable. But it's worth considering even these types of non-obvious trade-offs, not just the obvious ones. That's what an expert engineer does: ask the right questions and identify the right trade-offs, and they've become efficient in that process from experience.
An expert sees a problem space as a wide range of trade-off dimensions, including:
I don't think there is mastery in this work. I've been doing this for decades, and I still think my code from 6 months ago is crap, no matter how much I learn and improve.
Mastery would imply that you can do it all, and do it right, on any project, and there is just too much change and evolution for that to ever be true.
That's the right mindset for an individual but not particularly helpful for an answer. Constant improvement is part of mastery but not a great yard stick.
I also have trouble believing you've been coding for so long and can't take pride in any of the code you've previously written.
I think most "master" software engineers are a bit more idealistic than that... Not saying I am a master but I occasionally write something and consider it genuinely beautiful.
Length of time doing something doesn't imply mastery. It can contribute to mastery, but only if you're always pushing your skill to the next level I think.
For example: I know people who have been skateboarding for decades and are just barely OK at it. How you practice matters.
I think there is mastery, but only in knowing that there is no mastery.
"I seem, then, in just this little thing to be wiser than this man at any rate, that what I do not know I do not think I know either."
[Apology of Socrates, Henry Cary literal translation]
codingdave, I think you really hit on exactly the right point, and in doing so have also spoken to my mind exactly where we've gone off the rails as an industry in our hiring practice.
When we interview people, we do so in such a way that we believe someone is capable of showing mastery of the field, where those who have been in the field a long time and agree with your philosophy here know that about all we can can hope to achieve is humility.
It also feels like this fundamental to why this field has not been able to mature to a point of being a profession in the same form as other fields - because while we can teach a lawyer the law and we can teach a doctor a particular way to perform a technique, software is all just a bit wet and sticky.
Is that code from 6 months ago crap or just different than your current preference? I see a fair number of people churning between solutions that are more different than actually better.
Mastery is when the code you wrote 6 months ago, or a year ago, or five years ago is still running, and not causing problems. Mastery is when the code you write brings long term benefits, and not future problems.
I've both seen and written code like that, so it does exist. Like anyone one else I've also written code that causes future problems. I believe I'm at a point in my career (25 years in) where I now write more of the former than the latter.
I also have this battle. It feels like each time I take another step up the mountain, the mountain itself grows.
Doesn't matter how much time/effort I expend, there is just no way to keep up to date with every moving piece in our industry (from hardware, to firmware, to software, to laws, to programming languages, to cryptography/security, to web standards, to multimedia standards/codecs, to physics etc). It seems like every single action that we take comes with trade-offs and that we can never make an ultimate good choice - that choice is only valid for a short time before it's trade-off gets triggered.
A carpenter has a set of tools. He has a set of materials. He has a set of blueprints/techniques. He can consistently build table with x qualities. When he is done with the table, it is done. He doesn't have to spend all of his time relearning how to build tables over and over, and for old tables to quickly self-destruct if he stops baby-sitting them. Or for some 12 year old carpenter in Russia to cause his table's wood to rot.
Every year that passes by, humans accumulate more and more knowledge. It would be impossible for a few individuals to know everything as their time in this life is limited. Thus we get forced to specialize, else we can get pulled into the general knowledge stream which can be overwhelming.
Crap how? Is there mountains of commented out code, dead code, and misdirection? Has the code been adequately tested? Do you use appropriate code analysis tools (lint ect.), to enforce the above. Still having nitpicks in the code, questionable design, lines that could be eliminated, ect. is different from sloppiness and having no clue how to address these issues.
Highly competent in at least 3 languages; general understanding of 5 others; good written communication and presentation skills; able to create gui; understanding of distributed systems and networking; unix command line; can at least rebase in git
Why on earth would number of languages have anything to do with it. That doesn’t make sense. Parochial number of languages doesn’t capture the idea of understanding multiple paradigms, low level / high level, type system considerations.
A person might have zero experience except Node.js yet still show high mastery of functional programming, mixed OOP, design patterns, compilers, etc.
Setting any threshold on languages just sets up a meaningless metric.
I suppose this list could be called necessary but not sufficient. The question is what does a master “look like”. If you can only work with one or two languages you are by no means a master.
"Who is John Galt?" Mastery could most closely be viewed as a state of enlightenment. To the non-enlightened it will appear to be dismissive and lazy.
There are a variety of positions one could take once reaching enlightenment:
1. Domination. One could choose to be controlling of the entire field and use it for selfish benefit.
2. Dismissive. Once you reach the pinnacle it becomes apparent that both humanity and the society it has created are horrid and aren't worth interacting with.
3. Appreciation. One could choose to be thankful for being able to reach such a position and to respond by generously using the mastery to use it for the good of humanity.
4. Constructive Legacy. One could choose to form ones own group around that mastery to preserve it beyond themselves.
I believe the dismissive position is the most reasonable and hence most likely to be chosen by those who reach true mastery.
All four positions regularly occur and you can see it readily in masters of respective fields.
The way a master appears depends on their reaction to having gained mastery.
I think the dismissive position is also most likely because I believe there are far more masters than we ever notice. They are not noticed because the majority of them don't bother to use their mastery in an obvious way.
Why should a master bother to correct those beneath them?
If a master interacts with society they have an uphill battle to being treated with the respect they deserve. For the true master there is great value in going their own way and rejecting society.
That's an interesting question and I'm going to give you a somewhat snarky but truthful answer: Software mastery is exhibited by the code that I wrote today - and oh how horrible it is to look at what I wrote six months ago.
Through-out my career, which is now over four decades if you include the software I wrote as a teenager (before I got paid to do it), I've continuously found a better, more elegant (which can almost always be read simpler or clearer) way to tell the computer what to do.
So my non-snarky view-point is that there's no such thing as mastery but rather that it's the proverbial journey. There is however a baseline at the other end ... if it's not functional and maintainable, there's a definite lack of mastery.
More likely that you a) overrate the quality of the code you wrote today (recency bias) and/or b) have forgotten the context/constraints that you wrote the code 6 months ago under
This is of course a matter of opinion, but for me, seems like there are three sides to this:
1) Knowledge of the core theories of computers: core ideas of computer architecture, good understanding of computational complexity (how long a program will take to run), networking...
3) Knowledge of software engineering concepts: Software architecture, software process (agile, requirements gathering, testing), software complexity (and how to minimize it through good architecture and processes), general engineering principles (redundancy, fault-tolerance, solving problems early in the dev process...).
To me at least, the one that is neglected the most is the third one, "software-engineering concepts." Being able to think about systems at a high-level and well at the detail-level is one the most useful skills for an engineer.
Example masters: John Carmack, Jeff Dean, Bram Cohen. What do they do? I think they're all extremely effective at getting things done. So be smart, hard working, and laser focused on shipping.
A long time ago I saw a good talk by Jonathan Blow wherein he made a point that really resonated with me. The point was that to get good at programming you need the experience of shipping a lot of programs and thus you should optimize your career around shipping as many good programs as you can. Contrast this with the common advice of being an expert in databases, or distributed systems, or OOP, or testing, or something like that...
To play devils advocate. Perl 6 is an example of where not shipping was a major contributor to the decline of the language. Shame, as it looks interesting.
If you think it looks interesting, you should have a look at Raku (https://raku.org using the #rakulang tag on social media). Looks familiar? It should be, as it is the new name of Perl 6. It now also comes with an IDE (https://commaide.com) if you're so inclined :-)
I recently got a masters in software engineering, not that this means I've mastered the discipline, but I think it set a decent idea of what is important.
Here's a incomplete list of topics that I is worthwhile:
Creating a data schema (sql, no-sql, api based, whatever). This seems simple and obvious, but it becomes very hard very fast, especially when you need to model a complex subject that you know nothing about. Consider being hired by a hospital to model their records, or an insurance company. You need to talk to many people and understand how everything they say and do relates to everything else.
Project management: You may not want to be a project manager, but you should familiar with scrum, agile, waterfall, lean, and kanban. At least you can have an informed opinion when you argue for or against how things are done.
QA: You should know how to write a test plan for a large feature or application. This includes unit tests, manual testing, deciding what areas need more effort because getting it wrong is very bad and what is less important. This may include security and performance testing.
Architecture and design: How would you design a new application? How would you add a new feature to an existing application? How do you communicate this design to a team or many teams?
How to evaluate new things like languages, tools, libraries, technologies, etc. This isn't just reading a couple blogs, this is doing small scale test, proof of concepts, looking at the code to an extent, getting opinions for others you trust.
Finally, communicating. Software engineering is a team sport. You need to know how to write design documents, test plans, maybe architecture plans, progress reports, etc. If you can't effectively communicate your designs, plans, or even why someone else's thing is bad or good, you won't be able to convince others that you are right. And remember, you need to not just convince your peers, but also non-technical people, such as managers, VPs, and many other.
Your point about databases strikes me as domain-specific. If you write high-performance game engines, or avionics software, or drivers, I imagine you could go your whole career without encountering a conventional database.
> Creating a data schema (sql, no-sql, api based, whatever)
Schemas apply to more than databases. That may be less important for drivers, where the developer is also the domain expert for what they are implementing, but not necessarily so for avionics software. Lets saw you are implementing a black box. Are you the developer also a flight expert that knows everything that needs to be recorded and how each data point relates to each other? Not just in the sense of the hardware, but everything an FAA investigator wants to know. Probably not. So you need to work with pilots, FAA, and other non-developers to understand that domain.
Knowing how to construct a data model by talking to the experts of that domain is not just for databases.
When is "mastery" a meaningful concept for a field of endeavor? I think we wouldn't believe it makes sense to "master" physics or math, because the advanced work in those fields focuses on open problems. Even if one understands classical physics well, one cannot make specific predictions about medium-sized, low-ish energy systems like a double pendulum. Can anyone claim to be a "master" of an area in which there are such large known gaps in everyone's knowledge?
Though in software engineering we have a lot of prior examples of successful and unsuccessful projects, we also have lots of open frontiers. And in some ways, it's much harder to "know" that an engineering approach or paradigm is "right" than it is to know that a theorem is true.
How can we make types track the "important" invariances of a system? How can we convince ourselves that a distributed system can guarantee certain properties, or that a modification to that system doesn't break those guarantees? If I build a homomorphic encryption system as a service, how would I build debugging tools for it? Acknowledging the halting problem and its cousins, when can static analysis tools make useful, meaningful predictions about programs?
In the bronze age, you could build an impressive stone tower. Sometimes you could even make it stay standing. But I don't think there were any master civil engineers.
Ironically, it's often when there's no mastery that you realize what mastery is. (I.e. when things just work, are shipped on time and are easy to maintain, there's not much fanfare about it)
290 comments
[ 1.8 ms ] story [ 271 ms ] threadWisdom is to acknowledge uncertainty and that estimates are uncertain in nature because the information available at the time is always incomplete.
[1] https://www.infoq.com/presentations/Simple-Made-Easy/
Thanks for the links.
That's big commits, you're not supposed to do that. Masters commit individual characters that replace entire modules flawlessly.
and then stops.
He understands that the universe
is forever out of control,
and that trying to dominate events
goes against the current of the Tao.
Knowing that the correct answer to almost every question is "it depends".
Knowing how to somewhat-less-inefficiently produce larger-scale software that adequately meets the need. (Note that I did not say "efficiently", and I did not say "bug-free".) This takes at least some skill in dealing with people, not just computers.
You wouldn't ask what it looks like to master the practice of medicine today because there are dozens or hundreds of specialities and roles all of which contribute to medicine.
Firmware engineering and 'agile' website development are very different practices with different assumptions, requirements, etc. I wouldn't expect someone to master both web development and video encoder development just like I wouldn't expect a doctor to be a master pancreatic surgeon and develop vaccines.
On the other hand there is a 'journeyman' level of software development and engineering skill that carries over between different development areas. It's definitely not algorithmic interviews though.
This is like saying you can't have a favorite type of rectangle because some rectangles are squares. If you're a master of neurosurgery, a reasonable person would also consider that to be a master of medicine. Just as I would expect a master neurosurgeon to not be a master dermatologist but be able to master dermatology much faster than I could as a non-medical person, I would similarly expect a master C++ developer to master Javascript frontend development much faster than a junior dev, let alone a non-software person.
The other two I thought of are Peter Norton and Linus Torvalds.
Fabrice, despite his tremendous contributions to the field, doesn't reach for acclaim and just seems to go his own way.
Peter Norton was a genius in his field. His book on Assembly Language was one of my favorites as a youth. Norton tools were impressive for their time.
Linus, as the namesake for Linux also made a huge impact on the field... and comes off as somewhat of an uncontrolled lunatic.
I don't though think that these three people are the highest level masters in the field. I think they are just the ones who have chosen to do things that have high impact and visibility.
Ha! Would you consider Musk also as a Lunatic? :-)
What? Linus is one of the most Rational, Direct, No-Nonsense, Get Things Done Leader there is. Whenever i hear some self-appointed "expert" pontificate on Leadership/Management/etc. i always think of what Linus would say/do in that situation. People only think of him as a "Technical Guy" but his Leadership/Management of something so vital, so huge, so distributed, with so many "employees" and with so few resources, so many stakeholders is what is amazing to me. Nobody comes close.
You really need to take back the "uncontrolled lunatic" phrase.
I'll agree that many of the things he says are reasonable. An example is his recent defense of long lines and dismissal of the insane "80 characters per line" peeps.
What I mean specifically by "uncontrolled lunatic" is that he often ignores convention and current opinion and does and says things that get him into trouble. He also has a temper that he often is bad at controlling. I myself have this issue, and I don't see how one could say he does not have issues in this regard.
While it requires some amount of perpendicular thinking to be a master of software, having such an attitude and working with other people in that way will cause too much conflict that is unnecessary.
I think what you are missing in my statement is that I consider calling someone an "uncontrolled lunatic" as a sort of compliment.
I agree also that he is less of a technical person and more of a manager. His skills lie in herding.
On the other hand; Git is a flaming piece of garbage and he is directly responsible for its popularity... so... He did though reasonably tell people he didn't intend for anyone outside of kernel developers to use it. I just wish he corrected its deficiencies. Maybe someday he can apply his "arbitrary character restrictions are silly" logic to Git comments and become less contradictory.
Genius cannot be constrained by every single artificial social more and etiquette; we are Humans not Automatons. As long as they do not lead to destruction of product/project/self/others it is merely a character trait which makes the person unique. All successful companies know this and "manage" their most important people accordingly.
Justifying treating others poorly just enables more people being hurt. We should not reward people for this sort of behavior. Turning a blind eye to it because "they are great software engineers" is not right.
Your comment treads very much into the "for the greater good" mentality where you can justify anything based on the results.
"Unique character trait" my ass. Obnoxious aholes are just that. Obnoxious aholes. They need to fucking stop and be more kind.
It is also quite ironic that while you talk about "being nice/kind" your posts are the ones containing inflammatory phrases like "uncontrolled lunatic", "arrogance and obnoxious attitude", "my ass" and "obnoxious aholes" !
Coming back specifically to Linus, he has never been any of the above. He has been direct and harsh when needed (there is a world of difference between this and the phrases you have used), otherwise he has shown a lot of restraint. For example, he has said that due to the nature of email communication, one has to be direct since a lot of non-verbal and personal nuances cannot be conveyed which is exactly right.
I am sure we can all recollect email chains/discussions/meetings where nothing was ever achieved because people did not want to offend somebody with thin skin in spite of the fact that they were squarely to blame. Being direct and calling them out is not being obnoxious/arrogant but simply moving things along towards a goal. To be nice/kind does not mean everybody has to be mollycoddled and tip-toed around due not wishing to "hurt their feelings". Engineering is a hard science and while we have to follow social mores and etiquettes this should be regulated (i.e. we should know when to break them) so that progress towards the end goal is not derailed.
Linus has said many attacking things publicly under a position of authority.
What I say here bears no weight because if what I say is disliked I am simply downvoted and dismissed.
I disagree that being offensive solves anything. I am fine ( as you've pointed out ) with being extremely offensive and attacking the shit out of anyone I see fit, but I don't think it is particularly effective or a quality trait.
Anytime you say things in an offensive way, you are intentionally being hurtful. This is not a good thing, and I don't think is justifiable.
My doing it in no way makes it right, nor invalidates my judgement of those who make similar statements to a broad audience.
Consider the effect of me ranting here. No one is going to go: "Wow I'd better pay attention to what permille42 says he is an expert. He must be attacking people for a good reason."
Nope. Anytime I rant most readers will just immediately dismiss everything I said, mostly because my tone tends to be so aggressive. I don't think such people are wrong to dismiss as such aggressive statements do tend to be from people who are offbase and don't know what they are talking about...
Just like a master woodworker would consistently and reliably deliver good furniture.
Or a master author would consistently and reliably deliver good books.
The key points are:
* Consistent - It wasn't just a fluke or luck, you know what you're doing and can repeat it.
* Reliable - You know how much effort it will take and can deliver it.
* Good - It's not crap, people want it.
- Should know many tools to the level where they know their strengths and weaknesses. This allows them to select a useful collection of them for any new piece of work.
- Should produce incremental project changes which all either improve the value or reduce the cost.
- Should balance all the improvements to produce useful, working software with a reasonable cost. For example, producing super-secure software with no useful features, featureful software with a bajillion bugs, or bug-free software with a gigantic cost are all bad.
Novice engineers will struggle to get something done. Intermediate engineers can get things done but fail to see the tradeoffs. Expert engineers can identify possible solutions with different approaches.
- Working with experts
- Writing and deleting lots of code
- Participating in code reviews
- Reading even more code from others
- Watch & read technical presentations
You can never truly be perfect, you can only approach it. Understanding your limitations is just as important as understanding your capabilities. Depth vs breadth is hard and you only have so much life to go so deep into so many domains.
Attending retrospectives after things go wrong is a good way for experts to introduce their thought processes to less experienced engineers.
Asking "why?" when senior engineers deliver feedback during design review can help you learn that a trade-off existed when you did not see it.
Only, I didn’t like having two to chose from. Something told me three would be better. So we paired, I wrote mine, then hers, then just made another one up on the spot. Once we agreed they were complete and implementable, I shopped them to the team.
About 2/3rds preferred the made up one. Including me. So that’s what I implemented.
An intelligent man learns from his own mistakes. A wise man learns from the mistakes of others.
In more detail, the steps go like this.
Find as many solutions to a given problem as you can. Bad engineers run with the first solution that comes to mind, letting confirmation bias drive them.
Evaluate each solution for its costs and benefits. Imagine two steps in the future when the solution is implemented. What pains are there?
Search for creative new solutions that create win-win scenarios. That's riding the solution curve.
Given all viable scenarios, compare the costs and benefits against the quality measures for your specific context. Some projects value speed over precision. Some projects value performance over extensibility. Some solutions are easier to change later than others. This is choosing a specific point on the solution curve that best fits your context.
You can apply that method of problem solving to any problem, large or small. You don't need a ton of experience to practice it.
I’ve repeated it that exercise with junior engineers to great effect. Some catch on over time and start intuitively considering the trade offs of a few reasonable solutions to a problem; some don’t.
I never reflected on what he was doing there; thanks.
The best software engineers know when to solve a problem without using any code at all, like the classic "just do it manually" for a complicated task that is worth more than $x per task.
- Identify and document possible solutions. Think hard about why each solution is good and each decision is bad.
- Within each solution, attempt to find common patterns in that problemspace (when working with Sharded databases, this approach brings these results, etc)
- Try to match your personal and organization requirements against the various patterns you find and the pros/cons you defined previously
During implementation
- Keep a running list of things that seem weird, or things that seem great, or things that turned out to be untrue
- Don't stop implementing, but for each thing you found that wasn't great in step 1, try to find a better way to approach that thing (while moving forward)
Post implementation
- Compile all the notes you made in pre/during into a manageable list of goods/bads.
- Use this to drive A.) iterations of your solution, and B.) future solutions
If you do this enough times, you'll have a pretty decent list of your experiences in a space, and you'll start to notice patterns as you try different things and become exposed to problems, their solutions and their tradeoffs. Eventually you won't even have to look at your previous notes very often, as you'll have built up a pretty decent amount of experience in various problemspaces such that you can predict what goes well/doesn't go well.
Also worth noting, this type of behavior doesn't really have to be applied to software, but will also gain you big points with future employers when interviewing. You'd be surprised to find how many candidates never stopped to reflect on the work they'd done, why it was sub-optimal, or how it could be corrected moving forward. Response like "We did it ____ way because that's how we always did it." Which, from a progress standpoint, is basically a none-answer.
Note: this is obviously just my opinion, and I'm no expert in "Becoming a Master of Stuff", but these are the methods that I have used and seen my peers use (in one form or another) over quite a long time. Sometimes not as obvious (doing these things in your head vs. writing them down), but the shape is always pretty similar. Ultimately I think being reflective is one of the best skills a person can possess (with respect to employment and I guess also relationships). Acting without thinking is reckless, I think.
As they say, pain + reflection = progress. If you don't wanna reflect, good luck enjoying that pain without progress.
Naturalistic Decision Making - https://en.wikipedia.org/wiki/Naturalistic_decision-making
Expert engineers have the ability to think about a problem in a broader context than the engineering problem at hand.
- Howard Aiken
Now, you might be thinking "that's not true, if I implement a function to add two numbers, and it returns 1+1=3, that's wrong. So that's a clearly wrong solution." But I argue: is it wrong? What's your context? What are your acceptable trade-offs? In many real-world applications, precise correctness isn't always critical. Sometimes, sacrificing accuracy might be acceptable, and may save on cost and/or complexity. Other times, that's not acceptable. But it's worth considering even these types of non-obvious trade-offs, not just the obvious ones. That's what an expert engineer does: ask the right questions and identify the right trade-offs, and they've become efficient in that process from experience.
An expert sees a problem space as a wide range of trade-off dimensions, including:
Mastery would imply that you can do it all, and do it right, on any project, and there is just too much change and evolution for that to ever be true.
My goal is simply that I continue to improve.
I also have trouble believing you've been coding for so long and can't take pride in any of the code you've previously written.
I think I do what I do pretty well. The folks I work with seem to like it.
For example: I know people who have been skateboarding for decades and are just barely OK at it. How you practice matters.
I think there is mastery, but only in knowing that there is no mastery.
"I seem, then, in just this little thing to be wiser than this man at any rate, that what I do not know I do not think I know either." [Apology of Socrates, Henry Cary literal translation]
I take it one step further, I don't believe mastery exists, or rather we don't have a consistency on what mastery 'is'.
I prefer the saying something along the lines of "expertise* just means you get in more trouble when you do it wrong"
* Expertise is replaced with 'black belt' for martial art expertise
I play violin, and know and admire many masters. What makes them so good is fun to study and pursue because they are worth emulation.
But it's not some secret or hack. It's a million small things and a lot of feel.
When we interview people, we do so in such a way that we believe someone is capable of showing mastery of the field, where those who have been in the field a long time and agree with your philosophy here know that about all we can can hope to achieve is humility.
It also feels like this fundamental to why this field has not been able to mature to a point of being a profession in the same form as other fields - because while we can teach a lawyer the law and we can teach a doctor a particular way to perform a technique, software is all just a bit wet and sticky.
I've both seen and written code like that, so it does exist. Like anyone one else I've also written code that causes future problems. I believe I'm at a point in my career (25 years in) where I now write more of the former than the latter.
Doesn't matter how much time/effort I expend, there is just no way to keep up to date with every moving piece in our industry (from hardware, to firmware, to software, to laws, to programming languages, to cryptography/security, to web standards, to multimedia standards/codecs, to physics etc). It seems like every single action that we take comes with trade-offs and that we can never make an ultimate good choice - that choice is only valid for a short time before it's trade-off gets triggered.
A carpenter has a set of tools. He has a set of materials. He has a set of blueprints/techniques. He can consistently build table with x qualities. When he is done with the table, it is done. He doesn't have to spend all of his time relearning how to build tables over and over, and for old tables to quickly self-destruct if he stops baby-sitting them. Or for some 12 year old carpenter in Russia to cause his table's wood to rot.
Every year that passes by, humans accumulate more and more knowledge. It would be impossible for a few individuals to know everything as their time in this life is limited. Thus we get forced to specialize, else we can get pulled into the general knowledge stream which can be overwhelming.
A person might have zero experience except Node.js yet still show high mastery of functional programming, mixed OOP, design patterns, compilers, etc.
Setting any threshold on languages just sets up a meaningless metric.
If you have these skills there is a very high likelihood that you can do most other things that are thrown at you
Most of the other answers have all these high level wishy washy specifications, I thought I’d take a more brass tacks approach.
I suppose this list could be called necessary but not sufficient. The question is what does a master “look like”. If you can only work with one or two languages you are by no means a master.
* Consistent delivery of quality work
* Instead of complaining about bad requirements that threaten system integrity, happily works with stakeholders to compromise on them accordingly
* Can consistently describe every detail of every bit of work they have done, and answer any curveball questions about it too
* Succinct and to the point, doesn’t bog down meetings with tangential arguments and what-aboutisms
Could probably keep going but that’s good for now haha. I know I don’t even satisfy all of these points myself at all times, but I certainly try to!
There are a variety of positions one could take once reaching enlightenment: 1. Domination. One could choose to be controlling of the entire field and use it for selfish benefit. 2. Dismissive. Once you reach the pinnacle it becomes apparent that both humanity and the society it has created are horrid and aren't worth interacting with. 3. Appreciation. One could choose to be thankful for being able to reach such a position and to respond by generously using the mastery to use it for the good of humanity. 4. Constructive Legacy. One could choose to form ones own group around that mastery to preserve it beyond themselves.
I believe the dismissive position is the most reasonable and hence most likely to be chosen by those who reach true mastery.
All four positions regularly occur and you can see it readily in masters of respective fields.
The way a master appears depends on their reaction to having gained mastery.
I think the dismissive position is also most likely because I believe there are far more masters than we ever notice. They are not noticed because the majority of them don't bother to use their mastery in an obvious way.
Why should a master bother to correct those beneath them?
If a master interacts with society they have an uphill battle to being treated with the respect they deserve. For the true master there is great value in going their own way and rejecting society.
Through-out my career, which is now over four decades if you include the software I wrote as a teenager (before I got paid to do it), I've continuously found a better, more elegant (which can almost always be read simpler or clearer) way to tell the computer what to do.
So my non-snarky view-point is that there's no such thing as mastery but rather that it's the proverbial journey. There is however a baseline at the other end ... if it's not functional and maintainable, there's a definite lack of mastery.
1) Knowledge of the core theories of computers: core ideas of computer architecture, good understanding of computational complexity (how long a program will take to run), networking...
2) Knowledge of software technologies: Python, Javascript, Java, C, Docker, Functional Programming, Object-Oriented Programming...
3) Knowledge of software engineering concepts: Software architecture, software process (agile, requirements gathering, testing), software complexity (and how to minimize it through good architecture and processes), general engineering principles (redundancy, fault-tolerance, solving problems early in the dev process...).
To me at least, the one that is neglected the most is the third one, "software-engineering concepts." Being able to think about systems at a high-level and well at the detail-level is one the most useful skills for an engineer.
A long time ago I saw a good talk by Jonathan Blow wherein he made a point that really resonated with me. The point was that to get good at programming you need the experience of shipping a lot of programs and thus you should optimize your career around shipping as many good programs as you can. Contrast this with the common advice of being an expert in databases, or distributed systems, or OOP, or testing, or something like that...
Edit: found the talk: https://www.youtube.com/watch?v=JjDsP5n2kSM
Here's a incomplete list of topics that I is worthwhile:
Creating a data schema (sql, no-sql, api based, whatever). This seems simple and obvious, but it becomes very hard very fast, especially when you need to model a complex subject that you know nothing about. Consider being hired by a hospital to model their records, or an insurance company. You need to talk to many people and understand how everything they say and do relates to everything else.
Project management: You may not want to be a project manager, but you should familiar with scrum, agile, waterfall, lean, and kanban. At least you can have an informed opinion when you argue for or against how things are done.
QA: You should know how to write a test plan for a large feature or application. This includes unit tests, manual testing, deciding what areas need more effort because getting it wrong is very bad and what is less important. This may include security and performance testing.
Architecture and design: How would you design a new application? How would you add a new feature to an existing application? How do you communicate this design to a team or many teams?
How to evaluate new things like languages, tools, libraries, technologies, etc. This isn't just reading a couple blogs, this is doing small scale test, proof of concepts, looking at the code to an extent, getting opinions for others you trust.
Finally, communicating. Software engineering is a team sport. You need to know how to write design documents, test plans, maybe architecture plans, progress reports, etc. If you can't effectively communicate your designs, plans, or even why someone else's thing is bad or good, you won't be able to convince others that you are right. And remember, you need to not just convince your peers, but also non-technical people, such as managers, VPs, and many other.
Schemas apply to more than databases. That may be less important for drivers, where the developer is also the domain expert for what they are implementing, but not necessarily so for avionics software. Lets saw you are implementing a black box. Are you the developer also a flight expert that knows everything that needs to be recorded and how each data point relates to each other? Not just in the sense of the hardware, but everything an FAA investigator wants to know. Probably not. So you need to work with pilots, FAA, and other non-developers to understand that domain.
Knowing how to construct a data model by talking to the experts of that domain is not just for databases.
Though in software engineering we have a lot of prior examples of successful and unsuccessful projects, we also have lots of open frontiers. And in some ways, it's much harder to "know" that an engineering approach or paradigm is "right" than it is to know that a theorem is true.
How can we make types track the "important" invariances of a system? How can we convince ourselves that a distributed system can guarantee certain properties, or that a modification to that system doesn't break those guarantees? If I build a homomorphic encryption system as a service, how would I build debugging tools for it? Acknowledging the halting problem and its cousins, when can static analysis tools make useful, meaningful predictions about programs?
In the bronze age, you could build an impressive stone tower. Sometimes you could even make it stay standing. But I don't think there were any master civil engineers.