Agree. Depends on your domain. But for web development as I am working on right now, it is never relevant. Fundamental for me is, testing, oop design, distributed systems design.
Thank you for that. After five solid years of professional web development, I have to say that I only have to implement or think about algorithms exceedingly rarely, while naming and organization consume the lion's share of my time.
My view on coders who insist so vehemently that "performance ALWAYS matters!!" dims more and more every year. At this point they sound more like basement trolls than real craftsmen.
Performance does always matter, but for practical software development, everyone should realize that someone else did it better. The most optimal data structures and sorting algorithms have already been implemented numerous times in all programming languages; a good craftsman only needs to know when to use them and why. On the other side, the person implementing said most optimal sort should know why it's most optimal and be able to explain to others every property of their implementation.
While agreeing to the fact that performance does always matter, I think the idea that "someone else did it better" is not always a good one. One should always understand the limitations of standard implementations and see if hand-crafting something of their own would benefit their application.
For example, consider the case of sorting. Even though asymptotically efficient algorithms have been known for about 70 years now, Timsort - the currently standard implementation found in Python, Java and probably other languages - was invented as recently as in 2002.
This is not all. What if you have multiple cores available to run your code? What if your dataset is large and locality of reference matters? What if your dataset has certain special properties which the standard implementation doesn't exploit?
I agree that maybe 99% of the time, we can get away using the standard implementation but that 1% is what differentiates great from the merely good.
Performance does always matter, but for practical software development, everyone should realize that someone else did it better. The most optimal data structures and sorting algorithms have already been implemented numerous times in all programming languages; a good craftsman only needs to know when to use them and why. On the other side, the person implementing said most optimal sort should know why it's most optimal and be able to explain to others every property of their implementation.
Performance always matters. People often tend to shut that argument down quoting "Premature optimization is the root of all evil".
My opinion is that this quote was never meant to say "Performance/Optimization as a whole is the root of all evil".
If you don't have to care about it, it's because someone else did it for you. And if everyone cared a little, the web would be better place for the users(40MB for a single website maybe ?). (Not talking about other areas than web development here...)
Just think about it twice before dismissing that argument.
It doesn't always matter if your application only has a handful of users (plenty of internal applications) and defaults works at a reasonable speed. Also there are plenty of scripts that need to be run once and once only to get data from one place to another. Optimizing for performance then is pretty pointless.
Nope. I happen to be a product manager for one of such internal applications with around 60 users. This can be considered as a handful of users when compared to the internet scale. Getting the product adopted even for such a small group was a big pain. The primary complaint - it is slow! After two rounds of performance optimizations, the adoption velocity for new features has increased 10x.
I think the impression may come about because algorithmic problems have mostly been solved for you. But programming languages, frameworks and libraries don't magically appear, so advocating for a more "practical" CS curriculum may lead to stagnation if it means a lack of people knowing the fundamentals.
There's also the argument frequently made about math in schools that it's not so much about the specifics, but a way to teach people to think. I believe algorithms are excellent to teach abstraction and naming & organization are fundamentally about abstraction: what do these five functions have in common? what _is_ this thing?
I'd add that it's quite difficult to teach "naming & organization". It requires working on a large code base because the problems & benefits just aren't obvious in a typical 200-line student program. So students will have no experience of the problems you have and consequently be rather unmotivated, whereas I still remember the fascination at the genius that is the Smith=Waterman algorithm, watching it solve a problem in 10sec compared to the 1h+ that my naive solution took.
Algorithms just don't go out of style, either. If you had been taught "practical skills" in the 2000's, you'd know a lot about svn and java spring and php's $_GLOBAL, but the stuff you learnt sophomore year would have been old by graduation.
(Except svn, because Linus wouldn't have had a chance to implement git without a solid understanding of data structures and algorithms).
I saw the title and thought the article would be about how to write code that is easy for anyone to maintain. Code that scales well horizontally and across multiple processors. Code that is documented well and tests itself to let you know when a change breaks something.
I was expecting notes about architecture and how to ensure the security of your software. To always keep your end users in mind and to make everything easier for them--even if it means more work for you.
I was expecting to see examples of such craftsmanship. Best practices and praise for working tools that seem to stand the test of time.
Instead I found an article about Computer Science fundamentals and math.
Computer science isn't about craftsmanship. If it were the courses would involve code reviews and students would be marked down for lacking tests, not using source control, insufficient comments & documentation, and using conventions that don't fit with the language (e.g. camelCase variables instead_of_underscores in Python).
I'd argue that craftsmanship is about delivering a well crafted end product to the user. The user (hell, even the interpreter/compiler) doesn't give a damn what convention for variable names you use.
Craftsmanship is about delivering a well crafted end product to the customer. More often than not, that's some sort of boss or product manager. And more often than not, they don't understand which design has a better total cost of ownership. But a good software craftsman does.
If it were, there would be a lot less frustrated hiring directors and despairing grads discovering that four years of comp sci doesn't qualify you for anything except teaching comp sci.
I wish to God my courses had included any of that stuff you mentioned.
To add, craftsmanship is synonymous with experience (gleaned or direct). It means that when you start a project, you know exactly what sort of problems will come up (both for you the craftsman and the customer's use of it), and know what you need to do up front to attempt to mitigate them.
In any craft, watching a craftsman work displays a large amount of vision and foresight right from the very start, knowing what parts to do first, and what rough sketches need to be completed up front. All of these decisions feed both into making the craftsman's job easier through the process, and are tuned around envisioning what the end user wants from the product and might try to do with it even if it's not part of the explicit requirements.
It looks like black magic to the uninitiated, with a bunch of random seemingly non-essential stuff happening, with all of a sudden a great product being birthed at the end.
There's no good way to bring all that experiential knowledge into a simple curriculum. Apprenticeship is the primary way to accelerate bring somebody up to a level of craftsman, after the basic fundamentals are known. This is a primary failing of modern companies; they don't allow for such internal paths of progression. Everybody's expected to be an expert craftsman straight out of university. (cue my rant about university & academia NOT being about job training)
I'm a product designer so work at a pretty high level and have no use at all for knowing how to implement different sorting algorithms or data structures. Nor am I sure how acquiring this skill would help me at all in the development I do beyond knowing the basic properties of maps, lists, immutable seqs, or whatever else I'd make use of. And sure, if you're a JS dev you need to have a rough understanding of prototypal inheritance (though classes make this less necessary now), hoisting (though deprecating var makes this less necessary), this (though arrow functions probably let this understanding be postponed a bit now), etc, but that's just basic requirements for doing your job... not so much the art or craft of it.
I can see a case being made for studying basic architecture patterns on the other hand...
I think the author misused the word "software development craftsmanship". Or he/she is not talking about THE software development craftsmanship as http://manifesto.softwarecraftsmanship.org/
The software craftsmanship movement seems to be completely divorced from reality. For one, I can't for the life of me work out why they would list "The Creative Habit by Twyla Tharp" as a reading. That book was terrible.
It basically seems to just be a short list of tautologies. No one in their right mind would disagree on what is on the list, but they leave off basically anything about software that might actually help you achieve anything or improve the way you do it.
It seems to be some zen-like meditation based method of improvement, completely devoid of substance.
Fibonacci was a man. The Fibonacci sequence is a mathematical sequence. The various ways to calculate the Fibonacci sequence absolutely are algorithms.
Bubble sort's arguably useful as a first learning point and as a stepping-off point to learning more practical sort algorithms...but probably not something that you'll need after that.
More often than not, the sort function of the tech stack you use is the"best" already. Unless you're doing stuff at the peta scale, knowing bubble sort or not isn't going to be instrumental in the success of the project.
Bubblesort's a great learning tool, though. Knowing the implementation of bubblesort is like knowing my base-10 multiplication tables. In a realistic case, I'll use sort(), just like I'll use a calculator for any multiplication over a couple digits. Still, coding the algorithm (and some others for comparison) in class and seeing the practical differences in performance is a wonderful illustration of why algorithms have to be chosen carefully.
Will it help in this project (or any other)? Almost impossible. But it provides a nice jumping off point during education, and really, you're not likely to actually forget it, right?
The only time I found bubble sort to be of any use was when I was in pub and tried to use it for starting a conversation with non-tech friends about various algorithm. It failed badly.
I started my programming journey programming in AppleSoft Basic then 65C02 assembly language. Those were the only two languages I learned and used until college (1992) where I learned a lot of other languages but fell in love with C and then C++ and used that mostly from the beginning of my professional career until 2008 with a smattering of other languages (Perl, VB6, Rexx, Java, JavaScript, etc.)
I changed jobs in 2008 started using C# and realized that I was a good programmer, but a piss poor "engineer". I didn't know many of the fundamentals that he discussed, or unit testing, domain driven design, various frameworks (IOC, ORM's, MVC...) and didn't know how to manage a large project or even CI.
My algorithmic knowledge, and low level bit twiddling knowlege didn't mean anything in 2008 when it came to delivering real world business value.
I still like knowing the nuts and bolts, but it is secondary to getting stuff done.
Software craftmanship is about stop selling prototypes as if they were finished software.
A functional prototype is software that implements all features, but none of non-functional requirements: configuration, security, testing, deployment, monitoring, scalability, performance.
I tend to counter this argument with an analogy: "I'm building houses, not hammers." It's a problem if you don't know the right tool for the job. It's a good idea to have some idea about how the tool works, but it's not necessary to know how to build the tool from scratch, by heart.
Algorithms and data structures are tools, not products.
It's like learning blacksmithing to construct your own hammer. A valuable skill to have on its own, but, a craftsman will recognise who's good at making hammers and nails, who at sourcing wood, who at architecting houses, and who at building houses.
> It's like learning blacksmithing to construct your own hammer.
No. It's like using your blacksmith skills to construct your own hammer for
forging. Woodworkers call most of such shop-made tools "jigs" and it's a very
important part of craftsmanship.
Expected people skills, business domain knowledge, architecture and security as well. Good articles based on the arguments but fell short of covering the major points.
You do NOT need to know how to write a sort algorithm in order to be a great developer.
Just like you don't need to know processor architecture to design a website.
Encouraging beginners to study implementations of sort algorithms is a waste of time because they've got no context for it. They should write tons of really bad code first, like we all did. Then they'll get better and maybe some of that is studying the work of masters (depending on their learning style).
a great developer absolutely must know how to write a sorting algorithm, or three, and has to have a good idea how to start writing another five. great developers are building the infrastructure that good developers use to solve business problems. (not that great developers aren't doing that, too.)
That's a very specific role in the development community.
There are great developers scattered in all sorts of roles in the community. Not just building infrastructure or contributing to open source projects.
Besides, how many people need to write sorting algorithms before we say "enough! We know how to sort things now!"? I'd say that point was about 5 years ago. I think we can move on to other challenges now, eh?
Nope, new sorting algorithms are still cropping up all over place. Already published sorting algorithms are still being implemented. And people are still accelerating the already existing implementations on GPU, SIMD-capable and FPGA devices.
This doesn't even cover security issues such as side-channel timing or entropy attacks (hard to believe?) or handling of denormal numbers (which will reduce your processing speed by upto 100x). Ever heard of using Fast Fourier Transform to achieve asymptotically faster sorting under certain conditions? All of the above have practical applications.
As you yourself said, there are interesting challenges in all communities. Let's not be dismissive of the algorithm design community.
Just because we started of by writing tons of ad code and learned from it, doesn't mean that's the ONLY good way to become a good developer.
And yes, developers need to know the fundamentals. And algorithms are one of them.
We are living in an age where we have hundreds of languages and tens of thousands of frameworks and libraries.
The only way to become pro-efficient with all of these if you know the basics and know it well.
We are past the times where you learned one or two languages in school and at home and was enough to land a decent job. No it's the time for the 'Jack of all Trades'.
And please don't write tons of bad code. We already have tons of it in legacy systems and loads of libraries around the world.
I think the article can spark a healthy argument, but didn't manage to pave the way for it properly. I'm all for an "enough" theoretical and conceptual background for the common developer. I always talk about this in my circles.
A common developer should have technology-independent computer science, software engineering & architecture backgrounds. For example, unfortunately, I hardly saw a developer that can differentiate clearly between statically-typed and dynamically-typed languages, or compiled vs interpreted vs JIT, or know what is dynamic dispatch, or memory management (manual vs reference counting vs Garbage Collection), or what is the type of the database they are using, or what possible architectures we can design our system against, etc. And they all are good productive developers.
It's not uncommon to hear arguments about programming being an art or an engineering discipline. Both may be borrowing from both I think. But based on what I experienced, the scientific approach in problem solving is rare. Everyone seems to just hold the brush, and let the magic happens.
There cannot be one path to software development craftsmanship. It is a large space and there are several paths. It is continuous evolution and experience. You need to be a practitioner all your life. You have to be learning in the process of solving a problem and not learning for the sake of it.
Do today programmers start learning a useless framework without knowing anything about the programming language? Seriously???
I think that it is really crazy if it is true.
And for the other points knowing sorting algorithms (and btw the author doesn't even mention TimSort) is completely unrelated to the software craftsmanship as others already pointed out.
A craftsman knows how to build with simplicity and maintainability. Its not about how many algorithms you know, or how many languages.
We are at a point in software development where algorithms in particular are commodities. Knowing WHEN/IF to apply them is much more critical. Just like knowing when or more importantly when not to optimize is critical.
I think this article isn't bad, as no doubt design patterns and algorithms will allow you to use intuition to come up with slick code.
I'd argue though that it isn't enough to be a good programmer! Come on, do you want to be the one hammering nails in to a piece of wood, or do you want to be the one orchestrating the building of an elegant building.
You should strive to be a Software Engineer. To me, this is what being a Software Craftsman really means. Its about the entire process of writing Software, not just the drinking coffee and banging on keys part. Its past time we start taking ourselves more seriously, and gain the respect we deserve! Lets move beyond bit flipping!
57 comments
[ 5.5 ms ] story [ 90.3 ms ] threadAka abstraction!
Quoting SICP, abstraction is the means "by which compound elements can be named and manipulated as units"
My view on coders who insist so vehemently that "performance ALWAYS matters!!" dims more and more every year. At this point they sound more like basement trolls than real craftsmen.
For example, consider the case of sorting. Even though asymptotically efficient algorithms have been known for about 70 years now, Timsort - the currently standard implementation found in Python, Java and probably other languages - was invented as recently as in 2002.
This is not all. What if you have multiple cores available to run your code? What if your dataset is large and locality of reference matters? What if your dataset has certain special properties which the standard implementation doesn't exploit?
I agree that maybe 99% of the time, we can get away using the standard implementation but that 1% is what differentiates great from the merely good.
If you don't have to care about it, it's because someone else did it for you. And if everyone cared a little, the web would be better place for the users(40MB for a single website maybe ?). (Not talking about other areas than web development here...)
Just think about it twice before dismissing that argument.
There's also the argument frequently made about math in schools that it's not so much about the specifics, but a way to teach people to think. I believe algorithms are excellent to teach abstraction and naming & organization are fundamentally about abstraction: what do these five functions have in common? what _is_ this thing?
I'd add that it's quite difficult to teach "naming & organization". It requires working on a large code base because the problems & benefits just aren't obvious in a typical 200-line student program. So students will have no experience of the problems you have and consequently be rather unmotivated, whereas I still remember the fascination at the genius that is the Smith=Waterman algorithm, watching it solve a problem in 10sec compared to the 1h+ that my naive solution took.
Algorithms just don't go out of style, either. If you had been taught "practical skills" in the 2000's, you'd know a lot about svn and java spring and php's $_GLOBAL, but the stuff you learnt sophomore year would have been old by graduation.
(Except svn, because Linus wouldn't have had a chance to implement git without a solid understanding of data structures and algorithms).
I saw the title and thought the article would be about how to write code that is easy for anyone to maintain. Code that scales well horizontally and across multiple processors. Code that is documented well and tests itself to let you know when a change breaks something.
I was expecting notes about architecture and how to ensure the security of your software. To always keep your end users in mind and to make everything easier for them--even if it means more work for you.
I was expecting to see examples of such craftsmanship. Best practices and praise for working tools that seem to stand the test of time.
Instead I found an article about Computer Science fundamentals and math.
Computer science isn't about craftsmanship. If it were the courses would involve code reviews and students would be marked down for lacking tests, not using source control, insufficient comments & documentation, and using conventions that don't fit with the language (e.g. camelCase variables instead_of_underscores in Python).
If it were, there would be a lot less frustrated hiring directors and despairing grads discovering that four years of comp sci doesn't qualify you for anything except teaching comp sci.
I wish to God my courses had included any of that stuff you mentioned.
In any craft, watching a craftsman work displays a large amount of vision and foresight right from the very start, knowing what parts to do first, and what rough sketches need to be completed up front. All of these decisions feed both into making the craftsman's job easier through the process, and are tuned around envisioning what the end user wants from the product and might try to do with it even if it's not part of the explicit requirements.
It looks like black magic to the uninitiated, with a bunch of random seemingly non-essential stuff happening, with all of a sudden a great product being birthed at the end.
There's no good way to bring all that experiential knowledge into a simple curriculum. Apprenticeship is the primary way to accelerate bring somebody up to a level of craftsman, after the basic fundamentals are known. This is a primary failing of modern companies; they don't allow for such internal paths of progression. Everybody's expected to be an expert craftsman straight out of university. (cue my rant about university & academia NOT being about job training)
I can see a case being made for studying basic architecture patterns on the other hand...
It basically seems to just be a short list of tautologies. No one in their right mind would disagree on what is on the list, but they leave off basically anything about software that might actually help you achieve anything or improve the way you do it.
It seems to be some zen-like meditation based method of improvement, completely devoid of substance.
To be clear I need to not know bubblesort. And Fibonacci is not an algorithm.
Fibonacci was a man. The Fibonacci sequence is a mathematical sequence. The various ways to calculate the Fibonacci sequence absolutely are algorithms.
Bubble sort's arguably useful as a first learning point and as a stepping-off point to learning more practical sort algorithms...but probably not something that you'll need after that.
Will it help in this project (or any other)? Almost impossible. But it provides a nice jumping off point during education, and really, you're not likely to actually forget it, right?
To be clear I need to not know bubblesort. And Fibonacci is not an algorithm.
I changed jobs in 2008 started using C# and realized that I was a good programmer, but a piss poor "engineer". I didn't know many of the fundamentals that he discussed, or unit testing, domain driven design, various frameworks (IOC, ORM's, MVC...) and didn't know how to manage a large project or even CI.
My algorithmic knowledge, and low level bit twiddling knowlege didn't mean anything in 2008 when it came to delivering real world business value.
I still like knowing the nuts and bolts, but it is secondary to getting stuff done.
A functional prototype is software that implements all features, but none of non-functional requirements: configuration, security, testing, deployment, monitoring, scalability, performance.
"Product people" emphasize functional requirements, aka features.
You can sell features, but it is harder to sell non-functional requirements. Non-functional requirements are EXPECTED, and rarely verified.
Software craftmanship is a movement that seeks to make developers care about what they do, in the form of functional AND non-functional requirements.
People that do not care about error handling, logging, formatting and coding standards, profiling, scalability... are prototype engineers.
Prototypes can be used to market a product, but it's unethical to sell a prototype.
Algorithms and data structures are tools, not products.
No. It's like using your blacksmith skills to construct your own hammer for forging. Woodworkers call most of such shop-made tools "jigs" and it's a very important part of craftsmanship.
You do NOT need to know how to write a sort algorithm in order to be a great developer.
Just like you don't need to know processor architecture to design a website.
Encouraging beginners to study implementations of sort algorithms is a waste of time because they've got no context for it. They should write tons of really bad code first, like we all did. Then they'll get better and maybe some of that is studying the work of masters (depending on their learning style).
If we are talking "great _Wordpress_ developer", maybe.
a great developer absolutely must know how to write a sorting algorithm, or three, and has to have a good idea how to start writing another five. great developers are building the infrastructure that good developers use to solve business problems. (not that great developers aren't doing that, too.)
There are great developers scattered in all sorts of roles in the community. Not just building infrastructure or contributing to open source projects.
Besides, how many people need to write sorting algorithms before we say "enough! We know how to sort things now!"? I'd say that point was about 5 years ago. I think we can move on to other challenges now, eh?
This doesn't even cover security issues such as side-channel timing or entropy attacks (hard to believe?) or handling of denormal numbers (which will reduce your processing speed by upto 100x). Ever heard of using Fast Fourier Transform to achieve asymptotically faster sorting under certain conditions? All of the above have practical applications.
As you yourself said, there are interesting challenges in all communities. Let's not be dismissive of the algorithm design community.
And yes, developers need to know the fundamentals. And algorithms are one of them.
We are living in an age where we have hundreds of languages and tens of thousands of frameworks and libraries.
The only way to become pro-efficient with all of these if you know the basics and know it well.
We are past the times where you learned one or two languages in school and at home and was enough to land a decent job. No it's the time for the 'Jack of all Trades'.
And please don't write tons of bad code. We already have tons of it in legacy systems and loads of libraries around the world.
A common developer should have technology-independent computer science, software engineering & architecture backgrounds. For example, unfortunately, I hardly saw a developer that can differentiate clearly between statically-typed and dynamically-typed languages, or compiled vs interpreted vs JIT, or know what is dynamic dispatch, or memory management (manual vs reference counting vs Garbage Collection), or what is the type of the database they are using, or what possible architectures we can design our system against, etc. And they all are good productive developers.
It's not uncommon to hear arguments about programming being an art or an engineering discipline. Both may be borrowing from both I think. But based on what I experienced, the scientific approach in problem solving is rare. Everyone seems to just hold the brush, and let the magic happens.
Here is a similar article about the problem. http://www.rodenas.org/blog/2006/12/13/the-problem-with-prog...
And for the other points knowing sorting algorithms (and btw the author doesn't even mention TimSort) is completely unrelated to the software craftsmanship as others already pointed out.
We are at a point in software development where algorithms in particular are commodities. Knowing WHEN/IF to apply them is much more critical. Just like knowing when or more importantly when not to optimize is critical.
I'd argue though that it isn't enough to be a good programmer! Come on, do you want to be the one hammering nails in to a piece of wood, or do you want to be the one orchestrating the building of an elegant building.
You should strive to be a Software Engineer. To me, this is what being a Software Craftsman really means. Its about the entire process of writing Software, not just the drinking coffee and banging on keys part. Its past time we start taking ourselves more seriously, and gain the respect we deserve! Lets move beyond bit flipping!