This reminds me of the famous quote by Joe Armstrong -- You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.
Most modern software has incredibly complex dependency chains and that's what makes it fragile and unpredictable most of the time. If we focus on making the languages, runtimes and core libraries flexible enough that we don't need to assemble code from dozens of hobbyist GitHub projects to put up an app with a reasonably modern UI, we would make a huge step forward.
Who says that it's a competing standard? It's a simple local implementation that fits the needs of the application.
This isn't the right case all the time, but if implementing something is only a few dozen lines and isn't already provided in an appropriate form by a library that you're using, there's nothing wrong with just implementing it from scratch.
Agreed, I've had to compile pieces of C++ software that had dependencies on third party libraries for completely ridiculous trivial reasons. Like requiring OpenSSL for a project that's completely unrelated to networking/crypto just to be able to use its SHA-1 implementation to hash an image and check if it has changed from a previous version. 20+ dependencies later and it starts getting really annoying to compile that project for different platforms, because invariably there will be one little dependency that isn't supported because of some feature that's not even being used.
I disagree. I can relate to application setup and deployment feeling like its a fragile, error prone process but library usage in general is a good thing. Now if you're picking up obscure, poorly maintained libraries from GitHub I can see that going south quickly but as long as you're smart about not only which dependencies to choose from whom but also knowing when you really need them then you end up with a robust, easy to maintain system in the end.
The whole point of having a dependency chain is to benefit from well tested code that often does mundane things that you neither have the time or expertise to create yourself. I've found that as long as I choose libraries that are up to date, look to be maintained, are decently tested, and do something that I couldn't do as well or don't have the time to implement as well then all is well. I also have a rule that if a dependency is at all mission critical or not easily replaced I will either fork that library and maintain my own copy or create one for myself.
This has been true for every language I've worked with. Go, PHP, Node, or Ruby so far.
Updating might be cumbersome but if you're following best practices then you've got tests that ensure updating a library won't break your code. You also don't go around mass updating things. You update when there's a good reason to (security patches, language compatibility, etc.). Deployment should only suck the first time. Nowadays we have tools to ensure systems are replicable so you can be confident code will run fine regardless of the environment.
I don't think we'll ever have a plug and play experience. We have so many hobbyist libraries precisely because our languages and tools are so flexible. Hiding complexity is what programmers do for users. To hide from developers too would likely do more harm than good and result in a generation of programmers who don't understand how their systems really work. I point to Rails and Meteor as examples. There are Rails developers who are a thing in and of themselves. They're not Ruby devs who know Rails, they're just Rails developers. There needs to be the person like the Rails core team who know how to put dependencies together in a way that creates a functional modular system.
A perfectly valid case for using a library is when it implements a huge standard that you don't have the time, knowledge or expertise to implement yourself. Think of zlib, libpng or libcurl. These are de facto "reference implementations" of the respective standards, so you'd better rely on them in order to be future proof.
> We have so many hobbyist libraries precisely because our languages and tools are so flexible.
I beg to differ. The (initially) small hobbyist libraries are often working around the deficiencies of the language/runtime/core libraries. This has been the case for JavaScript since jQuery. If a third-party library doesn't implement a format or a protocol, or an intricate algorithm, but rather tries to augment the language, that is a huge red flag.
I often find myself feeling the same way about using gems in ruby. It's amazing how quickly I can get something extremely functional built out by adding a handful of gems, but after a few months I look back and realize how much extra (invisible) code I've added when I really just needed a tiny sliver of the functionality provided.
Without comment on its rightness or wrongness (because I'm still ambivalent myself since I've never personally been really bitten), it's been amusing watching some people in the Go community who have come from Ruby or Node who get really behind the idea of not importing little libraries for one or two functions. There's a lot of people who, if they see a little function that, say, computes a standard deviation in the middle of sixty other statistics functions they don't care about, would rather copy & paste the function into their own code than add an external dependency. (Hopefully they attribute it properly and watch the license.)
Perhaps some of them take it too far, but it's at least an interesting perspective, in light of stuff like the post and your experiences in Ruby.
I do agree in general that you have to be sure to consider the external dependency as a cost, and the benefit you get for it better outweigh the costs. Exactly how you determine that varies from language to language quite substantially, and probably varies by personal taste and experience level too. For instance, you might be less inclined to take an external dependency for something you find quite easy to write yourself, whereas someone not as skilled may find it preferable to take the external dependency if it means they don't have to learn the nuances of whatever it is enough to write it themselves.
Many times the "right" thing is a balance between using a library/module as a black box and implementing something yourself. It usually involves a lot of considerations.
* License is it compatible with the rest of your code?
* Relative size of module vs what you need from it. The size of the module introduces a non-trivial maintenance burden.
* Difficulty of just doing it yourself. Is the thing you need to do non-trival? like say encryption? or Distributed Consensus?
* Is the module compatible with the internals of your system? Will it require significant changes to how your code or application works?
* Platform support? Will the module work on all the platforms your code/application will run on?
All of these things influence the decision. A knee jerk response of "Just use X" may be right but you'll find yourself in the position of not knowing why it's right and thus unable to adjust if it ever stops being right.
>Also the code is faster, likely because it doesn't have to load another module at runtime.
Sure, but common sense would guide programmers about such tradeoffs. The extra time spent loading an additional library dependency would be amortized over the total execution time of the program -- IF -- the program makes repeated use of the library.
If a js file only has a single getElementById() call, substituting that with a JQuery library to get the "$" syntax would be overkill.[1] Howevever, if the javascript has many complicated DOM selects and bunch of animations, the extra JQuery load time can be justified.
One of the reasons to use a library the author didn't touch on was "insurance against unknown edge cases." For example, I could attempt to write 50 lines of code to uppercase a lowercase Unicode string. However, my attempt would have bugs in it. Instead, it would be more prudent to use the ICU library. It's a hassle to add that dependency and it's many thousands of lines more than my "simple" program but the ICU developers covered more edge cases than I ever thought of.
Certainly they may have covered many edge cases, but they may have covered them in a way that interacts poorly with your program. Unicode uppercase to lowercase is ill-defined enough there is no single good solution.
...would've said that module embodied Worse is Better philosophy of something that snowballed into a mess with uptake and feature creep. Just guessing: haven't read its source or anything.
Concise and correctness just often go hand in hand. Your approach is likely closer to The Right Thing. ;)
Yikes. Perhaps the best thing to do is not to "slurp" a file. It's a "quick and dirty" method, which certainly has its place, but perhaps not in critical production code (slurping a file of unknown size is a Bad Idea).
It's too bad the File::Slurp module has bugs, and the author seems unresponsive about them, but it's not a core module, and other authors are free to produce alternatives, which is exactly what happened. Looks like the change in the fine author's code would have amounted to a two line difference.
In my opinion, it would still make sense to use a CPAN module - I'm glad their orginal code is faster, and if it works with that specific use case - fine with me.
The problem, obviously, is if the fine author of the home-grown code needs similar functionality somewhere else. Do they make their home-grown code into a module for reuse, or just copy/paste? Say they copy/paste and then they find a bug in their implementation years down the line (or worst, someone else does, as this author has moved on to greener pastures).
Uh-oh - same problem as relying on a module, but now it's more work to fix (the code is everywhere and perhaps has evolved now independently), and there wasn't the framework of module authors, testing infrastructure, bug tracking system, peer review, etc, etc, etc there to have a drop in replacement for ya.
The line to draw for reusable code is certainly yours to make. My line not to cross is when I'm practicing, "Copy and Paste Programming" - easy enough to take the extra 5 minutes and package the code up for reuse. Makes it easier to make my unit tests up and all the rest of the good practices of a Perl programmer.
I don't really understand the point of calling File::Slurp "the right thing" in the first place. As the author says, you've got working code, why replace it with a module (or anything) unless you have an actual reason?
> During the summer rumours about a new attack against SSL started circulating.
> The only reliable way to defend against BEAST is to prioritise RC4 cipher suites, as proposed by PhoneFactor.
RC4? That heap of junk? You're kidding.
[18 months later]
> Last week, a group of researchers announced significant advancements in the attacks against RC4, unveiling new weaknesses as well as new methods to exploit them.
18 comments
[ 3.2 ms ] story [ 47.2 ms ] threadMost modern software has incredibly complex dependency chains and that's what makes it fragile and unpredictable most of the time. If we focus on making the languages, runtimes and core libraries flexible enough that we don't need to assemble code from dozens of hobbyist GitHub projects to put up an app with a reasonably modern UI, we would make a huge step forward.
https://xkcd.com/927/
This isn't the right case all the time, but if implementing something is only a few dozen lines and isn't already provided in an appropriate form by a library that you're using, there's nothing wrong with just implementing it from scratch.
The whole point of having a dependency chain is to benefit from well tested code that often does mundane things that you neither have the time or expertise to create yourself. I've found that as long as I choose libraries that are up to date, look to be maintained, are decently tested, and do something that I couldn't do as well or don't have the time to implement as well then all is well. I also have a rule that if a dependency is at all mission critical or not easily replaced I will either fork that library and maintain my own copy or create one for myself.
This has been true for every language I've worked with. Go, PHP, Node, or Ruby so far.
Updating might be cumbersome but if you're following best practices then you've got tests that ensure updating a library won't break your code. You also don't go around mass updating things. You update when there's a good reason to (security patches, language compatibility, etc.). Deployment should only suck the first time. Nowadays we have tools to ensure systems are replicable so you can be confident code will run fine regardless of the environment.
I don't think we'll ever have a plug and play experience. We have so many hobbyist libraries precisely because our languages and tools are so flexible. Hiding complexity is what programmers do for users. To hide from developers too would likely do more harm than good and result in a generation of programmers who don't understand how their systems really work. I point to Rails and Meteor as examples. There are Rails developers who are a thing in and of themselves. They're not Ruby devs who know Rails, they're just Rails developers. There needs to be the person like the Rails core team who know how to put dependencies together in a way that creates a functional modular system.
> We have so many hobbyist libraries precisely because our languages and tools are so flexible.
I beg to differ. The (initially) small hobbyist libraries are often working around the deficiencies of the language/runtime/core libraries. This has been the case for JavaScript since jQuery. If a third-party library doesn't implement a format or a protocol, or an intricate algorithm, but rather tries to augment the language, that is a huge red flag.
Perhaps some of them take it too far, but it's at least an interesting perspective, in light of stuff like the post and your experiences in Ruby.
I do agree in general that you have to be sure to consider the external dependency as a cost, and the benefit you get for it better outweigh the costs. Exactly how you determine that varies from language to language quite substantially, and probably varies by personal taste and experience level too. For instance, you might be less inclined to take an external dependency for something you find quite easy to write yourself, whereas someone not as skilled may find it preferable to take the external dependency if it means they don't have to learn the nuances of whatever it is enough to write it themselves.
* License is it compatible with the rest of your code?
* Relative size of module vs what you need from it. The size of the module introduces a non-trivial maintenance burden.
* Difficulty of just doing it yourself. Is the thing you need to do non-trival? like say encryption? or Distributed Consensus?
* Is the module compatible with the internals of your system? Will it require significant changes to how your code or application works?
* Platform support? Will the module work on all the platforms your code/application will run on?
All of these things influence the decision. A knee jerk response of "Just use X" may be right but you'll find yourself in the position of not knowing why it's right and thus unable to adjust if it ever stops being right.
Sure, but common sense would guide programmers about such tradeoffs. The extra time spent loading an additional library dependency would be amortized over the total execution time of the program -- IF -- the program makes repeated use of the library.
If a js file only has a single getElementById() call, substituting that with a JQuery library to get the "$" syntax would be overkill.[1] Howevever, if the javascript has many complicated DOM selects and bunch of animations, the extra JQuery load time can be justified.
One of the reasons to use a library the author didn't touch on was "insurance against unknown edge cases." For example, I could attempt to write 50 lines of code to uppercase a lowercase Unicode string. However, my attempt would have bugs in it. Instead, it would be more prudent to use the ICU library. It's a hassle to add that dependency and it's many thousands of lines more than my "simple" program but the ICU developers covered more edge cases than I ever thought of.
[1]http://www.doxdesk.com/img/updates/20091116-so-large.gif
What is the best number?
Where are my legs?
It's doubly frustrating when a library function is obsoleted and replaced wholesale rather than fixed of whatever defects it has.
And file I/O is triply frustrating because there are all sorts of corner cases, plus leaked abstractions from the file system and hardware.
The moral of the story is that you should never use libraries, and never do any reading or writing of files. Wait, what?
https://www.jwz.org/doc/worse-is-better.html
...would've said that module embodied Worse is Better philosophy of something that snowballed into a mess with uptake and feature creep. Just guessing: haven't read its source or anything.
Concise and correctness just often go hand in hand. Your approach is likely closer to The Right Thing. ;)
It's too bad the File::Slurp module has bugs, and the author seems unresponsive about them, but it's not a core module, and other authors are free to produce alternatives, which is exactly what happened. Looks like the change in the fine author's code would have amounted to a two line difference.
In my opinion, it would still make sense to use a CPAN module - I'm glad their orginal code is faster, and if it works with that specific use case - fine with me.
The problem, obviously, is if the fine author of the home-grown code needs similar functionality somewhere else. Do they make their home-grown code into a module for reuse, or just copy/paste? Say they copy/paste and then they find a bug in their implementation years down the line (or worst, someone else does, as this author has moved on to greener pastures).
Uh-oh - same problem as relying on a module, but now it's more work to fix (the code is everywhere and perhaps has evolved now independently), and there wasn't the framework of module authors, testing infrastructure, bug tracking system, peer review, etc, etc, etc there to have a drop in replacement for ya.
The line to draw for reusable code is certainly yours to make. My line not to cross is when I'm practicing, "Copy and Paste Programming" - easy enough to take the extra 5 minutes and package the code up for reuse. Makes it easier to make my unit tests up and all the rest of the good practices of a Perl programmer.
> During the summer rumours about a new attack against SSL started circulating.
> The only reliable way to defend against BEAST is to prioritise RC4 cipher suites, as proposed by PhoneFactor.
RC4? That heap of junk? You're kidding.
[18 months later]
> Last week, a group of researchers announced significant advancements in the attacks against RC4, unveiling new weaknesses as well as new methods to exploit them.
Jeez, who could have seen that one coming.