If you have to depend on Node.JS to run your service, it's a red flag. Still I'm seriously considering Node.JS for every backend project we start, just because it's so much easier to hire for, and the ecosystem is so healthy. We're living in a red flag time for software development.
For me personally, the fact that even a simple node project will have well over 1000 dependencies is a problem. Slightly more complex projects will have more than 10k dependencies.
Auditing is nearly impossible.
The complete lack of a standard stdlib means always relying on community packages.
Also, not a complaint directly against node directly, but for the ecosystem and developers. There is not a culture of compatibility, further fueled by the fact that multiple versions of the same dependency can be included in the same code base by different dependencies.
Coming from any other language were it is a goal to keep external dependencies down, nodejs is nightmare fuel.
Many people judge programming languages as not just the language, but also the tools, library, community, conventions and so on.
After all, if I love Java but hate AbstractSingletonProxyFactoryBeans, I'm probably going to hate being a professional Java developer.
Some would see the left-pad incident not as an isolated oddity, but as representative of the entire NodeJS ecosystem - showing a weak standard library, a culture of pulling random, unaudited code from the internet, and rats-nest dependencies.
It's much worse than that. String.prototype.padStart has been available in Node since 8.0, and browsers for a long time. It's not in IE but polyfills are common.
If your first thought is a library, whether you wrote it yourself or not, then you're doing JS badly.
One big problem here is that Node (and Javascript)'s standard libraries do / did not have a lot of these utilities yet. It's the same with e.g. Java, so Apache Commons is one of the most common libraries to find in a Java application.
Go seems to have done a better job with a lot of utilities found in its standard library. But, it also has a mindset that it's better to just copy code than to add a dependency. Another factor there was of course that dependency management was a bit painful and fragmented until a few releases ago, unlike NodeJS which had NPM and its ecosystem not long after it came out.
It seems to be really common to sling mud at the JavaScript/NPM ecosystem from developers who work in other domains.
One of the big constraints we have in front end development is that every added feature adds to the bundle size of code we have to send down the wire to the end user.
When I worked in mobile app development third party libraries tended to be very large with many many features. Only a tiny fraction of that library ends up matching your use case. However when you build you end up bundling up the whole thing into your app.
JavaScript has more of a culture of many many small hyper focused packages. The logical conclusion being single function packages like left pad.
This has two benefits. One - you have less dead code in your final build. Two - if one of your dependencies is already using leftpad you essentially get to use it "for free".
If everyone did what you are suggesting you would find multiple instances of leftpad. One version you wrote. One version your dependency wrote. One version your dependency's dependency wrote etc.
>When I worked in mobile app development third party libraries tended to be very large with many many features. Only a tiny fraction of that library ends up matching your use case. However when you build you end up bundling up the whole thing into your app.
Surely there have to be tools to process dependency libraries to strip out any unused code.
yeah it's called Tree Shaking. It's been a massive area of improvement over the last few years. However JavaScript being dynamic can be very difficult to effectively statically analyse.
A good example: There is a library called Moment for handling dates. Due to the way the interface was built it basically can't be tree shaken. It's an excellent library but it's on the heavy side. For a while now people have been moving to libraries like Date-Fns which are collections of individual functions. This library can be tree shaken as it's obvious which functions you are using and which you are not
The reason this exists is due to a lack of leadership in the front end space. More specifically, a need to hire untrained entry level developers and deliberately not train them whether due to insecurity elsewhere and a complete inability to write documentation.
> If everyone did what you are suggesting you would find multiple instances of leftpad.
It was about 8 lines of code a child could write that you probably don’t need in the first place.
However if the threshold for not including a package is that it's 8 lines of code that would rule out using most of the functions in Lodash and Underscore.
If the threshold is not importing code a child could write that would rule out using Babel which was in fact literally written by a child.
> JavaScript has more of a culture of many many small hyper focused packages.
> This has two benefits. One - you have less dead code in your final build. Two - if one of your dependencies is already using leftpad you essentially get to use it "for free".
This culture has really backfired in practice.
What we now have is a proliferation of packages which do the same thing. The problem is that it is less likely that your dependencies are going to be using the packages, even though desired functionality is the same. For example, if you have two dependencies which need to do HTML encoding, chances are that they choose different packages for that job. So now as an app developer you have to pay for both.
This combined with a culture which loves Semver but doesn't mind breaking backwards compatibility ("just update the major version!") gives a huge mix of packages which can't be fixed by tree shaking.
The headline makes it look like a mistake, like a developer lacking care, when this was the result of a deliberate - and I would argue: legitimate - action that spiralled out of control because of too many people trusting SPoFs.
Also, breaking some javascript code on the web is hardly "breaking the internet".
When I heard "breaking the internet" I think there was some massive BGP issue or root-DNS mess up... Not that some websites could not release new pages anymore...
I side with Kik messenger and npm immediately. I don't agree with lots of IP law, but Trademarks are different. Trademarks are an important consumer protection.
I agree in principle, but do think there should be a minimum length for a trademark, to avoid destroying useful (x)TLA acronym space.
"American Airlines" is rightfully trademarkeable, but "AA" should not be (and in fact can be understood to mean all kind of things, even in aviation)
"Lego" is a trademark for plastic bricks, but the London Electric Guitar Orchestra should be able to use "L.E.G.O" or even "LEGO" as an acronym - they make music, not construction toys.
Unfortunately, there is no indication in the article what kik (as in the developers package) stood for, but if the package had nothing to do with Kik, the service, I think it should not be a trademark violation.
I think the point about AA is that it was in general use in language before there were Airlines. Kik is different. I have never used Kik, but I was aware of it in 2016. When I read the article I immediately thought of the messenger. It also does not have any other common English use to my knowledge.
Trademarks are only valid in the domain you operate in. If you had a football school called kik there is no infringement, but there is an Internet company called kik hence this infringes.
According to [1], Koçulu's kik "a new module that would kick-start the set up of new projects, and he named this module “kik.”"
Also, Stratton seemed to try and prevent the use of a "kik-starter" as well.
That doesn't sound too much like the package to deal with social media to me. Arguing that they are in the same domain because of both of them being internet-related seems like American Airlines trying to sue Alcoholics Anonymous for an AA trademark violation, for both pilots and recovering alcoholics sometimes use roads.
> "Lego" is a trademark for plastic bricks, but the London Electric Guitar Orchestra should be able to use "L.E.G.O" or even "LEGO" as an acronym - they make music, not construction toys.
Whoah there. I think perhaps it might be time to take a step back and revise your basic knowledge of how trademarks work.
Trademarks get registered in classes (goods and services are divided into 45 classes - 1 to 34 for goods and 35 to 45 for services). Not only that, but you can register for parts of a class, not necessarily an entire class.
Trademark registration rules state that you should only seek registration in classes (or part-classes) for which you have actual business activities. This is because of the fundamental "use it, or lose it" rule that applies to trademarks (failure to use your trademark for a period of five years since it has been registered could lead to it being vulnerable to cancellation by a third party on the grounds of non-use).
Coming back to your LEGO vs LEGO, if you take a moment to look at the LEGO (toy) trademark registrations, you will note that they hold trademark registration for the word "LEGO" in the following classes:
- 6 with the following selection: Key rings and key fobs included in Class 6, all made wholly or principally of common metal.
- 17 with the following selection: All goods included in Class 17 (Unprocessed and semi-processed rubber, gutta-percha, gum, asbestos, mica and substitutes for all these materials; plastics and resins in extruded form for use in manufacture; packing, stopping and insulating materials; flexible pipes, tubes and hoses, not of metal. )
- 26 with the following selection: Buckles and textile smallwares, all included in Class 26; tie-tacks, tie-pins and badges, all for wear, and cuff links, none of precious metals or coated therewith.
- 28 with the following selection: Toy models and sets of parts for constructing such toys, all made of rigid plastics, ornaments and decorations for Christmas trees; all included in Class 28
- 35 with the following selection : Publicity, advertising and public relations services; shop window dressing and arranging of displays in shops, stores, exhibition halls and trade fairs; the setting up and operation of promotional shows; compilation and analysis of business statistical information; conducting business appraisals and marketing studies; transcription services; all included in Class 35.
- 40 with the following selection:Printing services, Binding services; services for the finishing and/or treating of plastics and of articles made wholly or principally of plastics; all included in Class 40.
- 41 with the following selection: Educational advisory services; arranging and conducting workshops; amusement park services; theme park services; all included in Class 41.
- 42 with the following selection: Computer programming; rental of computers; computer services.
- 43 with the following selection: Hotel, boarding house, tourist house, holiday camp, motel, restaurant, cafe and canteen services; food preparation services; travel agency services for booking accommodation; arranging and conducting of exhibitions.
- 44 with the following selection: Sanatoria, rest home, convalescent home services.
Now, IANAL, but to be fair to LEGO (toys) I would say they have nice tight definitions on their trademark classes which are clearly only for their own business purposes. I also don't see any mention of "music" in the above classes or anything that might stop an orchestra calling themselves LEGO (or indeed attempting to register a music-related trademark).
You agreed with kik threatening legal action against the publisher of a npm module in an unrelated trade category, who after unfairly being given it by npm let it lapse into a security vulnerability that npm then had to unpublish?
Kik: Give us your package name
Azer: No im using it
Kik: not being a dick, but give it to us or we will start legal action
Azer: That is being a dick..
Kik: (ccs npm) we will compensate you
Azer: it will cost 30k because you are being dicks
Kik: (ccs npm) he is breaching your terms by calling us dicks, please give me the package name
Npm: Here you go kik, have the module name.
Kik: Thanks npm.
Azer: ?????
To clarify, they had NO RIGHT to the name. At the time, their trademarks did not cover the relevant category [1], it wasnt until a MONTH AFTER the dispute that they registered in a category that could even potentially cover its prior use [2]. NPM just gave it to them because they felt like it, not because they were obliged to. NPM even made a statement to this effect at the time. NPMs choice of course (well, Isaacs at the time) - but lets not pretend this was about trademarks - this was corporate bullying.
i linked the conversation for you to read. I paraphrased. was it the "fuck you" in response to the legal threat that bothered you so much, because i didnt cut much else of substance.
"Un-un-publishing"... THIS is one reason i highly distrust any online service that is not under my full control (e.g.: I can only fully trust a server in my basement).
If i want some services, code or programs i have created gone i want the power to MAKE them gone for good.
Unfortunately websites like archive.org mean that if you post any code anywhere online at any time, you forever lose the ability to stop your code from replicating, even if it was originally posted on a server you control.
You are right. Part of the magic that drives FOSS is that everyone is willingly contributing. If a developer no longer wants to do that, they should be able to make that decision, not npm. That’s without mentioning that the fallout was a direct result of their poor decision-making up that point anyway. npm should have just laughed at kik for coming to them about an unrelated, unmalicious package like this.
If you don't want to contribute anymore you don't have to, just go and do something else - but once the code was published it's not just "yours" anymore, it belongs to the community, that's the whole idea behind OSS. You give it away to everyone and anyone to use it and fork it and un-unpublish it as much as they like...
This is (in my opinion) the problem with (most? all?) OSS licence models: What if your software or parts of your software are used against your will or intention? What if your software or parts thereof is taken by the "Big Tech", commercialized and you think this is a total perversion of the intention you had while writing this software?
The more i think about the possibilities the less i like the idea...
If you want the power to make your creations disappear whenever you want, then the last thing you should do is publish them under an Open Source software license. An important aspect of OSS and Free Software is that other people can use the software, modify it, and redistribute it, and there is no one out there, not even the creator, who can withdraw permission whenever they feel like it.
NPM had every right to republish those packages. Koçulu gave full consent when he originally published them as OSS.
If this bothers you, then maybe you should reconsider your licensing choices.
Kind of ironic in the Alanis way to say that inon HN where you can't even delete your own comments after the threshold. It makes me more careful what I comment here
Note that this wasn't careless programming. The developer who was publishing the module pulled the release because he was threatened by a company named Kik who wanted to take over his namespace. After pulling it Joyent had no choice but to restore the module to prevent the whole ecosystem from collapsing. I think the core problem here is that releases should be immutable, just like with Java (Maven Central). Once you put something out there it should stay there until nuclear fire destroys the world, otherwise you risk bringing down the whole ecosystem, just like how it happened with `left_pad`.
Or people copy dependencies to their own repos and build from there. You can't rely on a company, even one like github/MS, to exist until end of times.
What if it contains copyright infringement, Government classified information, Child pornography, personal data, and all the data the law prohibit to distribute?
Our society cannot stand the immutability yet.
Our build process shouldn't relies on somebody else's server over the Internet for each minor deploys to the web server.
I hate to point it out, but we already have immutability. If you add something to, say, the Bitcoin blockchain, it'd take erasing the entire blockchain to destroy that information. That's a tall order.
Since Bitcoin allows for storing small amounts arbitrary data in a transaction. Seems like an attack vector is to put illegal content into the Blockchain.
Bitcoin BTC has very limited capacity, and high cost per byte. But I wonder why more capable blockchains aren't used as mass doxbins and leaks storage yet. There are many popular blockchains where anyone can upload many megabytes per dollar.
Bitcoin has not been famous for trafficking platform for the illegal data(yet). But it has its potential. All we need is a visualization tool. A tool that display the interesting data or automatically download data if it looks like URL. Complete with upload feature which spend BTC to record the user supplied data(or upload to somewhere resistent and record URL of it) to the Bitcoin blockchain, it can be a practical platform to trafficking illegal data.
Actually, I think Bitcoin blockchain has accumulated enough illegal data so just a fool-proof visualization tool is enough to make authorities serious enough to prohibit the Bitcoin. It's more of a presentation than technical though.
It would make no sense since contrary to popular belief you can be easily tracked through the use of the blockchain. And you also can't erase evidence.
> What if it contains copyright infringement, Government classified information, Child pornography, personal data, and all the data the law prohibit to distribute?
I think content-addressing is a good solution to this problem: it separates the identification of a thing from the retrieval of a thing. In this example, software can depend on '<cryptohash>-left_pad-1.2.3', without having to know or care where that comes from (NPM, bittorrent, IPFS, sneakernet, etc.). Build tools can try to fetch these dependencies from a bunch of different sources, including caches on the local machine, on a company intranet, on well-known mirror services, etc. and verify that it got the correct file by matching the hash.
A nice aspect of this is that it's resilient for things that people want to host (like left_pad, for example), but nobody is under any obligation to host everything. Mirrors are free to ignore/delete anything with illegal/malicious/spammy content.
You check all of that __before__ the deploy. You don't let that stuff in. And once you have something up it stays up. Otherwise a single idiot can bring down the whole system.
I remember this issue.
It was funny and scared a bit how node dependency works.
Some said, node ecosystem was a bit too exagerated to relay on very tiny library like that.
On other side having on a good number of dependencies on your package.json is a good sign, because you have a strong structured ecosystem to relay onto.
Keep in mind a singular angular app has about 1Gb of node_modules dependencies whereas a java spring boot app is hardly so big :)
> On other side having on a good number of dependencies on your package.json is a good sign, because you have a strong structured ecosystem to relay onto.
What's "strong structured ecosystem" supposed yo mean?
That when you need a given feature, you will more probably find an actively maintained package which cover it and is compatible with the rest of your dependencies/framework.
Serious question from someone not in the web development field: What's keeping developers from just downloading the js libraries they depend on and hosting them themselves?
Technically nothing. Practically, it's not the default behavior and none of the Getting Started documentation you'll read will teach you how to do it. So almost nobody does.
It's not uncommon for companies to host a proxy that theoretically can guard against depublishing; probably a good practice as well since it offloads the npm repo. But, someone has to set that up and take ownership of it, which is a problem in most companies (it's often some guy that sets it up and it runs fine for years afterwards, until it doesn't)
Nothing. There are even off the shelf solutions available (like Artifactory). But it's not part of standard tooling and many web developers don't really know what they're doing when it comes to this stuff. I do a lot of interviewing and I'm amazed by how many developers not only don't seem to care much about dependencies but don't even understand that it's a problem at all.
In another industry this would be used an example failure for training new engineers. But people seem to have an aversion to formal training and prefer on-the-job training. But that means you can easily end up with an engineer who simply "got lucky" for 5 years by not following any kind of good practice. Do you really want him to learn these lessons with your data/time/money?
64 comments
[ 3.2 ms ] story [ 125 ms ] threadI'm saying this as someone that hasn't used either.
Auditing is nearly impossible.
The complete lack of a standard stdlib means always relying on community packages.
Also, not a complaint directly against node directly, but for the ecosystem and developers. There is not a culture of compatibility, further fueled by the fact that multiple versions of the same dependency can be included in the same code base by different dependencies.
Coming from any other language were it is a goal to keep external dependencies down, nodejs is nightmare fuel.
After all, if I love Java but hate AbstractSingletonProxyFactoryBeans, I'm probably going to hate being a professional Java developer.
Some would see the left-pad incident not as an isolated oddity, but as representative of the entire NodeJS ecosystem - showing a weak standard library, a culture of pulling random, unaudited code from the internet, and rats-nest dependencies.
If your first thought is a library, whether you wrote it yourself or not, then you're doing JS badly.
Go seems to have done a better job with a lot of utilities found in its standard library. But, it also has a mindset that it's better to just copy code than to add a dependency. Another factor there was of course that dependency management was a bit painful and fragmented until a few releases ago, unlike NodeJS which had NPM and its ecosystem not long after it came out.
One of the big constraints we have in front end development is that every added feature adds to the bundle size of code we have to send down the wire to the end user.
When I worked in mobile app development third party libraries tended to be very large with many many features. Only a tiny fraction of that library ends up matching your use case. However when you build you end up bundling up the whole thing into your app.
JavaScript has more of a culture of many many small hyper focused packages. The logical conclusion being single function packages like left pad.
This has two benefits. One - you have less dead code in your final build. Two - if one of your dependencies is already using leftpad you essentially get to use it "for free".
If everyone did what you are suggesting you would find multiple instances of leftpad. One version you wrote. One version your dependency wrote. One version your dependency's dependency wrote etc.
Surely there have to be tools to process dependency libraries to strip out any unused code.
A good example: There is a library called Moment for handling dates. Due to the way the interface was built it basically can't be tree shaken. It's an excellent library but it's on the heavy side. For a while now people have been moving to libraries like Date-Fns which are collections of individual functions. This library can be tree shaken as it's obvious which functions you are using and which you are not
The problem is Invented Here syndrome and coddling. https://en.m.wikipedia.org/wiki/Invented_here
The reason this exists is due to a lack of leadership in the front end space. More specifically, a need to hire untrained entry level developers and deliberately not train them whether due to insecurity elsewhere and a complete inability to write documentation.
> If everyone did what you are suggesting you would find multiple instances of leftpad.
It was about 8 lines of code a child could write that you probably don’t need in the first place.
However if the threshold for not including a package is that it's 8 lines of code that would rule out using most of the functions in Lodash and Underscore.
If the threshold is not importing code a child could write that would rule out using Babel which was in fact literally written by a child.
> This has two benefits. One - you have less dead code in your final build. Two - if one of your dependencies is already using leftpad you essentially get to use it "for free".
This culture has really backfired in practice.
What we now have is a proliferation of packages which do the same thing. The problem is that it is less likely that your dependencies are going to be using the packages, even though desired functionality is the same. For example, if you have two dependencies which need to do HTML encoding, chances are that they choose different packages for that job. So now as an app developer you have to pay for both.
This combined with a culture which loves Semver but doesn't mind breaking backwards compatibility ("just update the major version!") gives a huge mix of packages which can't be fixed by tree shaking.
Also, breaking some javascript code on the web is hardly "breaking the internet".
"American Airlines" is rightfully trademarkeable, but "AA" should not be (and in fact can be understood to mean all kind of things, even in aviation)
"Lego" is a trademark for plastic bricks, but the London Electric Guitar Orchestra should be able to use "L.E.G.O" or even "LEGO" as an acronym - they make music, not construction toys.
Unfortunately, there is no indication in the article what kik (as in the developers package) stood for, but if the package had nothing to do with Kik, the service, I think it should not be a trademark violation.
Trademarks are only valid in the domain you operate in. If you had a football school called kik there is no infringement, but there is an Internet company called kik hence this infringes.
That doesn't sound too much like the package to deal with social media to me. Arguing that they are in the same domain because of both of them being internet-related seems like American Airlines trying to sue Alcoholics Anonymous for an AA trademark violation, for both pilots and recovering alcoholics sometimes use roads.
[1] https://kikloginonline.com/kik-npm-package-name/
Whoah there. I think perhaps it might be time to take a step back and revise your basic knowledge of how trademarks work.
Trademarks get registered in classes (goods and services are divided into 45 classes - 1 to 34 for goods and 35 to 45 for services). Not only that, but you can register for parts of a class, not necessarily an entire class.
Trademark registration rules state that you should only seek registration in classes (or part-classes) for which you have actual business activities. This is because of the fundamental "use it, or lose it" rule that applies to trademarks (failure to use your trademark for a period of five years since it has been registered could lead to it being vulnerable to cancellation by a third party on the grounds of non-use).
Coming back to your LEGO vs LEGO, if you take a moment to look at the LEGO (toy) trademark registrations, you will note that they hold trademark registration for the word "LEGO" in the following classes:
Now, IANAL, but to be fair to LEGO (toys) I would say they have nice tight definitions on their trademark classes which are clearly only for their own business purposes. I also don't see any mention of "music" in the above classes or anything that might stop an orchestra calling themselves LEGO (or indeed attempting to register a music-related trademark).https://www.kik.de/
https://en.m.wikipedia.org/wiki/KiK
https://en.wikipedia.org/wiki/Klub_Inteligencji_Katolickiej
"Protection" indeed.
https://www.npmjs.com/package/kik
To clarify, they had NO RIGHT to the name. At the time, their trademarks did not cover the relevant category [1], it wasnt until a MONTH AFTER the dispute that they registered in a category that could even potentially cover its prior use [2]. NPM just gave it to them because they felt like it, not because they were obliged to. NPM even made a statement to this effect at the time. NPMs choice of course (well, Isaacs at the time) - but lets not pretend this was about trademarks - this was corporate bullying.
If i want some services, code or programs i have created gone i want the power to MAKE them gone for good.
The more i think about the possibilities the less i like the idea...
NPM had every right to republish those packages. Koçulu gave full consent when he originally published them as OSS.
If this bothers you, then maybe you should reconsider your licensing choices.
Our society cannot stand the immutability yet.
Our build process shouldn't relies on somebody else's server over the Internet for each minor deploys to the web server.
I hate to point it out, but we already have immutability. If you add something to, say, the Bitcoin blockchain, it'd take erasing the entire blockchain to destroy that information. That's a tall order.
Actually, I think Bitcoin blockchain has accumulated enough illegal data so just a fool-proof visualization tool is enough to make authorities serious enough to prohibit the Bitcoin. It's more of a presentation than technical though.
I think content-addressing is a good solution to this problem: it separates the identification of a thing from the retrieval of a thing. In this example, software can depend on '<cryptohash>-left_pad-1.2.3', without having to know or care where that comes from (NPM, bittorrent, IPFS, sneakernet, etc.). Build tools can try to fetch these dependencies from a bunch of different sources, including caches on the local machine, on a company intranet, on well-known mirror services, etc. and verify that it got the correct file by matching the hash.
A nice aspect of this is that it's resilient for things that people want to host (like left_pad, for example), but nobody is under any obligation to host everything. Mirrors are free to ignore/delete anything with illegal/malicious/spammy content.
Some said, node ecosystem was a bit too exagerated to relay on very tiny library like that.
On other side having on a good number of dependencies on your package.json is a good sign, because you have a strong structured ecosystem to relay onto.
Keep in mind a singular angular app has about 1Gb of node_modules dependencies whereas a java spring boot app is hardly so big :)
What's "strong structured ecosystem" supposed yo mean?
If one of your dependency is deleted from GitHub, your project breaks, and you may never find your missing dependency again.
In another industry this would be used an example failure for training new engineers. But people seem to have an aversion to formal training and prefer on-the-job training. But that means you can easily end up with an engineer who simply "got lucky" for 5 years by not following any kind of good practice. Do you really want him to learn these lessons with your data/time/money?