41 comments

[ 6.6 ms ] story [ 79.6 ms ] thread
All of these packages are created by Jon Schlinkert (https://github.com/jonschlinkert). So if you want someone to blame...
https://github.com/jonschlinkert/is-odd/blob/master/index.js

Edit:

the original code, even before the float handling, was was a perf optimisation:

    !!(~~i & 1);
Nobody wants to see that in code unless it has a name. Maybe isOdd(). Maybe we should put it on GitHub to save others time. Sure beats copy-pasting from StackOverflow.

Original:

Did you think of handling floats in your home built isOdd implementation? Thankfully Jon (EDIT: or one of his contributors - doesn't change the point) did. And they added tests.

The (flagged) article makes the staggering assumption that "we can't code" because we're not constantly reinventing the wheel. It talks about DRY being taken too far and talks about the functions being replaced as "one liners", when following a couple of links would show they're not. We can code, we're just coding on actual features, not rewriting the same poop.

Isn't this evidence that the system is working?

Because it's a package, a bug/oversight was found, and it was made more robust.

I still wouldn't personally use `is-odd`, but isn't this proof that having something even this trivial as a package can have benefits?

(comment deleted)
I find it interesting that the author added an `isInteger` dependency then went on to not use it.
I'd say there are very few circumstances where you want to know if a number is odd but could plausibly have floats as an input.
The universe is chaotic and insecure
Maybe so, but that doesn't really address the concern.
The article already addressed its own concerns:

- Manually audit and lock down your dependencies.

- Stop using nonsense dependencies, writing basic code is not reinventing the wheel.

Except that if your dependencies in turn use nonsense dependencies you're back to the problems.
Should we not count dependencies that are comprised of nonsense dependencies as nonsense themselves?

In my opinion, fork it and fix it or stop using it. Also, warn others if you have the capacity.

left-pad broke non-trivial projects like Babel, as I recall.
Yep, but there’s a big difference with how I treat dev and production dependencies. I wouldn’t use something nearly that large in production. Typically I look at a dependency to see if I could maintain it myself if necessary because it’s always risky to depend on a third-party.
... because it has node.js in it. ;)
Oh come on...

> Nonsense dependencies are hard if not near impossible to manually audit due to the sheer number of them.

Why do you need to audit dependencies you don't use? Don't use `is-odd`, nobody is forcing you too.

If you aren't able to audit a 20 line file and determine it's worth to you, then maybe you shouldn't be installing any dependencies at all.

The existence of garbage doesn't mean the system is broken, or that the ecosystem is "chaotic and insecure". It means that it's inclusive, and anyone can make a package, no matter how terrible you might think it is.

Even if I don't use is-odd directly one of my dependencies could, and another dependency of mine could use a different version, thus auditing once isn't enough.

Of course I'd also have to audit an isOdd function if it were part of the dependent library directly.

Which is why js package managers have evolved to allow you to force a specific version if you want, and allow you to lock the version number of transitive dependencies.

But at the end of the day, if a dependency that you want to use uses a poor dependency itself, then you should treat the whole thing as bad.

Personally I'm happy that we have such widespread usage of packages in the js ecosystem, because it makes it easier for me to audit things. I can run a command and get the total number of dependencies that something I want to add comes with, the total number of lines of code it includes. I can have scanners that look for malicious packages and identify them quickly and easily (rather than searching the lines of code for a copy-pasted bug or security issue).

Every time someone complains about there being "bad" packages on npm, it reads to me like someone complaining that shitty cars exist, and somehow the existence of shitty cars makes all cars bad...

The problem is the number of dependencies and the number of people you are (implicitly) trusting by using those dependencies. It's an overwhelming engineering burden for those industries that are required to do that type of 3rd-party software auditing.

When I use Java, I can trust - at an organizational level - large blocks of 3rd-party code (e.g. Spring, Java SDK) and may even be able to get them to assume some liability via agreement if their repositories for their libraries are breached and malicious code is injected. Additionally, because those libraries do so much, I require many fewer other 3rd party libraries that I'm required to audit.

When I use JavaScript, I don't necessarily get those advantages. (NodeJS on the server side has an SDK, so this point does not apply there, so this is mostly directed at browser UI development.) Sure, we may use a framework like Angular, but I don't necessarily have confidence that Google has audited the 600+ transitive dependencies that the Angular team has pulled in. Because of this, I don't think that Google is prepared to assume liability in the face of malicious software injection, which means (in industries requiring audits) that we need to track down each one of those 600+ transitive dependencies and audit each one of them.

It's less about the number of "dependencies" and more about the raw amount of code you need to audit.

If you are capable of auditing 1 100k loc package, why aren't you capable of auditing 100 1k loc packages?

Once you have audited a specific version, you can either vendor your dependencies (which I always encourage), or at the very least can lock all transitive deps down which both npm and yarn do by default now.

If you vendor your deps, upgrades/changes are as easy to audit as a single project as you can get a diff of what changed right there in your own system.

Now I'll agree that auditing any large amount of code is extremely difficult, but it's not a problem you can solve by switching to another ecosystem, at best you'll just move most of the code outside the realm you decided you need to audit.

Auditing doesn’t necessarily mean a full code review. For 10 large frameworks doing random checks might be enough for most use cases, but the equivalent method would require a lot more time for 1000 small libraries.
If you don't have any dependencies then you are going to be writing a lot of code. You then need to consider whether you might be better with a batteries included platform instead.
I'm not against dependencies, I'm against the idea that you have to install every single one you see. I'm against poorly coded and untested dependencies. I'm against the idea that a "batteries included" platform is inherently different or better than one with a widespread ecosystem of small packages.

I've used (and still use!) several systems that are "batteries included". There are upsides and downsides, but the major downside is that in a "batteries included" system, it's more often than not "their way or the highway". Don't like their API for traversing XML? Well there aren't any other options. Do you need to trade off speed for correctness for something? Well you'd better hope that your blessed implementation provides the options to do that.

I'm against poorly coded and untested dependencies.

It's not an easy problem to solve when nearly every dependency has it's own dependencies.

Don't like their API for traversing XML?

I don't know about other frameworks but there are a ton of options for parsing XML in .Net. Just because there is something included doesn't mean you have to use it.

What we are really talking about is all the little stuff like array-flatten, a dependency in express as an example. Have you audited that package, are you sure it's written to a good standard, is the developer slipping in something you don't expect? It's unlikely you would want an alternative in .Net because the solution there is robust and well thought through.

I have no less than seventeen different point releases of lodash buried in my node_modules directory for one project. That's completely insane, but every plugin we use for something has it's own massive tree of dependencies, which have their own dependencies, ad absurdum, until I've got paths too long for Windows to be able to delete down in there.

Not to even get into the clusterfuck that is semver and how the default behavior with npm isn't to pin to a specific version of a package. It's bitten me in the ass several times lately because some dependency of ours will have a new patch version, and it completely breaks everything.

I'm living mostly on the back-end, .NET world. NuGet isn't great, but it's a breeze compared to fighting with JS-land. Packages are kept to a reasonable size, rather than fragmenting into micropackages, references only update when you tell them to, and stability is valued. Of course, you've also actually got a standard library, so you don't have to rely on third-party code for basic functionality.

You might find Package Phobia[0] useful. It determines the size of a package before you install it including it's dependencies. Disclaimer: I wrote it.

Another good tool to use before you pull in a dependency is this dependency visualization[1].

[0]: https://packagephobia.now.sh

[1]: https://npm.anvaka.com

What package manager are you using, and what version of it?

js packages are encouraged to have loose version numbers to prevent the "multiple of the same version" issue from happening, and for applications newer npm (and all versions of yarn) include lockfiles by default that lock in all dependencies to an exact version that is unchanging until you tell it to upgrade.

While I won't rehash the same "standard library vs widespread packages" debate again, deps only update when you tell them to now, and you are given the tools to overwrite those versions if needed (for example, you could run a command and force all 17 of those point releases of lodash to use exactly 1). It's obviously not ideal that you'd have to do that, but I still prefer it over not having the choice at all and being stuck in "dll hell" which tends to happen with larger projects in other areas.

> Why do you need to audit dependencies you don't use? Don't use `is-odd`, nobody is forcing you too.

So should I audit every dependency of every single package I use? Every time I upgrade every single package? Nobody is forcing me to use anything, but I can't very well use the ecosystem if I avoid every single package either.

> The existence of garbage doesn't mean the system is broken, or that the ecosystem is "chaotic and insecure".

That is exactly what it means. If you allow anything and everything with no quality checks, you gain "quantity" but lose out on "quality".

> It means that it's inclusive, and anyone can make a package, no matter how terrible you might think it is.

Would you be comfortable someone who can't drive being your chauffeur? Or perhaps someone with know experience with civil engineering building your bridges? Or perhaps someone with no knowledge of finance being your banker? All in the name of inclusiveness, of course.

Why is it any different for software? The things we are talking about are the integral components of The Web -- something we relay on more than we care to admit -- it is not some toy.

Be as inclusive as you want, but not at the cost of quality -- at least not for something that affects the entire world.

And everyone can make a package is like saying everyone can build a bridge. But would you use it?

>So should I audit every dependency of every single package I use?

Yes, if you feel you need to. Like with every other part of software, you need to draw a line at where you will "implicitly trust". For many they only "review" the direct dependency and rely on that dependency to not include garbage. For others that means they want to audit every single line of code in their codebase. For others still they want to audit OS packages, OS code, hell even the hardware.

Luckily the npm ecosystem gives you tools to help with your auditing if you want to use them. Tools that can graph the dependencies, tools that can scan for poor quality dependencies, tools that can lock versions down so when you update you only have to review code that has changed, tools to override dependencies that you don't like with your own code even if it's transitively required by someone else.

>If you allow anything and everything with no quality checks, you gain "quantity" but lose out on "quality".

This is such an elitist point of view. Just because everyone is allowed to make a package doesn't suddenly mean all packages are worse quality. Just like with the web, the lack of a gatekeeper lets the ecosystem flourish.

>Would you be comfortable someone who can't drive being your chauffeur? Or perhaps someone with know experience with civil engineering building your bridges? Or perhaps someone with no knowledge of finance being your banker?

No, No, and No. But you are able to decide for yourself if you want to. If you really want a "walled garden" (i know that's a loaded term, but i just mean a place where someone needs to decide if a package is worthy of inclusion or not), then you can still get one by only using a subset of the available packages. There are companies out there that audit npm packages, there are many that have a whitelist of what is allowed to be used. You can just stick to those and pretend that the rest don't exist, and the rest of us can freely choose from all of them if we want.

A walled garden will always have a subset of a free and open ecosystem, if you feel it's necessary, feel free to try and make your own package manager that enforces a level of quality on the packages. I'm sure many would enjoy it, but don't try to get rid of all packages in other systems that you don't feel adhere to your standards. And absolutely don't claim that all js packages are bad because bad ones exist.

Doesn't specifying a "version" in your package.json completely solve this problem? Can an author replace a package without changing its version on npm?
>Can an author replace a package without changing its version on npm?

No, and now both npm and Yarn have lockfiles, meaning you lock in the exact version of all dependencies (including transitive dependencies).

The ecosystem saw the issues that were starting to crop up, and fairly quickly created solutions that are now in widespread usage.

I have a great deal of antipathy towards npm and lock files.

I don't understand how rerunning shrinkwrap can change the roster. And I don't care enough to spend any more effort trying to figure it out. (Meaning, fresh out of goodwill.)

Maybe yarn is better. I kicked its tires; seemed okay. Alas, we have too many (more than zero) nodejs projects to do a clean switch, and I'm tired of pushing rope.

But even if yarn is better... I just can't grasp how we got to this point, or how any one involved thought any of this was acceptable, on any level.

Shrinkwrap < npm lock < yarn lock
Belated thanks to you & Klathmon. Just yesterday I learned about NPM 6.x's new lock files.

I'll suspend disbelief and give it another whirl.

Well luckily newer versions of npm and all versions of yarn handle the lockfiles for you, no extra effort required at all.

If you install something, it will find and lock the dependency versions at install time. When you update that dependency, it will repeat then lock in again. You actually have to go out of your way now to not use lockfiles in the js ecosystem.

And if you really do have some js projects, take 30 minutes and look into yarn. Not only is it significantly faster in my experience, but it's also mostly a drop-in replacement and provides some much needed features like the ability to install while offline from a local cache, and interactive upgrade tools.

Belated thanks to you & paulddraper. Just yesterday I learned about NPM 6.x's new lock files.

I'll suspend disbelief and give it another whirl.

That's great! But I would like to give you a warning that I personally don't use npm any more. While they have improved leaps and bounds, in my opinion they are still far behind yarn in many areas.

Yarn has deterministic installs, an offline cache (the recent yarn/npm registry downtime of like an hour didn't stop our devs from reinstalling dependencies because they were in the offline cache), fantastic tooling around license checking and validations, and the workspaces system which makes it significantly easier to work with massive monorepos with multiple sub-packages.

If you are in a willing mood, and are already spending the time to try npm again, I highly encourage you to give yarn a shot as well. You can install it with `npm install yarn --global` if you want to just try it out (they have a ton of more robust install methods to choose from if you want to permanently switch), and aside from some different names (like you use `yarn add dependency-name` instead of `npm install dependency-name --save`), it's mostly a drop-in replacement and can even happily work alongside npm (one of the packages I maintain we provide both an npm lockfile and a yarn lockfile quite easily, while using yarn internally).

Hell just last week I used yarn to force a transitive dependency (it was like 4 levels deep in my dependency graph) to upgrade to a version which fixed a security issue in a library I maintain, then put that in the lockfile so all users that install that dependency through my package now get the updated version by default.

Still, kudos for being open to trying it again! npm has never been the best package manager by any account, but the one thing they do really well is improve.

I find this article to be one year late malicious hyperbole and completely useless in redeeming itself with appropriate action items.

> Node modules are widespread and ripe for abuse.

Most package systems kind of are, especially when they become more useful.

For real advice: checkout from npm retire and auditjs (there's also more non free options). You'll likely get some false positives (such as, who would host jQuery on a public/untrusted cdn? or why would you even pass user input into that?), but if a real malware actually showed up, you can find out as part of your CI process. Using the lockfile provided by yarn/npm is also a good way to reduce accidental unnecessary package updates.

> Nonsense dependencies are hard if not near impossible to manually audit due to the sheer number of them.

No, they are easier to audit. It's a lot easier to verify that leftpad isn't installing a Bitcoin miner than to verify react isn't installing a Bitcoin miner. React is probably the size of 1000 leftpads, too.

> Manually audit and lock down your dependencies.

No mention of what this is suppose to be. Yarn will lock your dependencies by default. The yarn.lock file will have the exact version locked. I haven't used npm in a while.

> Stop using nonsense dependencies, writing basic code is not reinventing the wheel.

Yes, writing "basic code" is reinventing the wheel. You have at least 10 free fully tested leftpads available on npm but you expect me to write one more? I'm paid to build the application, not to apply non-sense rules. This outrage over microlibraries needs a bit more substance. Your alternatives are basically "write code that already exists" or "use bigger monolithic libraries", and, to be frank, most of us have used other languages before or worked on multiple projects, and we can decide for ourselves if a function should be in in-housed or npm'ed. For something as basic as leftpad, wasting time writing it is just a waste of time. As a hard rule, this rule is garbage. As a soft rule, it's not very useful, as case by case scenarios vary too wildly in terms of complexity and significance.