80 comments

[ 5.3 ms ] story [ 148 ms ] thread
The anti package.json sentiment seems too puritanical. Sure there are downsides to it but I've rarely had a negative experience with it
Npm has had such a rocky marginal existence for so many years, getting bought out & passed around, getting random-ish backing & support to stay afloat. Now Microsoft owns Npm package repo & seems like a trustworthy-ish benevolent dictator & stable, but it's more than just ideologically gross to have all the tens of millions of eggs in one basket, it's a constraint on exploration & growth & trying other paths.

I think simply being free to try for better is a necessary & obvious step for js. It's not puritanical at all, it's embracing a lost freedom, to improve & figure stuff out.

agreed, they got rid of package.json just for them to rediscover its convenience and reinvent it later
It is fairly bad. The scripts section, which can't call other scripts sections neatly, is the worst.
Tbh, i think the scripts section is what makes package.json become a mess. It's the #1 reason why packages break and why it takes forever to install some packages. If your package requires building, it's better to just create NPM/package build artifacts ahead of time.
Agree. Other than that it's decent. I say package.json with a minimal scripts section (so you can run dependency commands) and pop the composition of those commands in a Makefile.
One of the tricky things, and one of the reasons everybody reinvents the wheel, is that Makefiles aren't portable to Windows, which is a first-class platform for these runtimes. So you get a lot of second-system effect as a sidecar to "runs on Windows".

(I don't think you as an individual should worry about Windows if that isn't a platform you care about, of course. But they have to.)

Tangent - Why hasn't anyone ported make to windows yet? Isn't it "just" a task runner at the end of the day?
`make` has been ported long ago! The problem is that almost every Makefile wants a reasonably large subset of POSIX stuff, when it doesn't want specifically GNU stuff. (And like, I write specifically GNUmakefiles because I don't care about those platforms.)

`make` being portable doesn't help that `mv` and `cp` and all that don't port amazingly well. You can write binaries for them that take POSIX flags, of course, and people have, but now, to replace package.scripts with a Makefile, you have to pack a whole bootleg quasi-POSIX environment alongside your JS interpreter? That's not really going to fly with most people. But assuming you do that--how do you write a loop that works in sh, cmd.exe, and PowerShell?

You get the idea. The only way to make a Makefile a real, cross-platform tool is to bring a Unix with it. And that's not super practical in a lot of places! So instead Node (or Ruby/Rake, etc.) converge to a point where libraries like `rimraf` are both really annoying and kind of essential for building an adequate cross-platform dev experience.

Again: not endorsing. But it is why it's the way it is.

I wonder if npm for Windows just installing a wrapper for GNU Make, made available as npm run make, would solve that.
... and where you can't provide ready packages (native (C++ etc.) modules) easily it is still a bad idea to have scripts run automatically by default. Foreign code should only be run after (a chance for) review on request.
You can use a Makefile instead, package.json doesn't really need this section tbh
As mentioned elsewhere, you cannot reliably use a Makefile on Windows. A Makefile also is not aware of things like Node execution paths, the way `yarn exec` is. You're then pushing more stuff down to the writer of the package.json file, who is more likely to make mistakes in retail then you as a platform designer are likely to make in wholesale.

There are good reasons for things to converge the way they have. [EDIT: removed uncharitable reading]

I offered an alternative for the frustrated and did not make any such "just use X" claim

Converge is a weird word to use, given the OP is about a project that creates divergence, now having to add support for other parts of the ecosystem

Sorry. That was uncharitable of me. You did not make a "just use X" claim. But your suggestion does elide the significant difficulties in doing as you suggest. It's an alternative that will lead to more frustration due to the tons of design decisions wrapped up in package.scripts. And asserting that because you can do this, package.json "doesn't need this section tbh" is misleading and a recipe for further frustration for people who listen to your advice.

And yes, Deno not literally being Node is in fact its problem. I've shipped stuff using Deno--and I don't hate it, and I did use Makefiles--but yeah, that's the actual problem here. Node conventions are the ones everyone expects and every tool expects, and Deno does not have a unique selling point to it that makes using it more reasonable than just using Node.

yup, I've personally stopped worrying about Windows support since WSL-2. It makes things much simpler for most people / projects.

The idea that dependencies can run any script at install time has always seemed problematic to me. Node seems to have many less than ideal design decisions that we are now stuck with

> I've rarely had a negative experience with it

We all bury bad memories. Feels like yesterday TypeScript couldn’t resolve a package because I set “moduleResolution: node16”

Oh wait, that’s still happening, because a dependency does not use the correct package.json exports setup.

Got VC funding and now they realize their “we just want cool little scripts to do things” isn’t what the community wants, now Deno is gonna be Node but with TypeScript as a first class citizen and way less friendly.
(comment deleted)
But why json? json is only marginally human-editable as its pure version doesn't even support comments. toml is way nicer for this purpose I think.
For one, JSON has native support in the very language this runtime supports.
Deno has TOML in the standard library too.
I thought TOML was nice until I tried writing a parser that handled nested tables. There's very little convergence between libraries because it's such a hard / ambiguous problem.

I thought I was making good progress until I tried burntsushi's edge case test suite. It reminded me of implementing YAML.

TOML is slightly nicer to write than JSON if your structure is flat, like a php.ini or Cargo.toml file. Other than that, I prefer the simplicity of JSON as well as the buy-in of JSON Schema support in apps like VS Code.

We too often do this thing as craftspeople where we dramatize how much minor pain points impact our lives while ignoring the upsides.

I agree that nested data is a weak point of TOML. But writing TOML files that don't have huge levels of nesting is a charm, which would be precisely the place where php.ini, Cargo.toml and package.json operate in. Deno had the chance to innovate here, but they missed out on it.
This is the right way to go.

Nodejs is “good enough”.

package.json is entrenched.

Deno gains nothing from not supporting package.json, and has everything to gain from becoming a “better node than node”.

Deno will pick up steam if it can become a drop in replacement for nodejs. Then people can start to exploit its unique features.

Ideally I could replace node and switch to deno by doing “npm install deno” from within an existing node project

Reading the reasons...I could have told you this when Deno was released. Oh wait, pinned package versions are too specific??? Maybe you want some sort of tool where you can give it constraints on package versions and it'll find the minimal set that fits these constraints. Some sort of...package manager? And big url imports are kind of ugly no? If only there was a way to resolve them via a manifest that this package manager could also read. Some sort of package...file, like a package.json.

I dunno man, this feels like stuff the Deno people could have predicted. I don't understand why people treat packages like the boogeyman. You need a way to share and reuse libraries. Then you need a way to version these libraries. Then you need a way to reuse versions of said libraries. Then you need a manager to handle the reusing and constraint solving of libraries. It's a pretty logical sequence of events. I know it seems icky and gross and we should all just be real programmers and write our own dependencies, but you don't get paid to write dependencies.

I guess that's the issue. The people who write stuff like Deno or Node do indeed get paid to write dependencies. And it's too easy to listen to the people complaining about straw men like is-odd when in fact, for all the messiness, developers get a lot of utility out of packages.

> You don't get paid to write dependencies. I guess that's the issue. The people who write stuff like Deno or Node do indeed get paid to write dependencies.

Brilliant take. I realized something similar yesterday when my twitter feed was filled with people saying SSR/edge compute is the future... wait a second, you all work at Vercel, you make money by me computing stuff server side!

I wish Vercel (or Zeit at the time) never abandoned their original model of running arbitrary Docker containers at the edge. I understand the desire to focus on one platform, but you could implement a lot more interesting use cases with arbitrary serverless containers, especially if the build process was as smooth and integrated with GitHub as it is for JS (although arguably a big reason for that smoothness is the focus on one platform).
This whole ordeal completely smells like “15 competing standards”
Deno is the perfect real world example of a programmer looking at the complicated mess of code they wrote over many years, deciding to start fresh to fix all the problems, realizing their simple and elegant new product does not address all the use cases that the old one did, adding more and more complexity to it until it does, and eventually ending up with exactly what they had before.
At least we get typescript, I guess.
Typescript support has always been a one line addition to Node.js. What Deno did (moving ts-node to just work automatically behind the scenes) was nice to have but not exactly revolutionary.
If you are doodling around ts-node is great, but if you ever want to produce a ts library someone else might want to use it immediately becomes a endless eldritch nightmare maze of byzantine frustrating options almost all of which come with trade-offs.

Having typescript built in, having zero complexity, none of these compile-down difficulties is just so liberating, so much bad you don't have to tangle with.

When looking at my old code my internal dialogue flips between "What an idiot I was for writing this code" and "So brilliant and performant! I could never write code like this now."
I imagine it like I'm on my PS5, playing something like Elden Ring, and I'm on a roll. Then I reach a boss and get it down to a sliver of health before I make one fatal mistake and it kills me.

On the retry I still have the confidence from the previous attempt but I can barely get a hit before being completely destroyed. How hard could it be? I was one move away from victory last time. But now the challenge is far more apparent.

This happens repeatedly as I use trial and improvement to learn the fight I so effortlessly put up in the first instance.

It reminds me of just how much is down to chance and what you did in a moment in time. Trying to consciously recreate it, but better, is a Sisyphean task.

> On the retry I still have the confidence from the previous attempt but I can barely get a hit before being completely destroyed. How hard could it be? I was one move away from victory last time. But now the challenge is far more apparent.

One thing I've found with these types of games is getting into a "flow" works so much better than trying to consciously play the game. Unfortunately the only way to develop the muscle memory to be able to enter this flow state is to bash your face against enemies over and over until you're no longer thinking about how to react. I've also found that when I'm struggling with a boss, I'm playing in a much more conscious way which means I'm thinking about my responses instead of just reacting. When I get really close, but don't kill them, I often over focus on the piece which got me killed and which means my attention on the dozens of other things I've used to get to this point slips a bit and I get destroyed. At this point I usually know I'm too much in my own head and go off to areas I'm comfortable with to try to get back into a flow for the boss fight again.

We often hear about seeing our old bad code and cringing, but not as many people talk about seeing their old code and being impressed. I've experienced both as well.
That is precisely the signal I received from this headline.
I mean... not really?

The goals of Deno and Node are different, the architecture is different.

Adding a compatibility layer helps Deno grow while maintaining its original goals.

I doubt this adds much complexity if done properly...

Clearly you've never seen the at least 2 talks that Ryan Dahl has given about how he viewed package.json and npm as the two biggest blunders of Node's life and he was going to correct that in Deno.

And here we are.

I have, but you glossed over my point.

What's wrong with a compatibility layer with npm if it's implemented properly?

His views on package.json and implementing a compatibility layer to help others transition from node are not mutually exclusive.

The support for node is pretty simple. Using NPM packages is straight forward. Reading a package.json file and using those mappings isn't complex either. Deno does not use node_modules in any of those instances...

His point was that the mere ability was the issue, not some implementation.

This is why Deno has gone so long without a true package manager. The problem here is now they're fighting for adoption so they're going against their original values because they know what they wanted isn't what the broader community wants.

> His point was that the mere ability was the issue

Can you elaborate?

> so they're going against their original values because they know what they wanted isn't what the broader community wants.

No, they are still sticking to everything, but added support for the existing ecosystem. Yes they are adding this to combat the network effect that node has, but it doesn't really go against anything. You simply prefix npm in front of npm imports. Now you could use a package.json file if you already have one, but you don't have to.

No. They are adding "deno:package@version" native imports, and they are adding a deno.json file to specify them. They are adding dependency resolution logic for these native deno packages to deno itself.

The fact that all this interoperates with NPM as well is just an added bonus. But even if NPM died tomorrow, deno would go on having a package system, because they have realized how important it actually is (to fix things like what they call the "duplicate dependency problem").

Yes and that is nice, it'll allow future import map support, like in the browser, which is inline with Deno's goal of using browser APIs instead of proprietary ones.

But the package.json format part is a backwards compatibility feature. The preferred import maps will be the ideal solution (if you do need dependency management).

None of that really goes against direct imports, which work well for single file scripts, workers, etc. Choose what fits.

The blog post didn't make it sound like this was implemented just for the sake of backwards compatibility, it sounds more like an admission that Deno's take on package management isn't a superior solution to how Node did it before.
Could you point out a few things in the blog that made it sound that way to you?

It states many times it was added for backwards compatibility.

Even at the bottom of TFA they annotate the ideal solution (which will be implemented soon) is to use import maps instead of package.json, the same format the browser uses, which is in line with Deno's main goals.

You're ignoring the fact that, originally, deno wanted to eschew the concept of an import map entirely. They weren't against the format of package.json itself, or even against the fact that it's not a web standard. They were against the idea of having to specify imports sepeartely from your code, and against the idea of bundling multiple files into a concept of a module.

Fast forward a few years, and they are now adding these two things natively to deno, just as the web ecosystem did. Turns out, Node was right to have modules and an import file all along, and at worst there were some minor implementation quibbles to argue about.

Personally I don't really care about the faux drama.

I'm glad I can use direct url imports in simple Deno scripts.

I also see the need for import maps for larger projects.

I do think node_modules is an abomination and glad I don't have to deal with them anymore.

I prefer writing everything that works in Deno and the browser, not Node specific APIs.

I like the tech, I don't really care about the drama, and I doubt the Ryan does either lol.

That's perfectly fair, but then there's no need to jump into a thread about how a project reversed course on a major point it started with by claiming that it hasn't.
I disagreed with the GPs premise and I commented my viewpoint. This is how a forum works.

I disagree with how you summed it up, but I don't really care to continue.

It's getting too dramatic and not really about the tech at this point.

From what I remember, Dahl considered node_modules one of the big blunders, which afaict they have avoided with Deno.
Ryan Dahl's talk reminds me of the time I came back to a forum I created and then neglected after being distracted by university for four years.

The forum had taken off while I was MIA. In spite of me being MIA. And while I could write a manifesto about how the forum had gone in a direction I didn't necessarily want, I had to admit that my plans for the forum weren't better nor more correct than the plans that emerged organically from the community over the years.

When you create something with a community and ecosystem, it's easy to see the success of the ecosystem and the downstream success of the people within it as your success when in reality you are more like an early collaborator who set the stage for others to join in and take it from there.

A good example of this in my case was the hubris that, because I created that one big forum, I could create more successful forums. I never hit gold again. I realized I wasn't some master forum creator, just someone who started something at the right time and the right place with just enough TLC to attract other builders, and that I didn't have some divine insight into how to start a successful forum. Oof.

You hit the nail on the head here. This is exactly what's going on.
The difference in your analogy is he is currently working on an alternative (a good one imo), not only talking about it...

His product is currently out there, popular, and already infinitely better than node (imo).

I'm sorry you never "hit gold again", but you seem jaded and projecting that onto others.

Who cares if it never fully overtakes node? It's a great alternative. Competition is good.

I'd consider Deno a success already, it growing in popularity is always nice, but the tech already works.

> Who cares if it never fully overtakes node? It's a great alternative. Competition is good.

I'd say the people who gave them $21M[0] care if it's adoption doesn't take off.

[0]: https://deno.com/blog/series-a

I didn't say if "adoption doesn't take off" I said "fully overtakes node".

Should no other browser be invested in because Chrome has the largest market share?

Should no-one ever invest in a competing product?

I believe it is up to his investors what the expected growth rate and market share is.

(comment deleted)
This is not (just) about adding a compatibility layer. The original deno presentation complained about the whole concept of a module that Node introduced (rather than "simply" referencing script files the way an HTML page does), it complained about the need of an additional file to specify imports etc.

The blog post explains in detail why these things are actually required and useful, and introduces the new deno.js file that is functionally equivalent to package.json. So even purely deno packages meant to be used entirely in the deno ecosystem are encouraged to use the new package.json 2.0, for many of the same reasons package.json was introduced in Node all those years ago.

(comment deleted)
Hmm, I don't agree. I see Deno's package.json support as a pragmatic solution to interact with the rest of the world, but for my current use case (command line tools, switching over from Python) the npm registry is quite irrelevant, but the ability to directly import via URL is a real killer feature (also compared to Python).

(edited)

> the direct URL import is vastly superior to external package managers.

This is a tooling/ecosystem issue. E.g. in Elixir where the package manager is a part of the distribution, one-file scripts with dependencies are easy: https://fly.io/phoenix-files/single-file-elixir-scripts/

Well yeah, but Deno is a tool and ecosystem, just a different one from Node.js and NPM :)
So it could easily provide the same functionality if it so chose :)
Deno reminds me of how Python 3 decided to break compatibility in order to get rid of legacy code, and after finally replacing Python 2, has accumulated its own share of legacy.
This feels really short & snide & simplistic to me.

I'd like to see you argue the other side of this discussion & see if you have the eye to appreciate what is better & nicer.

There's a lot of complexity that's still way way way away in the rear view mirror. It's still super possible & super nice to use deps.ts to tackle a lot of the current situations.

Very side mention on deps.ts, I think there is a bit of missing potential to pass your tree of deps into other packages deps, but this is definitely just working around import-maps's intended use case in userland.

> It's still super possible & super nice to use deps.ts to tackle a lot of the current situations.

You mean:

- manually import and re-export every dependency you depend on

- cannot change resolution mechanism to a different package repository, for example, in a corporate setting. Especially not for dependencies of dependencies.

You could probably automate it by providing a package.json of sorts, and reading from there...

Deno seems to have decided to try to solve problems by facing the past. This is really good.
One thing I'm curious about is if it's just possible to sort of roll your own package version manager as part of your deno app.

For instance, the current thing, from what I understand, is <code>import foo from 'https://some-cdn/your-library@^1.1.1'</code> and deno loads that in. The problem that adding a package.json solves (also, from my limited understanding) is for cases when you import this library in several places and have different version requirements.

So, why not have something like:

  export const libraries = {
    "some-library": "https://your-cdn/some-library@^1.1.1",
  };
  export const load = (library) => import(libraries[library]);
Wouldn't that mostly solve the problem without just doing what NodeJS does?
Because now you have to write this manually for every package. And how do you deal with dependencies of that package that also need to be fetched from `your-cdn`?

Oh, perhaps write them in a JSON form in some sort of package.json and read them from there perhaps?

> Oh, perhaps write them in a JSON form in some sort of package.json and read them from there perhaps?

Maybe even have an executable that will read this file and understand your dependency graph and needs and install everything for you, and also allow you to keep it up to date.

Maybe put this in `utils/npm.js` // short for node package manager.

Brilliant. Lets all implement our own.

Well, you wouldn't need to for every package, it really depends on what you're bringing in and what dependencies they have. I do see your point, though.
Bare specifiers has been the tragedy of ESM. Nice module syntax... that is utterly u deoyable & which has had to have awful de-modularizing specifiers hard-coded into each file to make it work. Abominable sin to introduce "modules" to JS/es2015 then spend a decade dragging everyone along with no story for how to have modular modules.

Import-maps are like "here" to fix this on the web... finally... except they only are shipping to the happiest sunniest easiest case, with Web Workers being totally shit out of luck in spite of some very simple straightforward suggested paths forward. https://github.com/WICG/import-maps/issues/2

I think Deno is making pretty good tradeoffs along the way here. This looks like package.json at surface level, but there is a nightmare of complexity under the surface. Typescript, ESM, cjs all have various pressures they create & in Node it's just incredibly tight & tense dealing with packaging, where-as Deno's happy path of Typescript first does not slowly tatters one over time. It really has been super pleasant being free of the previous world, and having something much more web-platform centric, more intented, with less assembly & less building, and more doing the actual coding.

I confess I'm a bit surprised by this tack. The duplicate module issue felt somewhat solveable my offering more http urls... https://deno.land/std@0.179.0/uuid/mod.ts could be turned to https://deno.land/std@0/uuid/mod.ts to pick just the major version.

Also notably these un-pinned imports are going somewhat the opposite direction from hopefully upcoming work on subresource integrity in import-maps, where we we can really pin versions, https://github.com/WICG/import-maps/issues/221

I really hope import-maps eventually get broader support. Maybe this long-dwelling webworker issue should be brought up with WinterCG.

almost! now separate the namespaces from the imports and we are good:

"imports": { "@namespace":"deno", "oak": "oak@12",

My first draw to deno was to escape npm and the node_modules blackhole :(
The biggest issue with node.js was the terrible package manager. Looks like Deno is gonna make all the same mistakes.