First install the prerequisites:
SquashFS Tools 4.3
Python 2.6 or 2.7
Visual Studio 2015 Update 3, all editions including the
Community edition (remember to select "Common Tools for
Visual C++ 2015" feature during installation).
Why not just Node.js and Git for Windows as requirements? Python 2 and a specific VS version, really? Is using npm command to get further packages too uncool?
This is risky to say but... NPM is a bad package manager. I could blab about why forever but tons of other people already have. I've never used a popular language with a package manager as bad as npm.
It's not risky to say. It's risky to do a shallow critique with no solid analysis. I would personally enjoy hearing a critique of why npm is bad in comparison to other package managers.
As someone who has spent a lot of time with npm (not even as a webdev), it appears to solve all of my common use cases while also dealing impressively well with various versions of specific dependencies. (Like, A depends on B@0.0.1, C depends on B@0.0.2, etc.)
There's a ton of annoying little stuff (like doing nothing to prevent path names too long for your filesystem) that could be forgiven for being young, but the big problems are in its architecture.
Node module syntax is non standard and can't be tree shaken, with a little more forethought it could have been designed to support static analysis and they wouldn't have needed to reinvent the module syntax for ES6. You can't reliably tell if a piece of code is using a given package until you run it.
The dependency resolution sucks. It's better now that they finally made it so nested dependencies aren't duplicated, but why the hell do I have to run a manual dedupe command to do so? What that tells me is that npm doesn't actually track dependencies, its hardly more than a fancy file downloading tool.
The repository curation sucks. I'm not sure if you've used Python's package manager, or Java's, or C#'s, or Perls, but npm has zero package curation and is filled to the brim with garbage compared to the others. I have published packages for C# and Java and the semantics are well defined and reasonable, you even cryptographcally sign your code and the system checks to make sure you upload documentation etc... In the case of java, which has the best system I've seen, they even use something similar to DNS validation to make sure you control package namespace.
The above point is the absolute worst thing about npm and drawfs the other problems. Let me give an example by pasting the package publishing guidelines for Maven(Java) vs npm:
Publishing an npm package:
>Use npm publish to publish the package.
>Note that everything in the directory will be included unless it is ignored by a local .gitignore or .npmignore file as described in npm-developers.
>Also make sure there isn't already a package with the same name, owned by somebody else.
>Test: Go to https://npmjs.com/package/<package>. You should see the information for your new package.
I had to condense the Java guidelines but it's roughly this:
create JIRA account
apply for access
review requirements
verify namespace ownership
generate crypto keys
package coordinates/versioning
project description URL and license info
developer info
SCM info
package deployment
generate binaries
revolve package dependencies
crypto sign package
generate documentation
online verification
Sounds like a lot of work right? It takes a few hours to setup and verify your domain/keys the first time but after that all of these steps are automated. The signing/building/doc generation/ deployment is totally automated. Npm is a toy in comparison.
from their docs:
Why Do We Have Requirements?
In order to ensure a minimum level of quality of the components available in the Central Repository, we have established a number of requirements your deployment components have to meet. This allows your users to find all the relevant details about the components from the metadata provided in the Central Repository. The following sections will detail these requirements.
Left-pad is a perfect example of why npm sucks. Nuget, Maven, Phython, and CPAN would have never let something that stupid happen.
All of those left-pad libraries are a jest, it's trivial to do padding in those languages. I was wrong about Python and removing packages but on Nuget and Maven you can't, its not allowed.
I think it's worth noting that with Java at least, besides not allowing package removal, the namespace is tied to a unique identifier with ownership verification, so the cause of the left-pad fiasco (kik wanting his namespace) is extremely unlikely to happen in the first place.
If the curation process allows joke packages to pass through, it is completely useless. Looks like the only difference from npm is extra bureaucracy then.
- dynamic require()'s can't be tree-shaken: same is true of other langs, e.g. Python. Perfectly understandable IMO as Node was designed for the server, in a time tree-shaking (largely desirable for front-end code) wasn't a priority. Also it is a standard (CommonJS).
- long pathnames is a Windows only issue and afaik fixed by npm 3's "flattening".
- npm is worse than Maven because 'npm publish' is too easy?
- you don't need to run dedupe.
- you say npm isn't curated but I'm not aware those other registries are curated either?
- 'left-pad is a perfect example of why npm sucks' ... except the ability to delete packages has been fixed, so should that be 'sucked'?
(Edit: how on earth do you format lists correctly on HN?)
One of the basic features expected of a package manager is reproducibility (i.e. if two people build the same code with the same dependencies, it should produce the same result). On that count, npm is literally broken by design:
Well no, installing 2 packages, one after the other from the command-line (as in that issue) is not the way you 'share builds' with npm. You do that via a package.json + 'npm shrinkwrap' "lockfile".
Shrinkwrap does have known issues (mainly w.r.t. platform-specific modules and verbosity), though I've never had any problems with it, but if you want a better lockfile I suggest the Yarn client.
(Believe Python's pip behaves similarly to npm in that issue).
I prefer Yarn's lockfile handling, but saying npm is 'broken by design' is wrong.
There are evident security issues with this, as spelled out in the bug. Yeah, I'd consider that broken. Of course the repro is contrived, but it's not at all unusual to install some packages manually. The point is that once you do so, you might not get the same result later from package.json, either.
And yes, Yarn is the sane option for package management for Node. Why isn't it the default one yet?
> but it's not at all unusual to install some packages manually
If you're taking about deployments, that's definitely unusual. Outside of deployments, what's a scenario where deterministic builds are important, but it would be considered normal to manually install them anyways?
I don't see why deterministic builds would only be important for deployments. Even ignoring the security issues here (which don't necessarily apply only to deployments), there's this whole classic "works for me, not a bug" situation.
One other one is that it has hilariously poor performance, and bizarre failure cases. For some forms of network failures it'll just say "callback called too many times" and crash.
There are literally no worse major package managers.
A good package manager can be misused and still work reasonably well; a bad one doesn't even work all that well when used correctly.
I especially like the part where it says "This is most likely not a problem with npm itself", even though it clearly is a problem with npm itself (as evidenced by non-deterministic nature of the bug).
How many projects have you seen that are agnostic about VS versions? VS doesn't maintain a stable C++ ABI between versions, not to mention the fact that VS2017 has broken a fair number of projects. It's not like pinning a clang/gcc version.
No, Blink doesn't include V8, it's a separate project. WebKit consists of WebCore — layout engine, and JavaScriptCore — JavaScript engine. Google forked WebCore part of WebKit to create Blink. Before the fork it used a combination of WebCore and V8.
This information is readily available on, e.g. Wikipedia, so I'm not sure why you even try to argue about it.
Node.js doesn't include Blink, it only includes V8. Node.js Compiler doesn't include Blink into the build result — it includes Node.js.
It really depends on version of node, and the modules used. I'm more inclined to want to see what can be done as a minimal electron... or seeing electron go the adobe air route, and have a shared (continuous updating) base, but that could prove problematic as binary modules (namely sqlite) are common.
Depends, really. If the target is a common OS then having people install Node.js itself plus grabbing your application as a zip file that has the code and a few different files to act as soft executable (myapp.bat, myapp.app, etc) can be significantly smaller. 50MB is not "nothing" when it comes to asking people to download it (because someone still has to download that application, and way fewer people than you think have as nice an internet connection as you). It also means people have to download a _new_ 50MB executable when the app updates, so from an "on disk size", it's nothing only if this is the only node application someone will ever run. Otherwise this becomes bloat very fast. Like bundling a full JVM with every individual application instead of going "or download this much, much smaller thing when you already have java installed". I don't need eight copies of the JRE, in the same way I don't need eight compiled apps if I already have node.js -- but that's a consideration for people who make applications and are thinking of packaged distribution. Just because you can, doesn't mean you should. Or even "should only".
You need to download the runtime at least once. If that 50MB is a problem, you can't use it in the first place.
Who likes to download Java on desktop anyway? It's absolutely the wrong way to distribute Electron apps.
It's so annoying for App developers that they have to ask what version of shared Electron the user has and install that for themselves and debug instead of just be done with it with the exact version developers tested and distributed.
Perhaps. JIT compiler has to care deeply for compilation time. For AOT compiler compilation time is a little bit less pressing problem which leaves some space for optimization passes that take too long for JIT.
If you look at Hiphop vs. HHVM there is certainly a significant performance upside in taking advantage of clangs or gccs ability to produce efficient code.
I'm not sure why Facebook decided to stop maintaining hiphop. Perhaps manageability wasn't great.
This would be pretty useful. Something like hiphop which could transpile into C or C++ would mean we (IncludeOS) could build tooling to run a node application on directly in a vm without a underlying OS.
Arguably there are already couple of popular compilers [0] for JavaScript (Babel, Closure Compiler, minifiers etc.). They just don't target native CPU assembly.
[0] Compiler is just a program that transforms code from one representation to another.
But a compiler for what? Plain JS without any APIs? JS that uses the Node.js API? JS that uses the Web APIs? The problem with base JS is that it's a general purpose scripting language that lacks some of the basics that you need before you can compile the code as something that can "do a thing" on its own, without still needing an interpreter for the third party APIs the code tries to make use of that rely on out-of-language hooks (like Node.js)
One major advantage of Electron is not having to deal with browser compatibility issues - you write your code against exactly one runtime (sure, you may need platform specific flags for things - but no risk of the runtime changing/breaking after you deploy)
Leveraging existing OS engine will reduce file size but at the cost of not having a stable and known platform that your code will run on.
I can't help but think that electron could eventually become similar to Adobe Air (except actually open), where the main runtime is/can be installed separately, so multiple apps can share a runtime.
I am wondering if using some of the front-end techniques for back-end would be beneficial. Tree-shaking, minimization and gzip would probably help with the binary file size. But as we go there, maybe some more advanced things like the closure[1] compiler or prepack[2] would make it faster.
> But as we go there, maybe some more advanced things like the closure[1] compiler or prepack[2] would make it faster.
And then one could probably go even deeper in search of performance and instead of compiling JS to JS, compile it to C or even straight to native CPU assembly.
Yeah. Like all those compilers and runtimes for the back-end languages we were already using before JavaScript ate everyone's brains, that served as the inspiration for all of this madness on the front-end side.
> Pkg hacked fs.* API's dynamically in order to access in-package files, whereas Node.js Compiler leaves them alone and instead works on a deeper level via libsquash. Pkg uses JSON to store in-package files while Node.js Compiler uses the more sophisticated and widely used SquashFS as its data structure.
More sophisticated is not a great way to suggest something ks better. That said, I could image this probably does give better performance if you have a lot of I/O with files in the package.
The comparison table fails to mention one huge advantage zeit/pkg has: it is able to generate executables for all three platforms from any one of them, no need to setup three dev or build environments.
83 comments
[ 3.2 ms ] story [ 164 ms ] threadhttps://github.com/pmq20/squashfuse/
As someone who has spent a lot of time with npm (not even as a webdev), it appears to solve all of my common use cases while also dealing impressively well with various versions of specific dependencies. (Like, A depends on B@0.0.1, C depends on B@0.0.2, etc.)
Node module syntax is non standard and can't be tree shaken, with a little more forethought it could have been designed to support static analysis and they wouldn't have needed to reinvent the module syntax for ES6. You can't reliably tell if a piece of code is using a given package until you run it.
The dependency resolution sucks. It's better now that they finally made it so nested dependencies aren't duplicated, but why the hell do I have to run a manual dedupe command to do so? What that tells me is that npm doesn't actually track dependencies, its hardly more than a fancy file downloading tool.
The repository curation sucks. I'm not sure if you've used Python's package manager, or Java's, or C#'s, or Perls, but npm has zero package curation and is filled to the brim with garbage compared to the others. I have published packages for C# and Java and the semantics are well defined and reasonable, you even cryptographcally sign your code and the system checks to make sure you upload documentation etc... In the case of java, which has the best system I've seen, they even use something similar to DNS validation to make sure you control package namespace.
The above point is the absolute worst thing about npm and drawfs the other problems. Let me give an example by pasting the package publishing guidelines for Maven(Java) vs npm:
Publishing an npm package: >Use npm publish to publish the package. >Note that everything in the directory will be included unless it is ignored by a local .gitignore or .npmignore file as described in npm-developers. >Also make sure there isn't already a package with the same name, owned by somebody else. >Test: Go to https://npmjs.com/package/<package>. You should see the information for your new package.
I had to condense the Java guidelines but it's roughly this: create JIRA account apply for access review requirements verify namespace ownership generate crypto keys package coordinates/versioning project description URL and license info developer info SCM info package deployment generate binaries revolve package dependencies crypto sign package generate documentation online verification
Sounds like a lot of work right? It takes a few hours to setup and verify your domain/keys the first time but after that all of these steps are automated. The signing/building/doc generation/ deployment is totally automated. Npm is a toy in comparison.
from their docs: Why Do We Have Requirements? In order to ensure a minimum level of quality of the components available in the Central Repository, we have established a number of requirements your deployment components have to meet. This allows your users to find all the relevant details about the components from the metadata provided in the Central Repository. The following sections will detail these requirements.
Left-pad is a perfect example of why npm sucks. Nuget, Maven, Phython, and CPAN would have never let something that stupid happen.
O RLY?
* https://mvnrepository.com/artifact/coldnew/left-pad/1.0.0
* https://www.nuget.org/packages/left-pad/
* https://pypi.python.org/pypi/left-pad/
* http://search.cpan.org/~dagolden/LeftPad-0.003/lib/LeftPad.p... (LeftPad - Why should Node.js have all the fun? - comments on the package)
-- UPDATE
About package removal...
* http://stackoverflow.com/questions/20403387/how-to-remove-a-...
And by date of project creation you can assume that others are parodies too: "LeftPad - Why should Node.js have all the fun?"
I think it's worth noting that with Java at least, besides not allowing package removal, the namespace is tied to a unique identifier with ownership verification, so the cause of the left-pad fiasco (kik wanting his namespace) is extremely unlikely to happen in the first place.
- dynamic require()'s can't be tree-shaken: same is true of other langs, e.g. Python. Perfectly understandable IMO as Node was designed for the server, in a time tree-shaking (largely desirable for front-end code) wasn't a priority. Also it is a standard (CommonJS).
- long pathnames is a Windows only issue and afaik fixed by npm 3's "flattening".
- npm is worse than Maven because 'npm publish' is too easy?
- you don't need to run dedupe.
- you say npm isn't curated but I'm not aware those other registries are curated either?
- 'left-pad is a perfect example of why npm sucks' ... except the ability to delete packages has been fixed, so should that be 'sucked'?
(Edit: how on earth do you format lists correctly on HN?)
Perfect in that it was fixed over a year ago?
https://github.com/npm/npm/issues/10999
Shrinkwrap does have known issues (mainly w.r.t. platform-specific modules and verbosity), though I've never had any problems with it, but if you want a better lockfile I suggest the Yarn client.
(Believe Python's pip behaves similarly to npm in that issue).
I prefer Yarn's lockfile handling, but saying npm is 'broken by design' is wrong.
And yes, Yarn is the sane option for package management for Node. Why isn't it the default one yet?
If you're taking about deployments, that's definitely unusual. Outside of deployments, what's a scenario where deterministic builds are important, but it would be considered normal to manually install them anyways?
Great, then you should be able to mention at least one solid normal usecase as an example. I don't think there is one.
NPM has issues but most criticism of it is really complaints about bad dev practices (like not pinning versions correctly).
There are literally no worse major package managers.
A good package manager can be misused and still work reasonably well; a bad one doesn't even work all that well when used correctly.
https://github.com/npm/npm/issues/9633
I especially like the part where it says "This is most likely not a problem with npm itself", even though it clearly is a problem with npm itself (as evidenced by non-deterministic nature of the bug).
Again, bad dev practice.
https://github.com/bloomberg/chromium.bb
https://github.com/weolar/miniblink49
The latter is around 9MB. The author is working on a electron wrapper.
Blink is a fork of webkit, by your logic, most softwares based on webkit can not execute javascript???
This information is readily available on, e.g. Wikipedia, so I'm not sure why you even try to argue about it.
Node.js doesn't include Blink, it only includes V8. Node.js Compiler doesn't include Blink into the build result — it includes Node.js.
50MB is nothing and if it enables you do have a single standalone binary that you're easily able to deploy I'd say it's worth the extra disk use.
https://hub.docker.com/r/library/node/tags/
Regular images are ~260MB and "slim" images are 85MB, but Alpine-based images are a paltry 18MB.
So it seems something in the Node.js compiler is ballooning up the size of the code, by a factor of about three. I wonder what.
Who likes to download Java on desktop anyway? It's absolutely the wrong way to distribute Electron apps.
It's so annoying for App developers that they have to ask what version of shared Electron the user has and install that for themselves and debug instead of just be done with it with the exact version developers tested and distributed.
https://blog.docker.com/2017/01/whats-new-in-docker-1-13/
I'm not sure why Facebook decided to stop maintaining hiphop. Perhaps manageability wasn't great.
Edit: IncludeOS does not offer thread support, so it's unclear how the GC of the JS VM is supposed to run.
[0] Compiler is just a program that transforms code from one representation to another.
http://www.damnlol.com/the-adventures-of-technically-correct...
What I'm looking forward to is lightweight electron that will allow us to use the native browser-engine for each platform.
QT/GTK for Linux, WebkitView for macOS, Edge for Windows.
Hope the effort of node-compiler, zeit/pkg will be beneficial for the future of desktop apps.
Leveraging existing OS engine will reduce file size but at the cost of not having a stable and known platform that your code will run on.
Having all those polyfills / babel / webpack tools you can be quite safe about your app.
Nevertheless having a nice test suite also makes a benefit.
Besides what's the point when most people would only use just a few Electron apps anyway.
App developers also have to ask users to download versions between this and that system library only causing confusions for users.
[1] http://www.syntaxsuccess.com/viewarticle/closure-compiler-vs...
[2] https://prepack.io/
And then one could probably go even deeper in search of performance and instead of compiling JS to JS, compile it to C or even straight to native CPU assembly.
https://github.com/zeit/pkg
> Pkg hacked fs.* API's dynamically in order to access in-package files, whereas Node.js Compiler leaves them alone and instead works on a deeper level via libsquash. Pkg uses JSON to store in-package files while Node.js Compiler uses the more sophisticated and widely used SquashFS as its data structure.