Towards a JavaScript Binary AST (yoric.github.io)

The article is just reiterating an argument that Mozilla and Facebook made. If you’re saying Mozilla and Facebook are spreading Fear Uncertainty and Doubt with their proposal for a binary JS AST, then you may do so, but please realize that they probably had good reasons to start such a project.

EDIT: Even though I disagree with the parent, it is making some good points (specifically, that WebASM is going to add a GC), so flagging it to death doesn’t seem to be the right thing to do.

Are you saying this comment is false?

> I am not aware of any project that achieves compilation of JavaScript to wasm

I.e., Javascript to WASM is on the roadmap and something we can use in the next few years? Or am I misunderstanding the situation?

As I mentioned in another conversation, if you feel you can contribute to Wasm and make it solve the problems we are solving here, this is great, by all means, please do so.

In the meantime, we are going to continue trying to solve some of the performance problems of the web using the techniques discussed in the blog entry because we are convinced that they are going to work better.

Behold one of the implementational details that emerged out of this language that was indeed "designed in 2 weeks":

  <script type="text/javascript1.0">...</script>

  <script type="text/javascript1.1">...</script>

  <script type="text/javascript1.2">...</script>

  <script type="text/javascript1.3">...</script>

  <script type="text/javascript1.4">...</script>

  <script type="text/javascript1.5">...</script>

  3.  Deployed Scripting Media Types and Compatibility

    Various unregistered media types have been used in an ad-hoc fashion
    to label and exchange programs written in ECMAScript and JavaScript.
    These include:

      +-----------------------------------------------------+
      | text/javascript          | text/ecmascript          |
      | text/javascript1.0       | text/javascript1.1       |
      | text/javascript1.2       | text/javascript1.3       |
      | text/javascript1.4       | text/javascript1.5       |
      | text/jscript             | text/livescript          |
      | text/x-javascript        | text/x-ecmascript        |
      | application/x-javascript | application/x-ecmascript |
      | application/javascript   | application/ecmascript   |
      +-----------------------------------------------------+

> JavaScript versions

> There have been several formal versions of JavaScript.

> 1.0: Netscape 2

> 1.1: Netscape 3 and Explorer 3 (the latter has bad JavaScript support, regardless of its version)

> 1.2: Early Version 4 browsers

> 1.3: Later Version 4 browsers and Version 5 browsers

> 1.4: Not used in browsers, only on Netscape servers

> 1.5: Current version.

> 2.0: Currently under development by Brendan Eich and others.

The link above points to a bit of interesting general discussion about versioning, and isn't as dense as the RFC.

It's one of several cases where backward compatibility on the web trumped cleaning up the language.

Parsing is unavoidable, regardless if the code is interpreted or compiled. This proposal seems to further compress the AST somehow.

Basically the idea is to already preprocess the text source, so that the client can skip two steps on execution: lexical analysis and the parsing of the text source resulting in the AST. The post explains why the authors believe that this is a good idea.

Compiled C is not going to go away any sooner than interpreted Javascript - but a widening of the options available is great as it allows us to focus on developing things quickly and correctly, rather than making decisions based on which particular annoyance you're happy to put up with.

They have an "intermediate code" that they somewhat compile down to. But this is really a more compact version of the source than a binary though. You can see this easily by running a decompiler. Often times the decompiled IL is almost identical to the source. And before the JIT kicks in the IL is typically interpreted.

You get the best of both worlds. Easy to compile, easy to decompile, platform agnostic, compact code, and fast if it needs to be

I think there are two main drivers for fad-ism in software. One is we as developers, are required to chase the popular languages and frameworks to stay current, marketable, and employable. So there is always a sense of "what do I have to learn next, so I can switch jobs succesfully when I need to". Another driver is, developers seek to gain experience by re-doing things from the past. Example, creating new languages, or compiler for an existing language, provides inexperience developers with fantastic new experiences and knowledge. This is desirable.

So yeah, we got fads, and us old timers, or not so old timers with a historical perspective, watch our lawns, but everyone has to learn somehow. Ideally a lot of this would come from an apprenticeship/journeyman system, or a University degree of some sort. But for now it's all we have.

I wouldn't call this "compiled JavaScript". In broad strokes what's happening is you take the result of parsing JavaScript source to an AST and serialize that; then next time you deserialize the AST and can skip the parsing stage.

(Source: I spent a summer working on JS engine and JIT performance at Mozilla.)

At each step developers are trying to create the best thing they can with the tools at their disposal. There will always be a tradeoff between flexibility and ease of use. There's a reason Electron exists — the browser has limitations that are easier to work around in Electron. There's a reason we moved to putting stuff online, we could write it once and distribute it everywhere.

In the last 25 years or so, we actually got something like a pop culture, similar to what happened when television came on the scene and some of its inventors thought it would be a way of getting Shakespeare to the masses. But they forgot that you have to be more sophisticated and have more perspective to understand Shakespeare. What television was able to do was to capture people as they were.

So I think the lack of a real computer science today, and the lack of real software engineering today, is partly due to this pop culture.

...

I don’t spend time complaining about this stuff, because what happened in the last 20 years is quite normal, even though it was unfortunate. Once you have something that grows faster than education grows, you’re always going to get a pop culture.

...

But pop culture holds a disdain for history. Pop culture is all about identity and feeling like you're participating. It has nothing to do with cooperation, the past or the future — it's living in the present. I think the same is true of most people who write code for money. They have no idea where [their culture came from] — and the Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free? The Web, in comparison, is a joke. The Web was done by amateurs.

sources:

http://queue.acm.org/detail.cfm?id=1039523

http://www.drdobbs.com/architecture-and-design/interview-wit...

You develop one app, and it can run as both web and desktop application, or even on mobile with some extra efforts.

Say you want to quickly edit something like a document, you don't need to install Word, just open browser and load Google Docs or Office 365.

And if you find yourself constantly using one web app or website (travis CI, Trello), you can wrap it with nativefier [1] and use it as a desktop app.

[1] https://github.com/jiahaog/nativefier

Compilation and interpretation both have distinct advantages. It's hard to do both (read: it took a while) and each has tradeoffs.

I'm impressed and grateful that I can get compile time errors for interpreted languages, and I'd feel the same about being able to use a compiled lang in a repl.

Regarding the actual topic at hand, this isn't compiled JavaScript, just parsed JavaScript. They're shipping an AST, not bytecode/machine code.

We'll only know for sure once we can start testing in real conditions, though.

If you wish to follow the development of the reference implementation, you can find it here: https://github.com/Yoric/binjs-ref/ . It's very early and the format will change often as we

1/ fix bugs;

2/ remove debugging info;

3/ start working seriously on compression;

4/ optimize.

Facebook can just run a feature detection JS snippet to see if your browser supports the Binary AST, and select the URLs of the other JS assets accordingly. If Binary AST is a thing, I can easily see web frameworks implementing such a format selection transparently.

And hey, if it contributes to people perceiving Firefox as faster than Chrome, I'm all for it. :)

It's going through TC39 (the committee that decides how the JavaScript language should evolve) and recently reached stage 1 (proposals advance from stage 0 to stage 4). I believe that if it reaches stage 4, then that means that all browser vendors have agreed to implement it, so it certainly seems to be their goal.

At a personal level, I feel pretty confident that BinaryJS will help your case. The theory behind the gains was pretty solid before we designed the prototype. The prototype, for me, basically proved the theory.

My personal hope is that by the time we're done squeezing all we can out of this - which includes zero-costing lazy function parsing, and on-line bytecode generation, we can cut the "prepare for execution" time by 80%.

Some context:

https://v8project.blogspot.com/2017/05/energizing-atom-with-...

https://github.com/nodejs/node/issues/13877

What's proposed is structural compression of JS with JS-specific bits to speed things up even more. What's proposed is not compiled JS, in that the original JS is not meaningfully transformed at all. There is a very explicit design goal to retain the original syntactic structure.

OTOH WebAssembly is like a new low-level ISA accessible from web browsers. To use wasm, one does have to compile to it. As the parent here says, compiling JS -> wasm currently requires at least a GC but also much more. To engineer a performant VM and runtime for a dynamic programming language like JS is a time-consuming thing. It is curious to see so many folks think that JS can straightforwardly be compiled to wasm. Currently wasm and JS serve very different needs; and I also don't think JS will be going anywhere for many, many years.

Edit: formatting

Because, surprise, it's an MVP, and wok has already started to bring other languages (including JavaScript) to it.

So. The question is: one-two years from now, when there's support for wasm in javascript, where does it leave binary AST?

The author of Smalltalk/X proposed this for Smalltalk at one of the Camp Smalltalk meetups about 17 years ago. Doing this for Javascript would also facilitate the running of other languages which use Javascript as a compiler target, providing 1st class debugging support for other languages as a side effect.

A JS binary AST has the potential to transparently benefits all websites without any effort on their part if it's handled by a web server module, like GZIP, or a build step. This can remove redundant work that's done for billions of page load for all web sites.

And sadly, I can see the same happening here.

[1] https://www.corsix.org/content/malicious-luajit-bytecode

[2] The overhead will eat any performance gained by using precompiled Lua code in the first place.

As usual, read these numbers with a pinch of salt, they are bound to change many times before we are done.

https://news.ycombinator.com/item?id=15046750

The space optimizations in BinaryJS are to get us back down to the point where we are as good or better than minified compressed JS.

The main goal is to allow for much faster parse times, but to do that without compromising other things like compressed source size.

If you had to have a complicated toolchain to produce gzipped output to get the performance boost, that would create a performance gap between beginners and more experienced developers.

But today, almost every CDN worth its salt will automatically gzip your content because it's a stateless, static transformation that can be done on-demand and is easily cached. I don't see how going from JavaScript -> binary AST is any different.

I would personally prefer it being handled transparently by the middleware or the cdn, which would neatly separate the responsibilities between the browser, the cache+compressor and the http server.

Anyway, one of the reasons this project is only about a file format is so that we can have this kind of conversation about what the browsers and other tools can do once we have with a compressed JS file.

(You might be thinking someone could set CSP to disable eval on their pages, but eval is reasonably safe even in the presence of otherwise-CSP-protected XSS attacks as long as you aren't using code that goes out of its way to eval things from the DOM, and it's more than annoying that the only way to protect yourself from this cache issue would be to disable eval. ... Also, there are some libraries that interpret script commands from the DOM without using eval, so disabling eval doesn't even protect you if you previously hosted one of those javascript files.)

You could have the cdn.chrome.com cache aggressively drop the cache of things greater than a certain amount of time like a day. But then there's a question of whether the requests to the cache are all just wasted bandwidth for the many requests by users to scripts that hadn't been loaded in a day. And the whole system means that website security can be dependent on cdn.chrome.com in some cases. I'd rather just build and host the processed binary AST myself. I already use a minifier; for most people, the binary AST tool would just replace their minifier.

Indeed, one of the challenges was designing a format that will nicely support changes to the language. I believe that we have mostly succeeded there, and I'll blog about it once I find a little time. Now, of course, the next challenge is making sure that the file is still small enough even though we are using this future-proof format. We haven't measured this yet, but I expect that this will need additional work.