62 comments

[ 2.8 ms ] story [ 135 ms ] thread
Are there any tutorials on this so that we can compare it to emscripten?
We will publish extensive tutorials soon, for now you can find some examples on http://leaningtech.com/duetto/examples/

In addition, the full source code of Nontetris ( http://allievi.sssup.it/jacopone/cnontetris/ ) will be published in a couple of days, with a 'guide' on porting a C++ game to Javascript with duetto.

Stefano of Leaningtech

Hi Stefano,

there are four important things required for this to work out:

- List of benefits of your solutions compared to other C++ → JS compilers (emscripten)

- Great documentation and good community support (forum, wiki, dynamic list of projects using Duetto)

- Very good debugging capabilities in the produced JS (Developers need to work with the code, an invisible layer sometimes means a lot of trouble)

- Big tech partners who will do complex projects with Duetto to iron-out the compiler and show "real work" can be done with it

rgds, Kira

FYI: http://leaningtech.com/duetto/examples/ is broken on OSX in Chrome and FF for me, it looks like a mobile page is served which makes the code examples unreadable. It looks good in Chrome on Windows7, but also broken in FF on Windows7. Haven't test other platforms.

Wow: and a minute after I typed this, my Windows machine gave me a blue screen, not sure if it's related but it's the first in years ;)

For games I think you need to show that duetto is just as fast as emscripten when running. emscripten-generated code is very light on the garbage collector (inside emscripten generated code no garbage is produced at all, only in some wrapper code which needs to talk to JS libs), and having the "heap" in a plain typed array usually also gives a performance advantage, since as in C++ you have exact control of the memory layout of your data structures. Not having to construct and destroy JS objects all the time, and keeping all data in typed arrays is essential for performance I think. This is even true without diving into asm.js.
We have written a detailed post a few months ago comparing duetto to emscripten. We also explain why we think it's not a good idea to use a pre-allocated typed array heap: http://leaningtech.com/duetto/blog/2013/05/28/Comparing-to-a...
Disclaimer: speaking strictly from a game-devs perspective: I remember that post and from a game dev's perspective the pre-allocated-memory vs. reclaimed-through-garbage-collection argument is a bit weak IMHO. Games have predictable memory usage, and game devs are used to see memory as a static, finite resource anyway (since that's how it is on consoles). Grabbing a chunk of memory upfront is quite common for games. And as for the (big) size of the pre-allocated memory: either your game needs that much memory at a time or it doesn't, if the user doesn't have that much free you're f*cked either way.

The perf comparison is missing a "asm.js + V8" bar, asm.js code is usually faster then non-asm.js code, even if the JS engine isn't implementing AoT compilation as FF does.

Edit: fixed "...uncommon -> common..." above

From the link:

"Since I do not expect native platforms (i.e. GLibc) to preallocate gigs of memory at application startup to make it faster when dynamic memory is actually used, I do not expect this from a compiler for the JavaScript target either."

So, it would actually make very good sense to have the thing slab allocate a big honking array upfront, and then manage it with brk/sbrk/malloc, and perhaps just add more memory as needed.

Like, if we're going to be writing C/C++ in the browser, why not do this? Why make us deal with the GC at all?

I see your point. Still, I believe the browser environment is different from a console since it is not supposed to monopolize system resources while being used. Managing memory in smaller short lived chunks is more fair for the system as a whole.
Well, it's just about how much max memory you need at a time, no matter whether that size is split into many small chunks or allocated as a single block. True, an application which needs very little memory most of the time, and spikes to huge sizes from time to time is not optimal with emscripten's approach.

But I don't want to nitpick all day. The more options we have the better, keep up the good work :)

You should include a link to the examples in the copy
I'm not sure why you would want something like that, unless you really have a lot of existing code C++ business code you want to reuse, or you really love C++.
I really love C++ and I bet with the currently developing standard (C++11, C++14) more people will start loving C++ again :-)
I've been away from it for a few years and just recently picked up a new project in C++11. I'm loving the improvements.
Have you tried Rust? It's not an entirely stable language yet, but I've been absolutely loving it thus far in my few months of close involvement. C++11 and C++14 are vast improvements, but I reckon Rust is still considerably better than them.

As far as Rust for the web is concerned, I've got pretty big ideas which Rust is instrumental in (I've been primarily a Python programmer hitherto, FYI), and so I'm working on the lowest layer at present, in rust-http (https://githib.com/chris-morgan/rust-http). By the time it's entirely ready for production use, I expect someone to have got Rust working with Emscripten too, and then we'll be ready to do even better than Duetto.

"I bet with the currently developing standard (C++11, C++14) more people will start loving C++ again"

I doubt it. C++11 takes C++ even further from where it needs to be. Every high-level feature is bogged down by low-level annoyances. Do you capture variables by value or by reference? Do you need a weak pointer to break this cycle, or can you use a weak pointer somewhere else (or maybe just not both with smart pointers at all?)? Your destructor has an error to report -- now what?

The reason nobody likes C++ is that it is overly complicated for low-level programming and too low-level and annoying for high-level programming. At best the only thing people can say about C++ is that it is "good enough" and popular.

I like C++ because it addresses problems relevant for building performant applications in the real world. Details matter for performance - the memory hierarchy, cost of small allocations, allocating and accessing aligned data, copying to enable concurrency are all real things.
Details may matter for performance (though in my experience people tend to gravitate towards such things while ignoring asymptotic improvements that can be far more dramatic), but C++ is annoying even if that is something you are trying to do. C++ has a tendency to insert "surprises" into compiled code -- lots of implicit function calls, copies, etc.

I saw this sort of thing first-hand when I TA'd a data structures course taught in C++, when a student came to me asking for help with some very bizarre behavior: a member function was being called for an object that had never been allocated. The problem was that the student had forgotten to write a return statement (why is that even allowed? compilers can obvious detect this, since using -Wall picked it up), and a virtual destructor call had been inserted, but the vtbl for some other object was sitting there and so some other member function for an unrelated class was called.

So while you are sitting there trying to figure out if your class members are aligned optimally, your C++ compiler is inserting function calls and not letting you know about it. If it were just a few implicit calls or copies, it would not be a big deal, but C++ is a huge, complicated language with lots of such things.

Basically, almost everything you mentioned is a low-level concern for which high-level constructs are just a distraction. The point of high-level programming is to direct programmer effort away from such things (hence the implicit function calls and copies, etc.). If you want to program at a low level, fine -- but my point was that in C++, low-level and high-level programming are too tightly coupled.

Finally, it is worth pointing out that this:

"copying to enable concurrency are all real things."

Is not something C++ has any kind of advantage in. I do it in Lisp all the time. You see it in functional programming languages all the time.

The C++ philosophy is to only pay for costs of things that you actually use. Making every check in -Wall an error by default would make every one pay the costs of those extra checks at compile time.
Pray tell us, where does C++ need to be?

All C++ programmers who I have talked to welcome most things in C++11. It simplifies their C++-code in many ways. As an example: specifying how to capture variables was something they were already dealing with when writing functors manually, now they have a shorthand syntax for it.

As for the reason why non-C++ programmers don't like C++, perhaps you are in a better position to say.

Is there a book that teaches best practice C++11 without teaching any of the old cruft?
What an amazingly unhelpful, snarky, useless comment. If you don't like the question or I guess the topic, just skip it.
You're more likely to find up-to-date and useful resources on the 'net then in a dead-tree form.

Moreover, you aren't likely to be able to write any meaningful code without cruft, because you'll have to use existing libraries, where that cruft lives.

Giving you a reference to writing C properly--something most folks still don't do--is actually somewhat helpful as opposed to pure snark.

Does C++ have a fixed ABI yet? Is there a fixed name-mangling scheme across compilers? How do I bind other languages to the lambdas?

No, its just pure snark. C and C++ are long different. I love K&R[1] C, but is really doesn't show properly written C unless some update occurs that addresses the newer standards and all the fun memory issues.

> Does C++ have a fixed ABI yet? Is there a fixed name-mangling scheme across compilers? How do I bind other languages to the lambdas?

Pretty useless for the purpose of my question. I was asking a simple question, your just in it to spread snark.

1) The original edition was one of my favorite books and what I used to learn C back in the day.

"Pray tell us, where does C++ need to be?"

In my honest opinion, C++ needs to separate low level concerns from high level constructs. How variables are captured in lexical closures is a very low-level concern, but lexical closures are a high-level construct -- by mixing in such low-level details, the overall utility of closures in C++ is reduced. This is made even worse by the fact that the programmer must manually maintain the lexical environment of the closure -- and the new "smart pointers" are only marginally helpful, not nearly as robust as a modern garbage collector (which can, oddly enough, improve performance).

C++ could be a really great language if these things were better separated. I write my code in Lisp, and when I have low-level needs I usually have to write some C code and use an FFI. It gets the job done but it is awful to debug and there is a performance hit. C++ is in a much better position: you can write low-level code, or you can write high-level code, or both, and no language barriers need to be crossed (reducing costs in various places).

"As for the reason why non-C++ programmers don't like C++, perhaps you are in a better position to say."

Well, aside from what I mentioned above, I can give you one more: extending the language. You have a few narrow ways to do it, and then you are just stuck. Operator overloading is great, but what if you want to add a new operator? What if you want to change how your type checker works, so that you can have a dependent type system? There are a few hacks that people can do with templates, but on the whole adding a new feature basically means rewriting your compiler (see e.g. what it took to get support for lexical closures), and C++ compilers are pretty hard to write in the first place. This is something of a Lisper gripe, though there are a few languages beyond Lisp that allow programmers to add new syntax or semantics.

I love C++ to death--it gives me a level of control I need to get at very small fiddly bits when I need to, and also to abstract away great swathes of things when I want to.

That said, I very much agree with your point--the middle ground between those two modes is cluttered and there lies madness.

The problem with the new features of C++11/14/42 is that they are never going to remove the existing jank of the language; these folks don't seem to understand--or want to understand--that there is never going to be a magical fucking tabula rasa onto which they'll scrawl their code, unless they want to rewrite all the things from scratch (which may actually be a good idea).

That being the case, they're going to get run circles around by people who either stick to existing jank and libraries or use a language with a better set of batteries and more flexibility (Ruby, Python, etc.).

The power and speed of C++ is not worth the difficulty, usually, of finding people who can use it properly.

Ownership is something that is central in C++. Programmers who come from languages where they can rely on a garbage collector probably find it odd that they would have to specify how variables are captured. For a C++ programmer, it is natural to think about who owns something and in which scope it lives.

To generalize a bit: a lot of the things that might seem odd to non C++-programmers are perfectly natural to C++-programmers.

It's not often that I hear that C++ is too narrow. Being able to add new operators and changing the way the type checker works both seem like niche things and something that is more fit for a research language.

> Is there a book that teaches best practice C++11 without teaching any of the old cruft? http://www.stroustrup.com/4th.html
This might sound like an odd question, but what font are the code examples in?

I had an older version of the book that had all the code examples in a proportional italics font and it was some of the toughest reading I have done.

> As for the reason why non-C++ programmers don't like C++, perhaps you are in a better position to say.

As an ex-C++ programmer who did in fact do a large-scale web application deployment in C++ back in '99, and who used to do template meta-programming for fun:

It's too verbose.

C++11 does go some way to alleviate that, and might slow the number of people leaving C++, but it's unlikely to bring many people back, as the performance difference matters less and less, and the performance of language implementations for the languages people have jumped ship for (Ruby in my case) keeps getting better, narrowing the gap (and we already decided we could tolerate the much larger performance gaps of the past when we left in the first place).

Does this thing even support C++11? I see no mention of it on the home page...
It fully supports C++11, part of the machinery for client/server seamless programming is based on C++11 capabilities - Alessandro of Leaningtech
D has all the features C++ should have, and implemented in a sane way.
Maybe you have a project that you want to target both native platforms and the web. This becomes an option to let you do so with C++, in addition to being able to do so with Javascript. Maybe you don't like C++ personally, but more options is rarely a bad thing.
Being able to use the web as "just another platform" and use a single unified codebase across all platforms is a killer feature, especially for games. Also, maybe I'm old-fashioned, but I'd rather do a team-project with a "million-lines-of-code" in a compiled language with static type system which gives me compile and linker errors if someone adds a new function arg or changes the type of an arg. Of course also with other "compiled-to-JS" languages, but the killer-feature is multi-platform from the same codebase.
Or want to run an app on very few servers instead of hundreds.
Existing code and talent and probably it. It might sound very silly, but if someone does a COBOL[1] or RPG IV to Javascript translator there would be a demand.

1) There is at least one company located in MN whose website is done in COBOL.

That cnontetris example seemed to be having dire effects on Firefox's memory usage. It was using 21% of my 4GB, but then when I started that it went jumping up to over 80% and back down again and back and forth. Is it really causing the allocation and freeing of over 2GB of RAM just like that every so often?

Oh, by the way—the game was entirely unplayable for reasons likely to be related to this memory thrashing.

Our memory management approach is still not tuned and optimized. There are quite a few low hanging fruits that should provide great improvements, but we have not yet implemented them - Alessandro of Leaningtech
Would you mind if I suggested that arbitrarily using a couple of gigabytes of memory like that was a fairly severe problem? :-)

Seriously, I would like to see a warning put on links to that example to the effect that at present it will make your browser use up to 2½ GB more RAM and will thrash your memory. Because it did render my browser (and to a lesser degree my system) unresponsive for a couple of minutes while things swapped.

You are absolutely right, it is a severe problem and I apologize for this. We are adding a warning on our examples page right now.
i think you're out of memories brah:

    Fatal error: Out of memory (allocated 21757952) (tried to allocate 7680 bytes) in /var/www/techblog/wp-content/plugins/disqus-comment-system/lib/api/disqus/url.php on line 55
It's a slow morning, so this was good for a chuckle:

" Bring the robust­ness and proven scal­a­bil­ity of C++ pro­gram­ming to the Web "

Ha. Ha ha. Ha ha ha.

A bit unrelated, but does anyone know at what comment score one gets to downvote non-constructive comments, or is that what the flagging is for?
Downvoting comes around 500 maybe? Not actually sure.

Anyways, lest you think my observation is just a "non-constructive comment":

The great thing about the browser is that it is free of a lot of garbage we have to deal with in C++. The antics required to make C++ work in the browser, while laudable, are absurd.

If you need cross-platform code, write in Java and avoid native extensions.

If you need cross-platform code that has to be fast (i.e., you actually need to have inline assembly), write in C or C++ and actually do your proper cross-platform work (proper abstraction of I/O, proper defines, proper selection of types, etc.).

If you need to draw shit in the browser, use the wonderful ecosystem of tools that have evolved using the standard affordances of Javascript and CSS.

Here are the bits I take issue with:

"write web appli­ca­tions in C++, reusing exist­ing code and mak­ing port­ing of whole appli­ca­tions and games to the browser plausible."

Why? What type of code is worth reusing? What type of performance is this going to get me?

Most C++ codebases are janky and sad--game engines being some of the worst offenders.

" code both the fron­tend and the back­end of a web appli­ca­tion in the same lan­guage and codebase "

Why the fuck would you ever want to use C++ as that language? It's terrible, so much so in fact that it lost out to PHP and Perl.

Let that sink in for a second.

People said, "I would really like a language that's easier to work in!" and then picked PHP and Perl. PHP and Perl.

Moreover, go look at their docs:

http://www.leaningtech.com/duetto/examples/

" Duetto does not attempt at emulating numerical types which are not supported by JavaScript float is supported with 64-bit precision, like double. long long is defined as a 32-bit integer. 64-bit integers are not supported at all in JavaScript and duetto does not attempt to emulate them."

Well thank Christ we never will need numerical types not in Javascript--I mean, I've never used bit-twiddling in my engine code, or my decompression code, or my audio code.

~

Look, it's a really cool use of LLVM, but we don't need C++ in the browser.

Have you ever had to maintain a code base with more then a few hundred thousand lines of code across Flash, Javascript, iOS, Android, PC, OSX, Xbox, PS, Wii, Vita, NintendoDS, ...and whatever other gaming platform is popular at the moment? The "popular platform of the day" changes a lot faster then being able to write new code for it. You need to support that new platform within 3 months or less. Good luck rewriting your entire code base into another language in that time ;)
http://fabiensanglard.net/anotherWorld_code_review/index.php

This was well-solved a long time ago so...uh...yeah I guess? With much worse hardware?

Maybe Lua?

Do you actually have a real example, or were you just listing off all the magical platforms?

EDIT:

Ha, you do some pretty cool work.

Anyways, put more mildly, is there a case you've seen where you actually have to support all the platforms you've mentioned?

(I seem to be making all sorts of friends today. :| )

;) We've been an independent dev studio (Radon Labs) before being bought by Bigpoint, and the only way to survive was to support a wide range of platforms (we weren't big or successful enough to have the luxury to pick our target platforms). So our engine basically scaled from the NintendoDS to PCs. Being able today to just cross-compile our engine to JS+WebGL or mobile platforms(between 150k and 500k of C++ code, depending what modules you link) is very nice. Plus, for browser games, Flash and plugins as a whole are on the way out, so we need a solution to bring our games into the browser without plugins. A solution like emscripten or duetto to compile C++ (or maybe later other languages) to JS with a relatively small (and predictable!) performance hit is definitely "good enough" for us.
Frankly, I've had to maintain a huge C++-only codebase, and that's why I hope to never write another app in C++.

Yours is an argument for a high-level, typesafe, compiled-to-native and compiled-to-javascript language. But IMO the argument falls down when applied to C++, because it is not high-level, it is only marginally typesafe, and it doesn't compile to javascript well at all (tradeoff of either massive performance, or missing basic functionality like 64 bit ints).

It is very possible that no language will fit this role exactly for some time to come. But, to me at least, it is very clear that C++ is NOT the language for the job.

Yes I see your point to a degree, in a way we're working in a sort of "highlevel C++" most of the time at some performance cost: We have strict coding conventions in place which forbid low-level C/C++ stuff in high-level code (no raw pointers, no C-style arrays, no pointer arithmetics, no C library functions, etc...). Plus static code analysis and tons of runtime asserts. I can't remember the last time we had a buffer overflow or pointer-gone-wild. The performance hit of C++ compiled to JS is surprisingly small (1.5x native performance in Firefox, a bit slower in Chrome, but the gap is getting smaller, this is in the same ballpark as strongly typed bytecode languages like C# or Java).
i.e. That "robustness" is why we now use VMs instead of native code everywhere.
As many have noticed the blog is being seriously crushed by the heavy traffic. We are mirroring the content here: http://leaningtech.com/duetto/blog/2013/10/31/Duetto-Release...
snip

EDIT:

After some consideration, I've removed my post. Though the irony is great, the fact is that this is a cool use of LLVM and a pretty interesting thought-experiment made live. I know I wouldn't appreciate the kind of heckling I'd just posted were our roles reversed.

Good work on the technical stuff folks.

Cut them some slack, jeez.
Done.
I really appreciate it. We are doing our best with the limited capabilities of three persons. Thanks a lot. - Alessandro of Leaningtech
You guys are doing great work! Please don't confuse my disdain for the concept of C++ in the browser for a lack of appreciation for your work.
> reducing the limitations and obstacles given by Javascript

What are these perceived limitations? There are fairly standard idioms and pure JS libraries for getting around most problems people find