As far as I know, one guy made this (https://twitter.com/jarredsumner) working 80h-90h on it a week. His twitter has some cool performance insights into JS and JS engines. It's the biggest Zig codebase too I think.
The benchmark source code links on the homepage are "Not found".
Also a few questions:
What do you attribute the performance advantage to? How much of it is JavascriptCore instead of v8 versus optimized glue binding implementations in the runtime? If the latter, what are you doing to improve performance?
Similarly for the npm client: how much is just that bun is the only tool in a compiled /GC free language versus special optimization work?
How does Zigs special support for custom allocators factor in?
Yeah, the install function of npm/yarn/pnpm are all incredbly slow. And also seems to get slower super-linearly with the number of dependencies. I have one project where it can take minutes (on my 2015 MacBook - admittedly it’s quicker on my brand new machine) just to add one be dependency and re-resolve the lock file. If that can solved by a reliable tool I’d definitely switch!
This is one of if not the most insane thing in web dev at the moment. Git can diff thousand of files between two commit in less time than it takes to render the result on screen. But somehow it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ?
> it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ?
npm is famous for a lot of things & reasons, but none of those are "because it's well engineered".
To this day, npm still runs the `preinstall` script after dependencies have actually been downloaded to your disk. It modifies a `yarn.lock` file if you have it on disk when running `npm install`. Lots of things like these, so that the install is slow, is hardly surprising.
I don't know exactly the "since when", but recently I was caught off guard when issuing `npm i` by mistake in a yarn project. It modifies "yarn.lock" by changing some, if not all, the registry from yarn pkg registry to npm package registry.
People building language tooling often use the language itself, even if it is not very suitable for the task at hand.
This happens because the tooling often requires domain knowledge which they have and if they set out to write tooling for a language they tend to be experienced in that language.
I have just discovered yarn2 / yarn3. The main advantage over npm / pnpm seems be the Zero Install philosophy [1] and the Plug'n'Play architecture [2]. Have you some feedback about these features?
By the way, the yarn2 / yarn3 project is hosted on a distinct repository [3].
> Yarn v2 is backwards compatible though. You just need to use the node_modules "linker" (not the default one) and it's ready to go.
Last I checked, not quite. Yarn 2+ patches some dependencies to support PnP, even if you don’t use PnP. I discovered this while trying out both Yarn 2 and a pre-release version of TypeScript which failed to install—because the patch for it wasn’t available for that pre-release. I would have thought using node_modules would bypass that patching logic, but no such luck.
It seems like bun caches the manifest responses. PNPM, for example, resolves all package versions when installing (without a lockfile), which is slower. The registry does have a 300 second cache time, so not faulting you there, but it means your benchmark is on the fully cached path, which you'd only hit when installing something for the first time. Subsequent installs would use the lockfile and bun and PNPM seem fast* in that case.
If I install a simple nextjs app, then remove node_modules, the lockfile, and the ~/.bun/install/cache/*.npm files (i.e. keep the contents, remove the manifests) and then install, bun takes around ~3-4s. PNPM is consistently faster for me at around ~2-3s.
I'm not familiar with bun's internals so I may be doing something wrong.
One piece of feedback, having the lockfile be binary is a HUGE turn off for me. Impossible to diff. Is there another format?
* I will mention that even in the best case scenario with PNPM (i.e. lockfile and node_modules) it still takes 400ms to start up, which, yes, is quite slow. So every action APART from the initial install is much MUCH faster with bun. I still feel 400ms is good enough for a package manager which is invoked sporadically. Compare that to esbuild which is something you invoke constantly, and having that be fast is such a godsend.
> It seems like the main thing that bun does to stay ahead is cache the manifest responses. PNPM, for example, resolves all package versions when installing (without a lockfile), which is slower.
This isn't the main optimization. The main optimization is the system calls used to copy/link files. To see the difference, compare `bun install --backend=copyfile` with `bun install --backend=hardlink` (hardlink should be the default). The other big optimization is the binary formats for both the lockfile and the manifest. npm clients waste a lot of time parsing JSON.
The more minor optimizations have to do with reducing memory usage. The binary lockfile format interns the strings (very repetitive strings). However, many of these strings are tiny, so it's actually more expensive to store a hash and a length separately from the string itself. Instead, Bun stores the string as 8 bytes and one bit bit says whether the entire string is contained inside those 8 bytes or if it's a memory offset into the lockfile's string buffer (since 64-bit pointers can't use the full memory address and bun currently only targets 64-bit CPUs, this works)
yarn also caches the manifest responses.
> If I install a simple nextjs app, then remove node_modules, the lockfile, and the ~/.bun/install/cache/.npm files (i.e. keep the contents, remove the manifests) and then install, bun takes around ~3-4s. PNPM is consistently faster for me at around ~2-3s.
This sounds like a concurrency bug with scheduling tasks from the main thread to the HTTP thread. I would love someone to help review the code for the thread pool & async io.
> One piece of feedback, having the lockfile be binary is a HUGE turn off for me. Impossible to diff. Is there another format?
If you do `bun install -y`, it will output as a yarn v1 lockfile.
GitHub (disclosure: where I work) does respect some directives in a repo’s .gitattributes file. For example, you can use them to override language detection or mark files as generated or vendored to change diff presentation. You can also improve the diff hunk headers we generate by default by specifying e.g. `*.rb diff=ruby` (although come to think of it I don’t know why that’s necessary since we already know the filetype — I’ll look into it)
In principal there’s no reason we couldn’t extend our existing rich diff support used for diffing things like images to enhance the presentation of lockfile diffs. There’s not a huge benefit for text-based lock files but for binary ones (if such a scheme were to take off) it would be a lot more useful.
Any way to use `.gitattributes` to specify a file is _not_ generated? I work on a repo with a build/ directory with build scripts, which is unfortunately excluded by default from GitHub's file search or quick-file selection (T).
Does this really work for jump to file? (we're not talking language statistics or supressing diffs on PRs, which is mostly what linguist readme is talking about).
> File finder results exclude some directories like build, log, tmp, and vendor. To search for files within these directories, use the filename code search qualifier.
(The inability of quick jumping to files from /build/ folder with `T` has been driving me crazy for YEARS!)
Correct me if I'm wrong, but checking those two files:
I don't see `/build` matching anything there. So to me this `/build` suppression from search results seems like controlled by some other piece of software at GitHub :/
I checked and you're right: The endpoint that returns the file list has a hardcoded set of excludes and pays no attention to `.gitattributes`.
I think it's reasonable to respect the linguist overrides here so I'll open a PR to remove entries from the exclude if the repo has a `-linguist-generated` or `-linguist-vendored` gitattribute for that directory [1]. So in your case you can add
build/** -linguist-generated
to `.gitattributes` and once my PR lands files under `build` should be findable in file-finder.
Thanks for pointing this out! Feel free to DM me on twitter (@cbrasic) if you have more questions.
Of course, I can't say for sure that he looked at the fastest possible way to parse json here, but my intuition would be that if he didn't, it's because he had an educated guess that it'd still be slower.
You don't need to go straight to simdjson et al, something like Rust serde which desierializes to typed structs with data bllike strings borrowed from the input can be very fast.
Nobody is arguing that JSON is equally as performant as binary formats. What the others are saying is that the amount of JSON in your average lock file should be small enough that parsing it is negligible.
If you were dealing with a multi-gigabyte lock file then it would be a different matter but frankly I agree with their point that parsing a lock file which is only a few KB shouldn’t be a differentiator (and if it is, then the JSON parser is the issue, and fixing that should be the priority rather than changing to a binary format).
Moreover the earlier comment about lock files needing to be human readable is correct. Being able to read, diff and edit them is absolutely a feature worth preserving even if it costs you a fraction of a second in execution time.
> I agree with their point that parsing a lock file which is only a few KB
You mean a few MB? NPM projects typically have thousands of dependencies. A 10MB lock file wouldn't be atypical and parse time for a 10MB JSON file can absolutely be significant. Especially if you have to do it multiple times.
> Being able to read, diff and edit them is absolutely a feature worth preserving even if it costs you a fraction of a second in execution time.
You can read and edit a SQLite file way easier than a huge JSON file.
Still don't understand whhy we even need all these inodes.. The repo is centrally accessible (and should be read-only btw). Resolving that shouldn't be a problem. It's been more than a decade and npm is still a mess.
On Linux, not yet. I don't have a machine that supports reflinks right now and I am hesitant to push code for this without manually testing it works. That being said, it does use copy_file_range if --backend=copyfile, which can use reflinks.
Congratulations for the release! You are doing an impressive work with bun. I find particulary exciting the built-in sqlite, and I cannot wait to move all my projects to bun. Egoistically speaking (my 2012 mbp doesn't support AVX2 instructions), I hope that now that the project is public, since you are going to get a lot of issue reports about the failure on install, you will find some time to get back to issue#67. Thank you, and keep up the excellent work.
I'm ultra excited about Bun being finally open sourced, congrats on the amazing progress here Jarred!
Since JSC is actually compilable to Wasm [1] and Zig supports WASI compilation, I wonder how easy would be to get it running fully in WAPM with WASI. Any thoughts on how feasible that should be?
I've been following Jarred's progress on this on Twitter for the last several months and the various performance benchmarks he's shared are really impressive:
Easy to dismiss this as "another JS build tool thing why do we need more of this eye roll" but I think this is worth a proper look — really interesting to follow how much effort has gone into the performance.
someone demonstrated a while back how based on user agent you could serve innocuous code for a browser checking the code first, and then a different malicious payload for curl.
Why? If it's over TLS you can trust it's being served by the owner of the website. You're having to trust the person who wrote the script anyway. And before anyone says "I'm going to inspect the shell script before I run it", do also note that its purpose is to download and run binaries that you are never going to inspect.
Even if you do compile the kernel yourself, do you really own your OS if you haven't compiled the compiler yourself? Did you use a pre-built compiler binary to compile the compiler?
Now we're getting to the real questions in life. :)
(Incidentally, this is probably the most fundamental software supply chain attack vector - manipulate the compiler binary used to compile the compiler used to compile the kernel and userspace. The attack payload would never appear in any sources, but would always be present in binaries.)
IMO the main problem is that it isn't clear how updates will work. Some of the curl-to-bash scripts don't do anything in regards to updates at all, some add a PPA/similar on ubuntu/debian/fedora/etc.
It'd be nice to know what and how I should manage updates.
You could add additional security to the process by first validating some cryptographic signature or verifying that the downloaded content's hash matches one that the author published.
Both of those just push the overall security a bit down the line, but both are ultimately not completely safe. The only truly safe action to take is to not download it at all.
The other thing to be careful about is that the script is written in a way that a truncated script has no effect. This is because sh will execute the stdin line-by-line as it reads it, so if the download fails in the middle of a line the downloaded fragment of that line will be executed.
It can be the difference between `rm -rf ~/.cache/foo` and `rm -rf ~/`
The standard way to solve this problem is to put the entire script inside a function, invoke that function in the last line of the script (as the only other top-level statement other than the function definition), and name it in such a way that substrings of the name are unlikely to map to anything that already exists. Note that the bun.sh script does not do this, but also from a quick glance the only thing that could noticeably go wrong is this line:
rmdir $bin_dir/bun-${target}
A truncated download could make it run `rmdir $bin_dir` instead, in which case it'll end up deleting ~/.bun instead of ~/.bun/bin/bun-${target}, which is probably not a big deal.
But this technique lets you serve malicious code to a small number of people using curl|bash, rather than hosting obviously-bad binaries that anyone can inspect and call you out on. It also lets you target the attack to specific users or IP blocks.
"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead."
Congrats! Cool to see a new class of JS runtimes springing up. Lots to be excited about here, but cold start time seems like a game changer for building at the edge.
From the page: “Why is Bun fast? An enourmous amount of time spent profiling, benchmarking and optimizing things. The answer is different for every part of Bun, but one general theme: zig's low-level control over memory and lack of hidden control flow makes it much simpler to write fast software.”
ESbuild is much more mature than bun. The author of ESbuild cares a lot about compatibility with other bundlers and stability. Moreover it is already insanely fast. I am not sure there is any interest to switch from ESbuild to bun for bundling or transpiling code.
By the way, I think that bun does not apply the patches of ESbuild since the translation date.
I've been following this project for a while now, and it's incredibly ambitious. It will take a while to reach its full potential but Jarred is doing an extraordinary amount of very good work on it, and I think there is a very good chance this will turn out to be a pretty big deal.
I'm really excited about bun – it represents an opportunity to deeply speed up the server side javascript runtime and squeeze major performance gains for common, real-world uses-cases such as React SSR.
Also, Jarred's twitter is a rich follow full of benchmarks and research. If nothing else, this project will prove to be beneficial to the entire ecosystem. Give it some love.
Faster startup -> lower serverless bills for a lot of shops. Shaving 25ms off a request might not seem like it's important, but if that saves you a million serverless seconds in aggregate, you just earned a bonus.
I did a quick comparison for my own reasons of using them as an "absolutely stupid runner" which boots a fresh VM, runs some JavaScript that converts a piece of JSON, and gets out (this is likely mostly measuring VM boot and cleanup only). Bun was crazy fast: a factor of 2 over libmozjs, and a factor of 3 over nodejs
Not to be a cynic, but I wonder how much of the motivation to create a competing runtime in recent months is in response to the eye gauging ( I know, I know. It’s all relative ) tens of millions in funding Deno just raised.
I don’t intend this as a knock on this project. Competition is good and, this space, unlike the rest of JavaScript, could do with more players. There are some promising numbers and claims here. I hope it works out.
I’m genuinely posing this intellectual question of financial incentive as an theory for JavaScript fatigue as a whole.
High profile threads on JavaScript fatigue trend on HackerNews multiple times a week. The wide range of theories about why web developers reinvent the wheel strangely leave out the financial incentives.
Everyone claims their framework is the fast, powerful, light, batteries included, modern, retro futuristic, extensible, opinionated, configurable, zero config, minimal (3kb, gzipped minified, of course ). The list goes on. A few days ago, I was chatting with someone how all these JavaScript libraries make these words have no meaning. To demonstrate, I screenshared a bunch of landing pages. At this point, I haven’t exhaustively, in one sitting, cross referenced these landing pages. 90% of the libraries shared the same layout. 3 columns / cards with one of those hyperbolic words.
Previously I thought it was pretentious and weird that Svelte marketed itself as “cybernetically enhanced web apps”. What does that even mean? Then again, none of the descriptor words like light, dynamic, and ergonomic mean much. At least Svelte was memorable.
Occasionally, one of these libraries would describe their interface as being ergonomically designed. As if other developers designed their interfaces to not be ergonomic. It’s like how we’d all like to think we’re nice, good, decent people. The majority of people would not do bad stuff if they perceived it as such.
I do think most JavaScript developers have good intentions. Then I’ve also seen DevRel / Evangelist types who shill their library, with disingenuous benchmarks, knowing full well there are better solutions, or that they can help make existing solutions better, to everyone’s benefit. The spoils include consulting, training, speaking fees, books, swag, collecting Patreon ( there are some controversial projects which come to mind ), resume building, GitHub activity social capital ( I’ve talked to some recruiters who were fixated on the idea that you publish code on GitHub, specifically Github, because to them, VCS=Git=GitHub, or it doesn’t exist )
Jarred's been working on bun for over a year. I don't get the sense that it's a reaction to anything recent at all, Jarred is just super passionate about building the best JavaScript runtime.
The fact that this project uses Zig suggests to me the developer is talented , passionate, and willing to challenge the status quo.
When you choose a lesser known language like that to tackle a hard problem, chances are you are confident in yourself and the language.
The problems I pointed out with the JavaScript ecosystem as a whole is that it’s low hanging fruit. It’s not that there aren’t financial opportunities elsewhere, outside JavaScript. There definitely is. But the perception of financial incentives is the low hanging fruit, plus high reward.
In this case, it will boil down to how much Bun innovates versus just being a thin wrapped around existing solutions. And again, I don’t doubt this. Skeptical, in general, but not ruling Bun out.
These are legitimate concerns. Looking at the history of bun and its current homepage the main point is that it offers a dramatic speed improvement (plus misc. quality of life stuff).
At this point the ball is in your (and everyone else's) court to put these claims to the test. It should not be terribly hard to see if the speedup is worth your while or not, JS surely doesn't lack bloated projects that you can try to build. My own personal blog is a bloated Gatsby mess that takes half a minute to build.
That's the one true part of the experience that nobody can falsify.
> Replace npm run with bun run and save 160ms on every run
Maybe you can’t falsify this, but it’s a question of risk vs reward.
It’s currently at 0.1 release. Chances are it has a much higher chance of breaking. And when that happens, it would likely take occupy way more time to debug than the hundreds of ms saved.
Also by being new, it means it has not had a chance to cover all the cases yet. That’s the devil. It’s fast now, but it’s an apples to orange comparison until Bun is at a stable release.
Yes, making up your own mind with first-hand experience requires investing time and effort, that's why people like to have other people tell them what to think.
I think some creators pursue products as ways to fund their passions. For example, was deno deploy always the endgame or is it just a way to fund working on deno? Aside from motivation, how it's implemented and how it affects the community matters.
This part, very early on in the Bun page, stood out. That’s a monetizable product, even if the code is open source. That to me felt like positioning itself as a potential drop in replacement for Deno Deploy / Edge Functions.
There was serverless, and now the next trend is with edge computing. It’s already happening, but now specifically about runtimes on that edge.
It actually seems more vendor agnostic in description. It's an overall advantage of bun, but there isn't any first-class bun service pitched (at least at this point).
I'm also pretty cynical of most JS rebuild/reinvention projects. I'm very tentatively excited by this one _because_ it looks like all it does is incrementally improve. Having something that is a drop-in API compat replacement for yarn 1/npm and node makes it potentially really easy to get the benefits of incremental perf improvements _without_ needing to reinvent the wheel like yarn 2 or deno.
100% this. Being compatible with nodejs API makes it possibly useful, unlike some other projects which want to throw away the huge npm ecosystem. Why on earth would anyone use JS (or even TS) server side if not to benefit from the ecosystem? Unlike on the web, there are plenty of better languages to use if you don't want npm.
For me and possibly other full stack devs, because I don't want to stay current in two different languages. Using python and JavaScript sucked. Switching to Node and JS was much better.
Deno showed there is space for not-Node, and that developers would be receptive to this.
And yes, Deno is just one player in edge, but you can agree there is much more money involved with all those other players you listed.
It’s going to be a battle of eyeballs from those edge providers then wouldn’t it? Whether that’s consulting or licensing fees, or just an acquisition / acquihire player.
Maybe you’re suggesting these players would build a runtime themselves. From my experience, only a fraction of companies, rarely, tackle ambitious projects like this. It’d be hard to justify to management who need quarterly results. Instead, they’ll fork an existing technology and make it better, because you can show incremental progress but keep your baseline. For example, Facebook didn’t rewrite their PHP code right away. They wrote a faster interpreter.
I wouldn't agree that Deno showed that, as I said many companies are making a lot of money from non-Node JS runtimes.
The players I mention have built their own runtime, they're mostly all built on V8 isolates (including Deno Deploy).
This is why I struggle to see where Bun fits in the edge JS world, as far as I understand it JSC has no Isolate primitive meaning Bun would have to write this from scratch (or salvage the other parts of WebKit that offer isolation). Otherwise Bun will be limited to using Linux containers on the edge, at which point you re-introduce the startup time you gained by switching from node in the first place.
Someone suggested that Deno Deploy might not be using isolates as isolation boundary per account but processes instead (though, Deno Deploy may be using isolates to run different functions part the same account).
This is super cool and if I was working on more personal projects I would be tempted. In an enterprise context, moving to a reimplementation of core Node APIs is a terrifying prospect. There are infinite possible subtle problems that can appear and debugging even within official Node has been a challenge in the past. I don't know how this concern can be alleviated without seeing proven usage and broader community acceptance.
It is scary, but it already seems more stable and backward-compatible than Deno. With some testing and further stabilization, I have a feeling bun might be a much more feasible and beneficial move.
It's not backwards compatible, it's just Node compatible. Demo is not, and it's stated as much clearly. Thus the suggestion that bun is more backward-compatible than Deno is incorrect and speaks to a fundamental misunderstanding of the two projects.
Sure, a poor choice of words. Node compatibility is what matters to me (and likely many others) practically as a node developer. Deno's benefits don't outweigh the costs of migrating to it and adopting it more broadly, in my opinion.
tried deno briefly the other day, was a real pain. just wanted to hash some files. they had an std hash lib but dropped it in favor of web crypto which bazzarely doesn't have a hash.update method for streaming?? not very good. prefer node's impl
At the last company I worked at I only used C# + WPF (it was horrible). A couple jobs ago I only used R. There are companies with entire divisions that never have to touch javascript, and I'm certain there are companies that never use it. There's a very large world of programmers working for insurance/Healthcare/embedded/military/government that is nothing like modern web dev.
The open source JS ecosystem, though, has soooooo many folks working on it that you can almost always find something you need. Certainly compared to something like Java, which has a couple "800 pound gorilla" projects, but then it falls off pretty quickly.
Of course, this is very much a double edge sword, with all the "leftpad" type dependency nightmares and security issues, as well as the "Hot new JS framework of the month" issues. Still, I think the dependency issues are solvable and dependency tooling is getting better, and the frameworks issue has calmed down a bit such that it's easy to stick to a couple major projects (e.g. React, Express) if so desired, but more experimental, cool stuff is always out there.
Well those gorillas are really hard to maintain. Suppose you found that it wasn't quite what you wanted. Would you poke your head into the Kafka codebase?
Java Gorillas are far, far easier to maintain than JS mayflies. Static typing, top-notch refactoring tools, excellent IDE's with deep language analysis, stable ecosystems, minimal dependency chains, better performance, better profiling tools, trustable repositories with well-supported choices for self-hosting - the list goes on and on.
PS: I have already poked my head into the Kafka codebase in the past. Not the best written project and also confusing because of the Scala mix, but far more readable than several I have seen. And Java makes it easily navigable. Can even auto-gen diagrams to grok it better.
My point was that JS has its 800 pound gorillas too, and so if you only want to use that, you can.
But it has so much of a broader ecosystem of other tools that if you're willing to take that risk it's an option. Java basically just doesn't have that.
Yeah definitely a double edged sword. So much of the JS ecosystem is crap written by keen beginners who don't know what they're doing.
You might say "so what they're doing it for free, you don't have to use their stuff", but often you do because the existence of a sort-of-working solution means that other people are much less likely to write a robust solution for the same problem. So everyone ends up using the crap one.
Note that the bun install [1] seems to be hosted as an HTML file, not as a text file. I'm not sure to what extent that causes issues, but it seems atypical.
Definitely beta, but really awesome to see someone working on this stuff.
"I spent most of the day on a memory leak. Probably going to just ship with the memory leak since I need to invest more in docs. It starts to matter after serving about 1,000,000 HTTP requests from web streams"
I only added the additional note after the , to clarify that it is still cool stuff, even if it is beta and has memory leaks.
At least in my usecase, I do about 35m hits / day... so this would fall over in less than an hour. 1m isn't that large of a number and the author is willing to shrug that off until after launching.
The benchmark numbers for React server-side rendering are really impressive. How does bun manage to be so much faster, especially considering that React SSR is actually fairly compute-intensive (building up a DOM tree and all)?
Gaming benchmarks is pretty easy if you have a very repetitive task. A good compiler developer should be able to make his or her compiler best everyone else one one benchmark
Bun won my heart because:
- it uses Zig and not Rust
- it uses JavaScriptCore and not V8
- it has built-in support for my favorite database: sqlite
- it is all-in-one
While true, I think there's a lot of "this is better because it's been written in Rust" software around these days. The emphasis is on the language, not the problem being solved.
Memory leaks in rust might be less common than a fully GC'ed language but saying rust prevents or makes them less common than e.g. zig/c++ seems like a stretch.
Leaks are absolutely considered safe by rust. Just look at Box::leak. And if we're comparing it to GC'd "leaks", those are typically hashmaps growning out of control and that can also happen
That's mostly a thing teenagers are doing to learn CS, when it's not it's because the software is vulnerable or slow and benefits from a rewrite in a safe language.
It is effectively bait, irrespective of intent. Why would you say that “this pastry is great: it’s blueberry and not chocolate”. Is the implication that 50% of the enjoyment is on account of what it is not? Just weird.
this is very interesting difference too! can you please tell me what you like more about zig over rust and JavascriptCore over v8. very interesting cause I've always thought v8 is superior to JC cause v8 is chrome and JC = safari and zig is very interesting too
Also some things are wayyy slower on v8 compared to other engines. (I've only checked spidermonkey.) In my experience v8 is faster at running WASM and spidermonkey is faster at running optimized js (not quite asm.js but using some tricks like |0) and starting workers.
Every JS engine has interesting design choices. V8 has more documentation than other (the v8 blog is great resource [1]). This ism ore easy to have an overview of V8 internals. It could be nice to have more internal overview from other engine, in particular SpiderMonkey. It is hard to find up-to-date info.
however that fact that it is NOT using V8 is quite refreshing change. We need diversity in the JS eco system other than google v8. Didn't realize JavascriptCore is faster than v8.
This is the same weak argument that's being used to sell native compilation on Java. Tons of graphics showing startup time improvement, never a word on what happens after. But the only cases where this metric makes a real difference are CLI utilities and Serverless (if you know others, please tell me)
That’s not true, as far as I've read. I feel it is spelled out pretty thoroughly: Startup time and memory footprint are way lower, but peak performance is also lower: JIT outperforms static compilation due to it having more information. There are some tools to gather such information by running the program for a while, gathering stats, but JIT still outperforms.
True. I do allow myself some leeway when I get deeper in the conversation tree but it's still noise I'm adding since HN is much flatter than other forums. I really need an outlet that will fulfill my need for expression but without going full Reddit.
Or maybe I just need physical colleagues and coffee break humor.
JavaScriptCore is a constituent part of WebKit, which is pretty much available on washer-dryers these days. But the macOS/iOS/etcOS framework part of JavaScriptCore that adds Objective-C/Swift layers is only available for those platforms, yes: https://developer.apple.com/documentation/javascriptcore
Unrelated question, maybe someone knows, each time I opened a JavaScriptCore or Webkit source file, i've almost never seen any comments. Is it me and my bad sampling or these codes have almost no documentation at all? That's really uncanny.
328 comments
[ 3.1 ms ] story [ 1250 ms ] threadCongrats on the release :)
https://news.ycombinator.com/item?id=31993615
One of the things I'm excited about is bun install.
On Linux, it installs dependencies for a simple Next.js app about 20x faster than any other npm client available today.
Also a few questions:
What do you attribute the performance advantage to? How much of it is JavascriptCore instead of v8 versus optimized glue binding implementations in the runtime? If the latter, what are you doing to improve performance?
Similarly for the npm client: how much is just that bun is the only tool in a compiled /GC free language versus special optimization work?
How does Zigs special support for custom allocators factor in?
edit: should be fixed
npm is famous for a lot of things & reasons, but none of those are "because it's well engineered".
To this day, npm still runs the `preinstall` script after dependencies have actually been downloaded to your disk. It modifies a `yarn.lock` file if you have it on disk when running `npm install`. Lots of things like these, so that the install is slow, is hardly surprising.
https://github.com/npm/cli/blob/latest/workspaces/arborist/l...
This happens because the tooling often requires domain knowledge which they have and if they set out to write tooling for a language they tend to be experienced in that language.
Is it even a tree? Does NPM still allow circular dependencies?
Yarn v2 is backwards compatible though. You just need to use the node_modules "linker" (not the default one) and it's ready to go.
By the way, the yarn2 / yarn3 project is hosted on a distinct repository [3].
[1] https://yarnpkg.com/features/zero-installs
[2] https://yarnpkg.com/features/pnp
[3] https://github.com/yarnpkg/berry
Last I checked, not quite. Yarn 2+ patches some dependencies to support PnP, even if you don’t use PnP. I discovered this while trying out both Yarn 2 and a pre-release version of TypeScript which failed to install—because the patch for it wasn’t available for that pre-release. I would have thought using node_modules would bypass that patching logic, but no such luck.
If I install a simple nextjs app, then remove node_modules, the lockfile, and the ~/.bun/install/cache/*.npm files (i.e. keep the contents, remove the manifests) and then install, bun takes around ~3-4s. PNPM is consistently faster for me at around ~2-3s.
I'm not familiar with bun's internals so I may be doing something wrong.
One piece of feedback, having the lockfile be binary is a HUGE turn off for me. Impossible to diff. Is there another format?
* I will mention that even in the best case scenario with PNPM (i.e. lockfile and node_modules) it still takes 400ms to start up, which, yes, is quite slow. So every action APART from the initial install is much MUCH faster with bun. I still feel 400ms is good enough for a package manager which is invoked sporadically. Compare that to esbuild which is something you invoke constantly, and having that be fast is such a godsend.
This isn't the main optimization. The main optimization is the system calls used to copy/link files. To see the difference, compare `bun install --backend=copyfile` with `bun install --backend=hardlink` (hardlink should be the default). The other big optimization is the binary formats for both the lockfile and the manifest. npm clients waste a lot of time parsing JSON.
The more minor optimizations have to do with reducing memory usage. The binary lockfile format interns the strings (very repetitive strings). However, many of these strings are tiny, so it's actually more expensive to store a hash and a length separately from the string itself. Instead, Bun stores the string as 8 bytes and one bit bit says whether the entire string is contained inside those 8 bytes or if it's a memory offset into the lockfile's string buffer (since 64-bit pointers can't use the full memory address and bun currently only targets 64-bit CPUs, this works)
yarn also caches the manifest responses.
> If I install a simple nextjs app, then remove node_modules, the lockfile, and the ~/.bun/install/cache/.npm files (i.e. keep the contents, remove the manifests) and then install, bun takes around ~3-4s. PNPM is consistently faster for me at around ~2-3s.
This sounds like a concurrency bug with scheduling tasks from the main thread to the HTTP thread. I would love someone to help review the code for the thread pool & async io.
> One piece of feedback, having the lockfile be binary is a HUGE turn off for me. Impossible to diff. Is there another format?
If you do `bun install -y`, it will output as a yarn v1 lockfile.
If you add this to your .gitattributes:
It will print the diff as a yarn lockfile.Not applicable to GitHub etc though.
I'm also not seeing any speed differences when using -y/yarn lockfile. Why not make it the default?
GitHub (disclosure: where I work) does respect some directives in a repo’s .gitattributes file. For example, you can use them to override language detection or mark files as generated or vendored to change diff presentation. You can also improve the diff hunk headers we generate by default by specifying e.g. `*.rb diff=ruby` (although come to think of it I don’t know why that’s necessary since we already know the filetype — I’ll look into it)
In principal there’s no reason we couldn’t extend our existing rich diff support used for diffing things like images to enhance the presentation of lockfile diffs. There’s not a huge benefit for text-based lock files but for binary ones (if such a scheme were to take off) it would be a lot more useful.
Here's a test demonstrating that this usage works: https://github.com/github/linguist/blob/32ec19c013a7f81ffaee...
Quoting the docs on finding files:
https://docs.github.com/en/search-github/searching-on-github...
> File finder results exclude some directories like build, log, tmp, and vendor. To search for files within these directories, use the filename code search qualifier.
(The inability of quick jumping to files from /build/ folder with `T` has been driving me crazy for YEARS!)
Correct me if I'm wrong, but checking those two files:
- https://github.com/github/linguist/blob/master/lib/linguist/...
- https://github.com/github/linguist/blob/master/lib/linguist/...
I don't see `/build` matching anything there. So to me this `/build` suppression from search results seems like controlled by some other piece of software at GitHub :/
Also, files from `/build` are not hidden in diffs, so per this table: https://github.com/github/linguist/blob/HEAD/docs/overrides.... they are not "linguist-generated".
I think it's reasonable to respect the linguist overrides here so I'll open a PR to remove entries from the exclude if the repo has a `-linguist-generated` or `-linguist-vendored` gitattribute for that directory [1]. So in your case you can add
to `.gitattributes` and once my PR lands files under `build` should be findable in file-finder.Thanks for pointing this out! Feel free to DM me on twitter (@cbrasic) if you have more questions.
[1] Recursively matching a directory with gitattributes requires the `/**` syntax unlike .gitignore: https://git-scm.com/docs/gitattributes#:~:text=with%20a%20fe...
Considering the speed with which a fast parser can gobble up JSON I'm somewhat skeptical that this would be relevant for common operations.
Yes he did
I don't see it either. Perf data that shows that was the issue.
https://news.ycombinator.com/item?id=31993429
Of course, I can't say for sure that he looked at the fastest possible way to parse json here, but my intuition would be that if he didn't, it's because he had an educated guess that it'd still be slower.
It's just comparison of execution times of several different package manager.
Better would be parsing JSON vs binary in Bun.
If you were dealing with a multi-gigabyte lock file then it would be a different matter but frankly I agree with their point that parsing a lock file which is only a few KB shouldn’t be a differentiator (and if it is, then the JSON parser is the issue, and fixing that should be the priority rather than changing to a binary format).
Moreover the earlier comment about lock files needing to be human readable is correct. Being able to read, diff and edit them is absolutely a feature worth preserving even if it costs you a fraction of a second in execution time.
You mean a few MB? NPM projects typically have thousands of dependencies. A 10MB lock file wouldn't be atypical and parse time for a 10MB JSON file can absolutely be significant. Especially if you have to do it multiple times.
> Being able to read, diff and edit them is absolutely a feature worth preserving even if it costs you a fraction of a second in execution time.
You can read and edit a SQLite file way easier than a huge JSON file.
On Linux, not yet. I don't have a machine that supports reflinks right now and I am hesitant to push code for this without manually testing it works. That being said, it does use copy_file_range if --backend=copyfile, which can use reflinks.
Since JSC is actually compilable to Wasm [1] and Zig supports WASI compilation, I wonder how easy would be to get it running fully in WAPM with WASI. Any thoughts on how feasible that should be?
[1]: https://wapm.io/syrusakbary/jsc
https://twitter.com/jarredsumner/status/1542824445810642946
Easy to dismiss this as "another JS build tool thing why do we need more of this eye roll" but I think this is worth a proper look — really interesting to follow how much effort has gone into the performance.
This type of thing needs to stop
thanks to dementiapatien below for the link https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...
Going through package registries/repositories is a slow process, so obviously want something faster.
Just GitHub releases? Would you be fine if the URL instead pointed to GitHub in that case?
Now we're getting to the real questions in life. :)
(Incidentally, this is probably the most fundamental software supply chain attack vector - manipulate the compiler binary used to compile the compiler used to compile the kernel and userspace. The attack payload would never appear in any sources, but would always be present in binaries.)
https://www.quora.com/What-is-a-coders-worst-nightmare/answe...
It'd be nice to know what and how I should manage updates.
Both of those just push the overall security a bit down the line, but both are ultimately not completely safe. The only truly safe action to take is to not download it at all.
It can be the difference between `rm -rf ~/.cache/foo` and `rm -rf ~/`
The standard way to solve this problem is to put the entire script inside a function, invoke that function in the last line of the script (as the only other top-level statement other than the function definition), and name it in such a way that substrings of the name are unlikely to map to anything that already exists. Note that the bun.sh script does not do this, but also from a quick glance the only thing that could noticeably go wrong is this line:
A truncated download could make it run `rmdir $bin_dir` instead, in which case it'll end up deleting ~/.bun instead of ~/.bun/bin/bun-${target}, which is probably not a big deal.The previous HN discussion said it better than I can: https://news.ycombinator.com/item?id=17636032
`sponge` reads the full input before passing it on
`vipe` inserts your editor inline, so you can view/modify the input before passing it on to bash (change an install directory, etc)
https://news.ycombinator.com/newsguidelines.html
By the way, I think that bun does not apply the patches of ESbuild since the translation date.
This is making me reconsider using the node ecosystem in some of my projects.
Also, Jarred's twitter is a rich follow full of benchmarks and research. If nothing else, this project will prove to be beneficial to the entire ecosystem. Give it some love.
I don’t intend this as a knock on this project. Competition is good and, this space, unlike the rest of JavaScript, could do with more players. There are some promising numbers and claims here. I hope it works out.
I’m genuinely posing this intellectual question of financial incentive as an theory for JavaScript fatigue as a whole.
High profile threads on JavaScript fatigue trend on HackerNews multiple times a week. The wide range of theories about why web developers reinvent the wheel strangely leave out the financial incentives.
Everyone claims their framework is the fast, powerful, light, batteries included, modern, retro futuristic, extensible, opinionated, configurable, zero config, minimal (3kb, gzipped minified, of course ). The list goes on. A few days ago, I was chatting with someone how all these JavaScript libraries make these words have no meaning. To demonstrate, I screenshared a bunch of landing pages. At this point, I haven’t exhaustively, in one sitting, cross referenced these landing pages. 90% of the libraries shared the same layout. 3 columns / cards with one of those hyperbolic words.
Previously I thought it was pretentious and weird that Svelte marketed itself as “cybernetically enhanced web apps”. What does that even mean? Then again, none of the descriptor words like light, dynamic, and ergonomic mean much. At least Svelte was memorable.
Occasionally, one of these libraries would describe their interface as being ergonomically designed. As if other developers designed their interfaces to not be ergonomic. It’s like how we’d all like to think we’re nice, good, decent people. The majority of people would not do bad stuff if they perceived it as such.
I do think most JavaScript developers have good intentions. Then I’ve also seen DevRel / Evangelist types who shill their library, with disingenuous benchmarks, knowing full well there are better solutions, or that they can help make existing solutions better, to everyone’s benefit. The spoils include consulting, training, speaking fees, books, swag, collecting Patreon ( there are some controversial projects which come to mind ), resume building, GitHub activity social capital ( I’ve talked to some recruiters who were fixated on the idea that you publish code on GitHub, specifically Github, because to them, VCS=Git=GitHub, or it doesn’t exist )
The fact that this project uses Zig suggests to me the developer is talented , passionate, and willing to challenge the status quo.
When you choose a lesser known language like that to tackle a hard problem, chances are you are confident in yourself and the language.
The problems I pointed out with the JavaScript ecosystem as a whole is that it’s low hanging fruit. It’s not that there aren’t financial opportunities elsewhere, outside JavaScript. There definitely is. But the perception of financial incentives is the low hanging fruit, plus high reward.
In this case, it will boil down to how much Bun innovates versus just being a thin wrapped around existing solutions. And again, I don’t doubt this. Skeptical, in general, but not ruling Bun out.
At this point the ball is in your (and everyone else's) court to put these claims to the test. It should not be terribly hard to see if the speedup is worth your while or not, JS surely doesn't lack bloated projects that you can try to build. My own personal blog is a bloated Gatsby mess that takes half a minute to build.
That's the one true part of the experience that nobody can falsify.
Maybe you can’t falsify this, but it’s a question of risk vs reward.
It’s currently at 0.1 release. Chances are it has a much higher chance of breaking. And when that happens, it would likely take occupy way more time to debug than the hundreds of ms saved.
Also by being new, it means it has not had a chance to cover all the cases yet. That’s the devil. It’s fast now, but it’s an apples to orange comparison until Bun is at a stable release.
Now we've come full circle.
Especially if I can somehow hook this into my unit tests.
This part, very early on in the Bun page, stood out. That’s a monetizable product, even if the code is open source. That to me felt like positioning itself as a potential drop in replacement for Deno Deploy / Edge Functions.
There was serverless, and now the next trend is with edge computing. It’s already happening, but now specifically about runtimes on that edge.
bun isn't open source, it is source available (at this point in time, at least): https://github.com/Jarred-Sumner/bun/issues/241
Though now you have to stay current in two different runtimes which are subtly different.
Easier, but still a PITA.
And yes, Deno is just one player in edge, but you can agree there is much more money involved with all those other players you listed.
It’s going to be a battle of eyeballs from those edge providers then wouldn’t it? Whether that’s consulting or licensing fees, or just an acquisition / acquihire player.
Maybe you’re suggesting these players would build a runtime themselves. From my experience, only a fraction of companies, rarely, tackle ambitious projects like this. It’d be hard to justify to management who need quarterly results. Instead, they’ll fork an existing technology and make it better, because you can show incremental progress but keep your baseline. For example, Facebook didn’t rewrite their PHP code right away. They wrote a faster interpreter.
The players I mention have built their own runtime, they're mostly all built on V8 isolates (including Deno Deploy).
This is why I struggle to see where Bun fits in the edge JS world, as far as I understand it JSC has no Isolate primitive meaning Bun would have to write this from scratch (or salvage the other parts of WebKit that offer isolation). Otherwise Bun will be limited to using Linux containers on the edge, at which point you re-introduce the startup time you gained by switching from node in the first place.
https://news.ycombinator.com/item?id=31759170
https://github.com/Jarred-Sumner/bun/issues/159
Best of luck.
Of course, this is very much a double edge sword, with all the "leftpad" type dependency nightmares and security issues, as well as the "Hot new JS framework of the month" issues. Still, I think the dependency issues are solvable and dependency tooling is getting better, and the frameworks issue has calmed down a bit such that it's easy to stick to a couple major projects (e.g. React, Express) if so desired, but more experimental, cool stuff is always out there.
Better to depend on numerous 800 pound long-lived gorilla's than a million short-lived mayflies that keep dying.
PS: I have already poked my head into the Kafka codebase in the past. Not the best written project and also confusing because of the Scala mix, but far more readable than several I have seen. And Java makes it easily navigable. Can even auto-gen diagrams to grok it better.
But it has so much of a broader ecosystem of other tools that if you're willing to take that risk it's an option. Java basically just doesn't have that.
You might say "so what they're doing it for free, you don't have to use their stuff", but often you do because the existence of a sort-of-working solution means that other people are much less likely to write a robust solution for the same problem. So everyone ends up using the crap one.
Rust and Go are way way better in that regard.
[1] https://bun.sh/install
"I spent most of the day on a memory leak. Probably going to just ship with the memory leak since I need to invest more in docs. It starts to matter after serving about 1,000,000 HTTP requests from web streams"
https://twitter.com/jarredsumner/status/1543236098087825409
At least in my usecase, I do about 35m hits / day... so this would fall over in less than an hour. 1m isn't that large of a number and the author is willing to shrug that off until after launching.
https://twitter.com/threepointone/status/1543237413190901760
there are projects like uWebSockets.js (and hyper-express [1] built on top of it) [1] which show a 10x increase in throughput.
[1] https://github.com/kartikk221/hyper-express/blob/master/docs...
Should have used Nim, to make it interesting :D
Only RAM manufacturers like memory leaks.
I have never in my life heard anyone say they thought V8 was superior to JSC - what are you basing that on?
Since 2019, it seems like JSC is actually faster (they even beat V8 benchmarks).
[1] https://v8.dev/blog
OTOH, that could at last free Rust from being the shiniest kid on the block. Carry on...
Or maybe that’s just my criteria. Has nothing to do with the subject, and everything to do with what HN is for.
Or maybe I just need physical colleagues and coffee break humor.
The only time I've seen a SEGFAULT in Rust was when using a really badly implemented C library wrapper.
Isn't JavaScriptCore available only in Apple's ecosystem?
Zig is not perfect, Zig is full of sin, in a world full of righteousness, using Zig should be closely guarded.
Could you share what made you choose Zig over V for the project? It looks like both languages would have been an appropriate choice.