I find it hilarious that JS was basically invented overnight, and ever since, it's just been layers of hodgepodge added and held together with hot glue to make it all work. And now it's the most popular programming language in the world, even outside of its original intended domain.
It was the first programming language available to the general public that was easy enough for the public to use without a CS education - and it running on the web made it ubiquitous.
Then SV happened, and the web became Serious Business. Then Node.js happened and Javascript became Serious Business, and with that came the byzantine horror of complexity necessary to get a toy scripting language invented in a week to replace every enterprise programming language in existence because managers could toss a rock out of their office window and it would bounce off of three web devs before hitting the ground.
> It was the first programming language available to the general public that was easy enough for the public to use without a CS education - and it running on the web made it ubiquitous.
I find JavaScript's type system to be horrendous and bug prone. I'd rather operations between two types that don't make sense (like dividing an Array by an Object) to throw an explicit error than to just do some unpredictable shit that causes a subtle bug.
Also, BASIC is probably the easiest language to learn, though this is mostly because it lacks capabilities that create complexities. It's a completely obsolete language.
Most programming languages have underwhelming origin stories and are evolutionary hodgepodges held together with hot glue.
The most likely opposite to "basically invented overnight" is "designed by committee", and is that much preferable?
Origin stories aren't the end of the story either. Humble origins aside, JS has had many of the best minds in many of the largest companies contribute all sorts of feature work and performance efforts across a span of decades. How many languages can claim that many eyeballs and that diverse a set of language developers and engine writers?
Sure, a lot of it looks like a hodgepodge, especially with its mandatory, deep respect for backward compatibility, because no one wants to break an old website. But show me the language with any respect for backward compatibility that doesn't look like a hodgepodge?
Core systems languages like C and C++ started out as intentional hodgepodges. Their macro system was originally its own, different, language brought in from another existing project for some temporary duty, and now enshrined in the language as a forever feature.
Hodgepodges are how languages are built and evolved. That's how we got ourselves here: the natural way. Every programming language on the planet is its own beautiful house of cards, built piece at a time. Don't just look down on JS because it went "mainstream".
Yeah, the "Common" in Common Lisp comes from committee. Lisp itself was a "basically designed overnight" language that became an incredibly popular hodge podge of a million snowflake variants and then eventually two competing attempts to standardize and clean up the hodge podge happened: Scheme (to my understanding a BFDL-like situation of a strong figure building anew, with few backwards compatibility cares) and Common Lisp (a standards committee and then some, with a bunch of backwards compatibility shenanigans to sort out).
(ETA: I am a fan of Common Lisp, for what that is worth, if this sounds too critical. It's a very nice language. Some of what has made it nice was a massive committee and standards effort, very similar to the way JS has continued to evolve under TC-39 stewardship.)
Also yeah, my impression of Java was that it was very design-by-committee.
Per that own history, the first version "Lisp 1" was written as a project in a single University semester. Compared to the many committee years of, for instance, ALGOL, that is "basically overnight" by comparison.
The implementation of the first and relatively primitive version took roughly a year. McCarthy had two full-time programmers and six graduate students working for him.
There is something really special about that cancer.
It's the only language I've written in where no matter what code I'm writing, I can always think "this could be fully generalized and spun out into a package"
You're going to have to wipe out package-lock.json or whatever your lockfile is and recompute from scratch. I tend to squirrel away a backup just in case whenever I touch JS dependencies because I've been bitten by yarn/npm/pnpm bs too many times. Also make sure you wipeout all your node_modules, plus your yarn/pnpm cache. Technically, you should only do dependency manipulation sith p?npm|yarn add|remove, to get lockfile recalculation, but depending on version of those tools there are so many quirks you have to watch out for.
I always approach any library upgrade as a potential disaster that'll result in me having to nuke my entire node setup for that project and rebuild it from the ground up or last known good state. I never had the kinds of problems I have with JS with any other bloody languages and their respective build tools.
Now, using the "engine" and "packageManager" properties in "package.json", you can bind your project to a particular runtime version and package manager.
npm (Node Package Manager)
pnpm being a modified version thereof that adds a compression/content addressible store mechanism to speed things up in a development configuration.
Or
Yarn
...which is yet another dependency management solution cum build tool, which is Maven/Gradle like. Depending on version though there are so many quirks. Both are "staples" of any enterprise scale project I've run into.
I don't have an issue with nodeJS or JS per se. It's the ecosystem and enterprise tooling. I still cannot for certain say those damn build tools don't muck things up in horribly subtle ways, and the experience of debugging the average horrible nest of every dependency known to man has comsumed entirely too much of my life, and the experience overcoming that learning curve I still can't say with confidence wasn't somewhat deleterious to my personal sanity.
I can do it trivially now; and I can appreciate the things you can build with it when doing it correctly. The depth of hole you can quickly end up in though is not to be underestimated.
That and nigh every JS developer I've run into who says they are proficient knows nothing about really debugging it, or hasn't read the core library docs and really means they can use one framework or another.
...Bar like 3. Who you never want to bother, because they are usually handling issues way more important/wide-ranging that warrant their attention than hand-holding you through learning how it actually works.
If you "don't have an issue" with JavaScript but you think the tools you're using are bad because of the "depth of hole you can quickly end up in though is not to be underestimated", then perhaps you should reconsider your decision to continue using them when you're writing JS and why you ever thought they were necessary.
None of those things (package-lock.json, node_modules, or p?npm|yarn) actually come from ECMA-262...
> I've been bitten by (...) pnpm bs too many times.
I'm surprised, the only node package manager that has any semblance of sanity is pnpm.
No flattening BS. Not randomly adding extraneous undeclared dependencies on the dependency tree. Not allowing random dependencies to depend on extraneous undeclared dependencies on the dependency tree.
Of course, then you find out that in the real life people actually use other package managers, and depend on non-deterministic, package flattening behavior and extraneous undeclared dependencies requiring extraneous undeclared dependencies...
The headline makes it feel like you're going to go into the dependency struggles of JavaScript tools, which you do, but not until the end of the article after mentioning struggles with database dependencies and operating system dependencies...
Seems like this is more of a rant against tools upgrading themselves and breaking previous dependencies which causes this kind of domino effect of dependency issues when trying to upgrade the initial tool, not the "JavaScript ecosystem"
I spend vast amounts of time hacking around in Linux with whatever I aim to do. It's almost never easy. There's always this problem then that problem - you have to shave yaks until you're so deep you forgot what you were working on in the first place.
It ain't just JavaScript.
1: how to make a software thing work:
2: follow instructions for (software thing)
3: hmm, wrong instructions, out of date
4: find right instructions
5: do step one of instructions
6: hmmm.... an error find some new software thing to solve this problem, then for this new software thing, goto 2
Programming is problem solving as much as anything else.
Your step 3 is because you're looking at the wrong Google results. Read the manpages that shipped with the distro and the ArchLinux wiki for everything else.
ArchLinux wiki is great but manpages really only answer "what does this command/syscall/etc. do and what are all the options I have when calling it?". People Google stuff and copypasta from StackOverflow because these answers are to questions they actually have like "how do I integrate $x with $y?". The answers may not be good but they are at least to questions that real people have.
Programming is discrete math. You must line up all the functions correctly, or else it won't work.
Alas, the chain of functions you're managing nowadays (3 to 4 layers of primitive on top of primitive on top of primitive, ) leads to a combinatoric explosion of potential layers to encounter a fault.
IMHO because the happy path is easy, but guarding against error conditions, and then either taking action or advising someone of that outcome is horrifically hard. It's hard not only because it requires the experience to know that not every HTTP request is 200, not every method call is a-ok, not every disk has plenty of space, nor permissions to write to whatever, and even with a huge checklist of the way things go toes up, a disciplined library author can have 250 different illustrative return codes that the caller can then throw in the trash, iterate all the way up the call chain and everything has to align perfectly for the actionable error to reach the end user
That especially plays out, in my experience, in programming languages that place the "please check the return code" upon the programmer, as opposed to ones where errors are more "in your face," but the joy of `try { doSomething(); } catch (ex) { /* lolol */ }` makes it all too easy to ship something because It Works On My Machine
Being on the pointy end of your checklist makes me have a great deal more empathy for users of my software, but I am merely one person who thinks swallowing errors is a crime against humanity
One thing I'm quite confident is name based of thing. Naming is very coupled with versioning. A lot of things break if name changes. The world of hash or id based of thing is going to eliminate huge amount of problems (probably bring some of something else problems).
Imagine API consumers have the initial ids of things and remembers (implements) meaning of them, and then you can re-order parameters, change functions name, module names, fields name, even something moves could also be auto migrated on consumer side (or call site).
Also the database and operating system support issues both seem to specifically Ghost ecosystem issues (JS as a language/ecosystem mostly doesn't care what DB you want to use and Node itself runs largely the same on every platform and probably doesn't even know the specific difference between Debian and Ubuntu). This seems a lot more a "Why is the Ghost ecosystem like this?" more than a JS ecosystem problem.
(The answers to that question seem a lot easier to answer: small team, limited testing resources, and they focus on a paid SaaS platform as their business model to pay the bills and that likely comes first ahead of self-hoster's concerns/needs.)
- newest major version of self-hosted software (Ghost) no longer supports MariaDB or Debian
- newest minor version of the same software also depends on the latest LTS Node release, not the second-latest
- that's it
This really just sounds like garden-variety issues with running your own software — annoying, but not representative of or idiosyncratic to the "JavaScript ecosystem".
Its not the newest minor version, its the latest patch release for his version (that they can’t upgrade) that will need to run on a version of node that has stopped getting security updates.
His options are change the entire OS and database or run an old version of node.
But you are correct, this isn’t the fault of the JavaScript ecosystem, its a series of engineering decisions by the ghost team about what they target and support.
LTS versions are generally maintained for two years. Node 16 will continue getting security updates for at least another eight months: https://github.com/nodejs/Release
Why does npm allow people to publish a new version of their app that doesn't support a database that the old version did support? Is that the complaint here?
Complaints about an app that: 1. wants Mysql not MariaDB, 2. stopped supporting Debian packages & only ships Ubuntu packages, 3. complains about an old version of npm (unknown what the author started with), 4. complains about the author using a deprecated & unsupported version of node.js
This is just like one app. None of these are uniquely JS problems & half arent JS problems at all, in spite of the hater-header. In my view, the app didn't bend over backwards to cater to a bunch of tech-whims the user has, and the user is super salty about it. Maybe don't put yourself through such a struggle! This is exactly why we have containers: so you can easy-mode your way through, and never think about most of these problems & things just work. You don't get to take the hard path and then whine that your specific crusty old box isn't still supported with zero effort.
And to blame this on JS ecosystem? Typical. The hatred, the animosity against sprawling techno-diversity is unbelievably dark-side, has roots in a scared authoritarianism. It's just popular to hate on, and the complaints never seem to mirror what an educated JS developer might offer as a critique, of which I think we could/do offer many; it so regularly feels enormously low-signal, marks poorly upon the character of the complainers. I'm for catharsis and pouring out some of your hatred, but gee, so much of the JS anger feels facetiously low-dimensional.
JS- like all rife open source ecosystems- is by definition a bazaar, not a cathedral, and life is not going to be as blissfully simple as you might want. We balance what is possible against what is well defined, and interesting things keep happening when we favor what is possible (bottom-up) above what is well-defined (top-down). Try not to get so tarnished & bitter about it. There's a lot of upside too. Open source is good, actually. Us having done so much, explored so widely, is good, actually.
“Projects change what external tools are supported and other (including packaging/build) dependencies between versions” doesn’t seem to be unique to the JS ecosystem.
OTOH, the JS culture of microprojects makes it more lokely that any given project will have one or more problematically-managed dependencies.
Because it's the product of a collaboration between ~1M beginner programmers, 100k intermediates, and 10k experts. Because the language and runtime are "mushy" - aka "under-constrained", which allows for all the ideas to flourish, even the "bad" ones.
It's a mess, but a glorious one. JS is the first language to really solve the distribution problem, and to some degree it's design (or lack thereof) reflects the trade-offs required to hit the distribution problem out of the park. In hindsight, it was bound to turn out this way!
IMHO the single worst thing about js is the front-end build experience. It's so nice to write build-less front-end! And yet (most of) the community seems obsessed with WebPack and inventing new transpiled languages etc. It's really bad! Not only is it a far, far worse DX but it breaks the thing that makes the web cool, that you can "view source" and figure out how other people made stuff.
The browser itself performs a kind of build step. It happens per script tag, and you can catch "compilation errors" in your own code using window.onerror if you want to.
Believe it or not, using libraries, compiled or otherwise, is not a requirement of either the HTML or Javascript specifications! The single biggest benefit of front-end builds is modularization; however browsers support that natively now (e.g. script type=module), and have for some years.
>no substantial modern application can possibly be built without them.
That's certainly false, since substantial applications predate React, Angular, svelte, etc.
But you have a point. Steve Yegge once wrote that his company wanted to write a word processor in assembly and it seemed like a good idea at first, but then it was like drawing a straight line 100m long using only 2mm line-segments. The major purpose of these libraries is to increase the "length of the line-segments" to something more consistent with the distance you want to cover. The question is whether or not their exists a unit of abstraction in raw js/html that serves this same purpose. E.g. it's a demonstration proof.
> JS is the first language to really solve the distribution problem
Excuse me, what? I just spilled my coffee all over the desk.
Care to explain what you mean with the problem and how js "solved" it? In my view dependencies and distribution are terrible in javascript, but maybe we mean different things.
JavaScript is the only language in which you can write a program and distribute to every device in the world, today, cheaply and reliably. (technically this is an exaggeration - not everything has a browser. but you get the point.)
You don't need to write it though. The average Internet building person would be better off writing in a language which isn't terrible, and only using JS as the bytecode of the web.
Maybe. I think of it as an experiment. You may be right. It may also be that the potential of raw javascript hasn't been fully explored. That's my hypothesis. I don't think jquery was the best it could do. Time will tell who is right!
I don’t believe it. I refuse to believe that someone can become proficient in a technology which forbids runtime errors and doesn’t bitrot, and then think “you know what, I wish my programs would error more at runtime and also refuse to compile because dependencies changed.”
To want more failure is to be insane. I think rather a lot of programmers think they’ve tried alternatives, but in reality have experimented with syntax swaps of what is essentially the same language.
What language are you referring to? It sounds like you have something very specific in mind, because the usual “not terrible” suspects (Python, Ruby, Rust, Go, etc) don’t have the properties you’re talking about.
Ruby and Python are essentially syntax swaps of the same language. Also not sure how practical it is to write in any of Ruby, Python, Go, or Rust and generate JavaScript.
Elm would be an example of a language which is fundamentally better than JavaScript (and is a practical substitute for what most people are writing JavaScript for), but it isn't the only one.
I’ve been using Elm for over five years on personal projects, but it isn’t a replacement for Javascript as a language, just some client side projects and ones where its async ffi is sufficient.
I think you are falling into a trap of thinking there is a “best” in engineering when that’s basically never the case. Ask experienced developers why they prefer Javascript if you want to get a more nuanced view. I almost always reach for Javascript first, especially for server applications.
I can see why you have come to that opinion, but I would actually put myself immediately into tribe 3.
My code is not poetry; it kinda sucks. I’m not a gifted programmer, which is why I make up for my shortcomings with better tools. I use Haskell entirely for the sake of pragmatism [and have in fact bet my businesses on it]. My experience is such that I can build something that just works much more quickly and reliably in Haskell or Elm than I ever could in PHP or Ruby or JavaScript or Clojure[Script]. And I wrote each of these professionally for years.
That's all and good, but you said you can't believe other people could prefer certain languages. I'm saying that you can simply find them and ask them. And not beginners but veteran engineers.
In other words, there's a simple way to disabuse yourself of your position in this comment: https://news.ycombinator.com/item?id=34257196 -- just ask experienced engineers why they prefer languages that you consider to be irreparable.
I mentioned Rust, and I left Java and C/C++ out for a reason. Also, none of them have the properties in question (forbids runtime errors and doesn't bitrot).
I just spent most of the afternoon trying to make a government java program work. It won't apply electronic signatures. I know because it throws 47-line exception tracebacks and does nothing on the UI. Its documentation is for windows xp and windows 7. Yes it is the last version, from mid last year.
That's real bitrot.
Related to C pray they didn't hardcode implicitly something of the architecture. Or be surprised when a bad pointer fails in one OS but works in another.
The .net languages are wonderful. The best time I remember in my life was trying to debug what version of the .net runtime was required for a program. So much better than getting laid or going on holiday /s
Compiling to JS doesn't solve the problem because the language (JS) isn't the problem. The problem is the browser and its inadequacy for building applications in many areas - lots of APIs that are not fit for purpose as well as simiarly inadequate protocols.
This environment is additionally hostile to compilation - TypeScript had to bring some really amazing type system features to the table to get most engineers to reluctantly accept the headaches of compilation for the browser.
ES6 Modules, HTTP/2 and sourcemaps were supposed to fix these issues to some extent and all of them created major new headaches instead. Its like there is something fundamentally broken about the browser that we don't fully understand, successfully sabotaging all solutions we come up with.
The JS ecosystem is the furthest ahead in making this broken environment barely usable with a bunch of hacks to compensate for its shortcomings - a new language will need to fight hard to get close.
It is not always aspects of stability that are most important. I prefer js because it might be the fastest language to get an MVP out the door, among other things.
I think the economic argument you're making is the most sensible one, and it's going to be highly context dependent. Personally, I subscribe to the idiom that nothing is more permanent than a temporary solution.
> and to some degree it's design (or lack thereof) reflects the trade-offs required to hit the distribution problem out of the park.
How so? The first part I agree (as pure accident and the fact is thing bundled on the browser) but JS/C share the same problem: Terrible "design", ill-suited to the task, but sadly what was bundled.
And resistance to fix them (where in the case of JS could be super-mega-easy).
TS is probably as good as transpilers get. It really is quite good. I like that it maps one ts file into one js file and creates readable code (although output is normally bundled so readability is lost). I like that ts aids dev discovery in tools (intellisense) at write-time. But ts still needs a build step, and that's a big down-side to the immediacy of "plain" javascript.
If browser vendors bundled tsc or its equivalent then we'd both be stoked.
> If browser vendors bundled tsc or its equivalent then we'd both be stoked.
Not exactly this, but there was a proposal that would allow for typescript type hints to be considered as normal JS syntax: (in the same way python PEP 484 – Type Hints are considered optional type hints)
Deno on the server side is pretty cool and uses https://swc.rs/ (Speedy Web Compiler) - doesn't take forever to start, but kinda implements typescript as if it was just a type annotations without the more in-depth actual type checking, so it is possible.
I'm trying not to sound all gatekeep-y, but I think this is the downside of a low barrier to entry. JavaScript has attracted a lot of programmers who just don't have the experience to manage things like maintainability, dependency management, or non-breaking upgrades. Some of them go on to create things that are great in the moment and deservedly become popular, but then fall prey to problems in these areas as they age. The same has happened to various extents and at various times with Python, Ruby, PHP, and so on.
On balance lower barrier to entry is a good thing, but it does have characteristic problems too. In fact, I'd say fault lies partly with more experienced programmers and particularly packagers, who often tend toward avoidance (or worse) instead of helping these newer ecosystems avoid known pitfalls.
I really don't like PLs that's optimized for beginners because I believes they are not in "beginner" phase for that long (probably a few months), and often time the PL trades that with downside that developers have to live with in next 10 years+. The selling point I'd like to see is not "beginner friendly" but median developers friendly.
This pretty much sums up why I've switched to "low-code" platforms. I can still do about 95% of what I was doing before and I don't have to put up with this nonsense.
Yeah. I evaluated a bunch of these static site tools for my blog and they were all complicated to install and manage in their own ways. It was much easier and more flexible to just write a shell script that runs all the markdown files through pandoc.
I'm echoing some of my previous comments because this post resonates quite a bit with me. Ultimately, an ecosystem is only as good as its community and that's where I believe the root lies.
The community has not figured out how to maintain software. There is an unhealthy zeal towards building, so much so that the ecosystem is in a constant building phase with things being hacked up continuously. This is where the community needs to step in and nudge the ecosystem towards stability. They need to vote with their wallet with adopting and promoting the right software. There needs to be a certain amount of skepticism towards new ideas and regard for established ones. Stubbornness to change can be a wonderful attribute. Sometimes not doing something can be the best solution. But this kind of thinking is something I feel is somewhat missing here. Rather there seems to be incredible willingness to put up with pain to the point of being detrimental.
The Ghost community is paid to run a SaaS and supporting self-hosters is somewhat incidental to their business model. That community already is voting with their wallets and this poster was expecting to get better free support from a software company with incentives that don't align with the poster's. ("Non-profit" or not, it's still a software company getting paid to run a SaaS that incidentally allows self-hosting.)
I had the same issues when I started with https://emailengine.app - just like Ghost, it’s an app written in Nodejs. I tried multiple distribution options at first and finally went with complete self containment. All modules are pre-installed during the publishing step and thus the user never needs to run npm. Or if you download the “compiled” single binary version you don’t even need node as it’s bundled with the binary (I use the pkg module to bundle these executables). So upgrading is just downloading and replacing the old version files with new ones.
The dowside - no sane way to use compiled dependencies, everything has to be vanilla javascript.
I like posts like this, because software with these properties does deserve to be roasted. It seems like no one can figure out or remember anymore what's wrong with it, though. It's a valid question.
But a couple clarifications.
I don't develop or distribute software that depends on Node, but I imagine deciding which versions of Node to support, or conversely require, for software like Ghost, is a bit like deciding which Java VM versions to support, or which browsers to support. That is, there is a lot of compatibility between Node versions, but do you just develop and test on the latest one, or do you make an effort to support more/older versions? In this case, Ghost seems to be tested on v12, v14, and v16. The author seems to mistakenly think that only the "Active LTS" version of Node gets security fixes, but that's not the case, v14 and v16 are in "Maintenance LTS" and receive security fixes. That said, their "ends of life" are in April and September of this year, for whatever that's worth. At the end of the day, I think it is considered easy to update and/or run multiple versions of Node. I'm don't know why an app written for Node 16 wouldn't work with a newer Node 18, though, that kind of stuff annoys me sometimes.
For the MariaDB support, I googled and found this note from the Ghost devs: "Note: MariaDB is not an officially supported database for Ghost. It just happened to work given the similarities with MySQL, but we optimize and test for MySQL 5 and 8. As of Ghost 5.0 we are clarifying that official support is purely for MySQL8 in production so that we can double down on DB optimizations." Maybe that's what we're talking about? https://github.com/TryGhost/Ghost/issues/14446
Now, the deeper trend that I think is misguided is the rate of iteration and breaking changes, and apps and libraries having too many dependencies. The problems propagate through the dependency graph. Someone releases a library, and then adds a bunch of features, and then rewrites it, adds features, rewrites it again. That's fine for some hobby project that no one is using. I think anything I say won't be appreciated by someone who hasn't experienced programming and computing any other way, e.g. writing Java applications in the 1990s and 2000s. Or even just using desktop applications in the 90s. Software used to sort of "mature" and then continue to be used and supported, while staying pretty much the same, for years. It wasn't constantly rearranged to use new versions of things that are constantly rearranged to use new versions of things that are constantly rearranged etc. etc. etc.
It has nothing to do with JavaScript. It has to do with your software depending on N dependencies that you don’t control. And those N dependencies will randomly change and no longer work with your software.
So what is the solution? Reduce N as much as possible. Ideally N should be 0. That will never happen of course (you still depend on the browser/OS) but the closer you get to 0 the better.
There are also things you can do to design your software in such a way that there is a “firewall” between your code and external dependencies. That minimises “earthquakes” from spreading from dependencies to your code.
I have followed this rule when developing both client and server software. And I very rarely have problems with any ecosystems.
not to make light of the crappy parts about the JS ecosystem, but I have seen the nightmares of python, Java, Go, C and C++ tooling
Give me JS tooling madness any day before C++ linking issues, JNI crap or fucking pip (python package manager), go.mod direct-to-git deps madness, Android SDK madness and random XML files, fucking Cocoapods
Library version compatibility with runtime/compiler? What a joke, there is not a single language environment I have used that doesn't have this problem
Face it, IT ALL SUCKS. It is made by humans, only way to avoid this problem is top-down industry enforced language and practices and 5-year release cycles for new language features. Nobody wants that
On a small note about JS code bundling for browsers: it is a REALLY hard problem. If browsers could run, say, python, you would still have bundlers and bundler configuration galore. It is akin to video game engines building process where you need to merge all sorts of assets together, the main difference is that there is no IDE to press play button to do it for you (unity, unreal)
A good question is why is ghost so conservative about accepting new LTS versions to support. If I had to guess I'd say the answer is probably that the conservativism was induced by the great ES6 module breakages of previous node major versions, which in turn were induced by poorly thought out features in the ES6 module system which led to a bad transition experience for a variety of reasons (toplevel await, extension requirements derived from browser requirements resulting wiht typescript extension compatibility issues, etc).
So there were systemic problems with the way the JS language and browsers developed. Essentially, the browser (JS) ecosystem was left in an inadequate state to support web applications for far too long. This in turn is causing ripple waves of technical debt far into the future.
Stagnation in paying off techinical debt leads to workarounds (e.g. CommonJS, callbacks) which then get built on top of. Then when the "standard" is finally introduced (ES6 modules, promises) its already too late and a lot of the "buildings" built on top of those foundations become unstable or even collapse.
The combinatorial explosion of unstandardized and standardized things that need to be supported by foundational tools is also partly to blame. For example, thanks to the unaddressed shortcommings of CSS we have a smorgasbord of semi-adequate CSS solutions with poor compatibility. This in turn leads to problems for bundlers trying to support all kinds of combinations of those, and so on.
So the answer IMO is "technical debt induced by historically poor standardization efforts"
108 comments
[ 3.2 ms ] story [ 132 ms ] threadThat said, yea, the JS ecosystem is literal cancer.
How we got ourselves here, I don't know.
Then SV happened, and the web became Serious Business. Then Node.js happened and Javascript became Serious Business, and with that came the byzantine horror of complexity necessary to get a toy scripting language invented in a week to replace every enterprise programming language in existence because managers could toss a rock out of their office window and it would bounce off of three web devs before hitting the ground.
BASIC would like a word.
Heck, even Python came 4 years before JS.
I find JavaScript's type system to be horrendous and bug prone. I'd rather operations between two types that don't make sense (like dividing an Array by an Object) to throw an explicit error than to just do some unpredictable shit that causes a subtle bug.
Also, BASIC is probably the easiest language to learn, though this is mostly because it lacks capabilities that create complexities. It's a completely obsolete language.
Nowadays everyone has 10 JavaScript interpreters in every device, so it's much more accessible than finding out about and installing BASIC.
BASIC also lost all relevance in terms of actually being useful (if it ever had it), whereas JS is the bee's knees.
FTFY.
The most likely opposite to "basically invented overnight" is "designed by committee", and is that much preferable?
Origin stories aren't the end of the story either. Humble origins aside, JS has had many of the best minds in many of the largest companies contribute all sorts of feature work and performance efforts across a span of decades. How many languages can claim that many eyeballs and that diverse a set of language developers and engine writers?
Sure, a lot of it looks like a hodgepodge, especially with its mandatory, deep respect for backward compatibility, because no one wants to break an old website. But show me the language with any respect for backward compatibility that doesn't look like a hodgepodge?
Core systems languages like C and C++ started out as intentional hodgepodges. Their macro system was originally its own, different, language brought in from another existing project for some temporary duty, and now enshrined in the language as a forever feature.
Hodgepodges are how languages are built and evolved. That's how we got ourselves here: the natural way. Every programming language on the planet is its own beautiful house of cards, built piece at a time. Don't just look down on JS because it went "mainstream".
(ETA: I am a fan of Common Lisp, for what that is worth, if this sounds too critical. It's a very nice language. Some of what has made it nice was a massive committee and standards effort, very similar to the way JS has continued to evolve under TC-39 stewardship.)
Also yeah, my impression of Java was that it was very design-by-committee.
Roughly "overnight" for six years:
http://jmc.stanford.edu/articles/lisp/lisp.pdf
It's the only language I've written in where no matter what code I'm writing, I can always think "this could be fully generalized and spun out into a package"
I always approach any library upgrade as a potential disaster that'll result in me having to nuke my entire node setup for that project and rebuild it from the ground up or last known good state. I never had the kinds of problems I have with JS with any other bloody languages and their respective build tools.
Now, using the "engine" and "packageManager" properties in "package.json", you can bind your project to a particular runtime version and package manager.
lmao
npm (Node Package Manager) pnpm being a modified version thereof that adds a compression/content addressible store mechanism to speed things up in a development configuration. Or Yarn
...which is yet another dependency management solution cum build tool, which is Maven/Gradle like. Depending on version though there are so many quirks. Both are "staples" of any enterprise scale project I've run into.
I don't have an issue with nodeJS or JS per se. It's the ecosystem and enterprise tooling. I still cannot for certain say those damn build tools don't muck things up in horribly subtle ways, and the experience of debugging the average horrible nest of every dependency known to man has comsumed entirely too much of my life, and the experience overcoming that learning curve I still can't say with confidence wasn't somewhat deleterious to my personal sanity.
I can do it trivially now; and I can appreciate the things you can build with it when doing it correctly. The depth of hole you can quickly end up in though is not to be underestimated.
That and nigh every JS developer I've run into who says they are proficient knows nothing about really debugging it, or hasn't read the core library docs and really means they can use one framework or another.
...Bar like 3. Who you never want to bother, because they are usually handling issues way more important/wide-ranging that warrant their attention than hand-holding you through learning how it actually works.
None of those things (package-lock.json, node_modules, or p?npm|yarn) actually come from ECMA-262...
I'm surprised, the only node package manager that has any semblance of sanity is pnpm.
No flattening BS. Not randomly adding extraneous undeclared dependencies on the dependency tree. Not allowing random dependencies to depend on extraneous undeclared dependencies on the dependency tree.
Of course, then you find out that in the real life people actually use other package managers, and depend on non-deterministic, package flattening behavior and extraneous undeclared dependencies requiring extraneous undeclared dependencies...
Seems like this is more of a rant against tools upgrading themselves and breaking previous dependencies which causes this kind of domino effect of dependency issues when trying to upgrade the initial tool, not the "JavaScript ecosystem"
I host https://bytesizetheories.com on there and it's been an absolute breeze.
Why is computing like this?
I spend vast amounts of time hacking around in Linux with whatever I aim to do. It's almost never easy. There's always this problem then that problem - you have to shave yaks until you're so deep you forgot what you were working on in the first place.
It ain't just JavaScript.
1: how to make a software thing work:
2: follow instructions for (software thing)
3: hmm, wrong instructions, out of date
4: find right instructions
5: do step one of instructions
6: hmmm.... an error find some new software thing to solve this problem, then for this new software thing, goto 2
Programming is problem solving as much as anything else.
Man pages are great for just-in-case conceptual view of what you could do with a tool without a problem to solve
Alas, the chain of functions you're managing nowadays (3 to 4 layers of primitive on top of primitive on top of primitive, ) leads to a combinatoric explosion of potential layers to encounter a fault.
IMHO because the happy path is easy, but guarding against error conditions, and then either taking action or advising someone of that outcome is horrifically hard. It's hard not only because it requires the experience to know that not every HTTP request is 200, not every method call is a-ok, not every disk has plenty of space, nor permissions to write to whatever, and even with a huge checklist of the way things go toes up, a disciplined library author can have 250 different illustrative return codes that the caller can then throw in the trash, iterate all the way up the call chain and everything has to align perfectly for the actionable error to reach the end user
That especially plays out, in my experience, in programming languages that place the "please check the return code" upon the programmer, as opposed to ones where errors are more "in your face," but the joy of `try { doSomething(); } catch (ex) { /* lolol */ }` makes it all too easy to ship something because It Works On My Machine
Being on the pointy end of your checklist makes me have a great deal more empathy for users of my software, but I am merely one person who thinks swallowing errors is a crime against humanity
Imagine API consumers have the initial ids of things and remembers (implements) meaning of them, and then you can re-order parameters, change functions name, module names, fields name, even something moves could also be auto migrated on consumer side (or call site).
I was expecting a rant about react or something, but this is a more encompassing moving target problem with software maintenance.
(The answers to that question seem a lot easier to answer: small team, limited testing resources, and they focus on a paid SaaS platform as their business model to pay the bills and that likely comes first ahead of self-hoster's concerns/needs.)
The node version issue, for example, can be fixed simply by using tools like "nodenv" or "nvm".
- newest major version of self-hosted software (Ghost) no longer supports MariaDB or Debian
- newest minor version of the same software also depends on the latest LTS Node release, not the second-latest
- that's it
This really just sounds like garden-variety issues with running your own software — annoying, but not representative of or idiosyncratic to the "JavaScript ecosystem".
His options are change the entire OS and database or run an old version of node.
But you are correct, this isn’t the fault of the JavaScript ecosystem, its a series of engineering decisions by the ghost team about what they target and support.
Madness.
No, it's not.
This is just like one app. None of these are uniquely JS problems & half arent JS problems at all, in spite of the hater-header. In my view, the app didn't bend over backwards to cater to a bunch of tech-whims the user has, and the user is super salty about it. Maybe don't put yourself through such a struggle! This is exactly why we have containers: so you can easy-mode your way through, and never think about most of these problems & things just work. You don't get to take the hard path and then whine that your specific crusty old box isn't still supported with zero effort.
And to blame this on JS ecosystem? Typical. The hatred, the animosity against sprawling techno-diversity is unbelievably dark-side, has roots in a scared authoritarianism. It's just popular to hate on, and the complaints never seem to mirror what an educated JS developer might offer as a critique, of which I think we could/do offer many; it so regularly feels enormously low-signal, marks poorly upon the character of the complainers. I'm for catharsis and pouring out some of your hatred, but gee, so much of the JS anger feels facetiously low-dimensional.
JS- like all rife open source ecosystems- is by definition a bazaar, not a cathedral, and life is not going to be as blissfully simple as you might want. We balance what is possible against what is well defined, and interesting things keep happening when we favor what is possible (bottom-up) above what is well-defined (top-down). Try not to get so tarnished & bitter about it. There's a lot of upside too. Open source is good, actually. Us having done so much, explored so widely, is good, actually.
The experience with Eleventy (which involves Node js) is pretty much like with Hugo. Despite it being part of the javascript ecosystem. /shrug
OTOH, the JS culture of microprojects makes it more lokely that any given project will have one or more problematically-managed dependencies.
Because it's the product of a collaboration between ~1M beginner programmers, 100k intermediates, and 10k experts. Because the language and runtime are "mushy" - aka "under-constrained", which allows for all the ideas to flourish, even the "bad" ones.
It's a mess, but a glorious one. JS is the first language to really solve the distribution problem, and to some degree it's design (or lack thereof) reflects the trade-offs required to hit the distribution problem out of the park. In hindsight, it was bound to turn out this way!
IMHO the single worst thing about js is the front-end build experience. It's so nice to write build-less front-end! And yet (most of) the community seems obsessed with WebPack and inventing new transpiled languages etc. It's really bad! Not only is it a far, far worse DX but it breaks the thing that makes the web cool, that you can "view source" and figure out how other people made stuff.
But what is this "build-less front-end" you speak of? Surely there's build somewhere, you're just hiding it?
But no substantial modern application can possibly be built without them.
I can see that using modules could avoid the need for compilation.
That's certainly false, since substantial applications predate React, Angular, svelte, etc.
But you have a point. Steve Yegge once wrote that his company wanted to write a word processor in assembly and it seemed like a good idea at first, but then it was like drawing a straight line 100m long using only 2mm line-segments. The major purpose of these libraries is to increase the "length of the line-segments" to something more consistent with the distance you want to cover. The question is whether or not their exists a unit of abstraction in raw js/html that serves this same purpose. E.g. it's a demonstration proof.
Working on it.
Check out Firefox.
Preact has a no build path in their documentation: https://preactjs.com/guide/v10/getting-started/#no-build-too...
And here’s my no build react setup: https://github.com/jsebrech/create-react-app-zero
Of course, unless you do it tailored for your project, you don't get some nice features like tree shaking, but it's very convenient.
Excuse me, what? I just spilled my coffee all over the desk.
Care to explain what you mean with the problem and how js "solved" it? In my view dependencies and distribution are terrible in javascript, but maybe we mean different things.
To want more failure is to be insane. I think rather a lot of programmers think they’ve tried alternatives, but in reality have experimented with syntax swaps of what is essentially the same language.
Elm would be an example of a language which is fundamentally better than JavaScript (and is a practical substitute for what most people are writing JavaScript for), but it isn't the only one.
I think you are falling into a trap of thinking there is a “best” in engineering when that’s basically never the case. Ask experienced developers why they prefer Javascript if you want to get a more nuanced view. I almost always reach for Javascript first, especially for server applications.
This article might interest you: https://josephg.com/blog/3-tribes/ (you seem to be tribe 1, I'm probably tribe 3)
My code is not poetry; it kinda sucks. I’m not a gifted programmer, which is why I make up for my shortcomings with better tools. I use Haskell entirely for the sake of pragmatism [and have in fact bet my businesses on it]. My experience is such that I can build something that just works much more quickly and reliably in Haskell or Elm than I ever could in PHP or Ruby or JavaScript or Clojure[Script]. And I wrote each of these professionally for years.
In other words, there's a simple way to disabuse yourself of your position in this comment: https://news.ycombinator.com/item?id=34257196 -- just ask experienced engineers why they prefer languages that you consider to be irreparable.
Ah sorry, overlooked that.
Related to C pray they didn't hardcode implicitly something of the architecture. Or be surprised when a bad pointer fails in one OS but works in another.
The .net languages are wonderful. The best time I remember in my life was trying to debug what version of the .net runtime was required for a program. So much better than getting laid or going on holiday /s
This environment is additionally hostile to compilation - TypeScript had to bring some really amazing type system features to the table to get most engineers to reluctantly accept the headaches of compilation for the browser.
ES6 Modules, HTTP/2 and sourcemaps were supposed to fix these issues to some extent and all of them created major new headaches instead. Its like there is something fundamentally broken about the browser that we don't fully understand, successfully sabotaging all solutions we come up with.
The JS ecosystem is the furthest ahead in making this broken environment barely usable with a bunch of hacks to compensate for its shortcomings - a new language will need to fight hard to get close.
How so? The first part I agree (as pure accident and the fact is thing bundled on the browser) but JS/C share the same problem: Terrible "design", ill-suited to the task, but sadly what was bundled.
And resistance to fix them (where in the case of JS could be super-mega-easy).
If browser vendors bundled tsc or its equivalent then we'd both be stoked.
Not exactly this, but there was a proposal that would allow for typescript type hints to be considered as normal JS syntax: (in the same way python PEP 484 – Type Hints are considered optional type hints)
https://github.com/tc39/proposal-type-annotations
Deno on the server side is pretty cool and uses https://swc.rs/ (Speedy Web Compiler) - doesn't take forever to start, but kinda implements typescript as if it was just a type annotations without the more in-depth actual type checking, so it is possible.
On balance lower barrier to entry is a good thing, but it does have characteristic problems too. In fact, I'd say fault lies partly with more experienced programmers and particularly packagers, who often tend toward avoidance (or worse) instead of helping these newer ecosystems avoid known pitfalls.
My approach is to migrate away from any software like this.
The community has not figured out how to maintain software. There is an unhealthy zeal towards building, so much so that the ecosystem is in a constant building phase with things being hacked up continuously. This is where the community needs to step in and nudge the ecosystem towards stability. They need to vote with their wallet with adopting and promoting the right software. There needs to be a certain amount of skepticism towards new ideas and regard for established ones. Stubbornness to change can be a wonderful attribute. Sometimes not doing something can be the best solution. But this kind of thinking is something I feel is somewhat missing here. Rather there seems to be incredible willingness to put up with pain to the point of being detrimental.
The dowside - no sane way to use compiled dependencies, everything has to be vanilla javascript.
But a couple clarifications.
I don't develop or distribute software that depends on Node, but I imagine deciding which versions of Node to support, or conversely require, for software like Ghost, is a bit like deciding which Java VM versions to support, or which browsers to support. That is, there is a lot of compatibility between Node versions, but do you just develop and test on the latest one, or do you make an effort to support more/older versions? In this case, Ghost seems to be tested on v12, v14, and v16. The author seems to mistakenly think that only the "Active LTS" version of Node gets security fixes, but that's not the case, v14 and v16 are in "Maintenance LTS" and receive security fixes. That said, their "ends of life" are in April and September of this year, for whatever that's worth. At the end of the day, I think it is considered easy to update and/or run multiple versions of Node. I'm don't know why an app written for Node 16 wouldn't work with a newer Node 18, though, that kind of stuff annoys me sometimes.
For the MariaDB support, I googled and found this note from the Ghost devs: "Note: MariaDB is not an officially supported database for Ghost. It just happened to work given the similarities with MySQL, but we optimize and test for MySQL 5 and 8. As of Ghost 5.0 we are clarifying that official support is purely for MySQL8 in production so that we can double down on DB optimizations." Maybe that's what we're talking about? https://github.com/TryGhost/Ghost/issues/14446
Now, the deeper trend that I think is misguided is the rate of iteration and breaking changes, and apps and libraries having too many dependencies. The problems propagate through the dependency graph. Someone releases a library, and then adds a bunch of features, and then rewrites it, adds features, rewrites it again. That's fine for some hobby project that no one is using. I think anything I say won't be appreciated by someone who hasn't experienced programming and computing any other way, e.g. writing Java applications in the 1990s and 2000s. Or even just using desktop applications in the 90s. Software used to sort of "mature" and then continue to be used and supported, while staying pretty much the same, for years. It wasn't constantly rearranged to use new versions of things that are constantly rearranged to use new versions of things that are constantly rearranged etc. etc. etc.
So what is the solution? Reduce N as much as possible. Ideally N should be 0. That will never happen of course (you still depend on the browser/OS) but the closer you get to 0 the better.
There are also things you can do to design your software in such a way that there is a “firewall” between your code and external dependencies. That minimises “earthquakes” from spreading from dependencies to your code.
I have followed this rule when developing both client and server software. And I very rarely have problems with any ecosystems.
Give me JS tooling madness any day before C++ linking issues, JNI crap or fucking pip (python package manager), go.mod direct-to-git deps madness, Android SDK madness and random XML files, fucking Cocoapods
Library version compatibility with runtime/compiler? What a joke, there is not a single language environment I have used that doesn't have this problem
Face it, IT ALL SUCKS. It is made by humans, only way to avoid this problem is top-down industry enforced language and practices and 5-year release cycles for new language features. Nobody wants that
On a small note about JS code bundling for browsers: it is a REALLY hard problem. If browsers could run, say, python, you would still have bundlers and bundler configuration galore. It is akin to video game engines building process where you need to merge all sorts of assets together, the main difference is that there is no IDE to press play button to do it for you (unity, unreal)
> Oh, and the current LTS version is 18, so good luck with security.
NodeJS 16 EOL is September 11th, 2023 - https://nodejs.org/ar/blog/announcements/nodejs16-eol/
Ghost has just finished adding support for v18 https://github.com/TryGhost/Ghost/issues/16059
A good question is why is ghost so conservative about accepting new LTS versions to support. If I had to guess I'd say the answer is probably that the conservativism was induced by the great ES6 module breakages of previous node major versions, which in turn were induced by poorly thought out features in the ES6 module system which led to a bad transition experience for a variety of reasons (toplevel await, extension requirements derived from browser requirements resulting wiht typescript extension compatibility issues, etc).
So there were systemic problems with the way the JS language and browsers developed. Essentially, the browser (JS) ecosystem was left in an inadequate state to support web applications for far too long. This in turn is causing ripple waves of technical debt far into the future.
Stagnation in paying off techinical debt leads to workarounds (e.g. CommonJS, callbacks) which then get built on top of. Then when the "standard" is finally introduced (ES6 modules, promises) its already too late and a lot of the "buildings" built on top of those foundations become unstable or even collapse.
The combinatorial explosion of unstandardized and standardized things that need to be supported by foundational tools is also partly to blame. For example, thanks to the unaddressed shortcommings of CSS we have a smorgasbord of semi-adequate CSS solutions with poor compatibility. This in turn leads to problems for bundlers trying to support all kinds of combinations of those, and so on.
So the answer IMO is "technical debt induced by historically poor standardization efforts"