The problem is that npm just migrated their registry to use CloudFlare. Yarn already used CloudFlare, and had https://registry.yarnpkg.com CNAME'd to https://registry.npmjs.org. CloudFlare has a security restriction that one CloudFlare account cannot have DNS records pointing to another CloudFlare account, so npm moving to CloudFlare broke yarn's existing CNAME to npm's registry (thus breaking yarn).
Sorry but this is kind of funny. It would have been easy for the default value in the app to have been "https://registry.npmjs.org". This is an example of "every level of indirection can fail". [EDIT: removed unfounded speculation that CF wasn't necessary for yarn's requirements.]
Based on the (recent) comment on the issue [0], they use the Cloudflare proxy instead of an actual CNAME /and use workers to do yarn-specific changes on the edge / (see below).
I work on Yarn. The message you're quoting is from the npm CTO. We don't do yarn-specific changes. The main reason we're using a mirror is because it was massively faster than directly querying the npm registry when Yarn got released. The difference decreased during the past year I think, but Cloudflare was still a bit ahead in most scenarios.
Fun fact: you actually used to be able to do this and it caused some security problems! There are probably zones out there that are still able to do it today from back when it was allowed.
This is why you have a Testing environment you can sandbox this kind of thing out on first ;-)
Or, as the old sawhorse goes, "every company has a Testing environment. Some companies are fortunate enough to have a separate production environment as well"
True in general, but I don't think this one's that simple. npm made a perfectly reasonable configuration change that incidentally happened to break yarn. Hard to predict since you can't sandbox every internet service that might be unexpectedly depending on you, and arguably not really npm's problem anyway. (Much to their credit, npm jumped on the issue and contacted their CloudFlare rep to help unbreak yarn, but I don't think they're responsible to proactively anticipate things like this.)
On the contrary this is exactly why you do it on a holiday weekend. Minimal impact to the world as hopefully your users are yarn installing a beer instead of a package.
This was our plan: prod load, but not super load, not a lot of work that we'd disrupt with bug shakeout. I don't have a good way to simulate production load on our staging environment right now (some day) and that wouldn't have shaken this one out anyway.
well, you can look at it at both ways, if yarn just used npm's registry then there wouldn't be a problem. I am not sure what's their reason to add a registry (a mirror at that) if they don't plan to make their own.
ISTM this was a marketing ploy for yarn. If CF could proxy npm's registry faster than npm could serve it directly, yarn could claim to be faster than npm. Even if the apps themselves had been identical, network speed is still speed. As we could expect for any competitive advantage that is purchased by one party, it has now been purchased by another party, so it ceases to be an advantage at all. The Red Queen lives!
I think it's probably a good idea to run your own proxy/mirror of whatever package registry your language uses if you heavily depend on it being available. We proxy packagist at work, I'm starting to think we should do the same with npm. Has anyone here had success with any out of the box solutions?
Has anyone here had success with any out of the box solutions?
No, but we have had multiple failures with various tools when attempts to remove the remote dependencies came into conflict with not particularly unusual real world requirements. This is a pretty serious flaw in the JS ecosystem right now.
Unfortunately I can't offer much constructive advice here. We have found no good solution so far that doesn't have significant limitations, except in the case that all of your developers are in the same place and you are willing to literally clone the entire thing also in the same place, have everyone use that one clone, and incorporate the necessary maintenance work into your processes (which is a viable if somewhat awkward strategy if these constraints are acceptable in your situation).
We use Sonatype Nexus OSS to host private as well as mirror public Maven, npm, and Docker repositories. It took a small amount of fiddling but in less than a day we were up and running, using our LDAP for auth, and it’s been pretty much trouble free from the get go. In fact, we even use it to host our own completely custom internal package system (based on Ivy) for our content packages and database modules. It’s been a huge boon to us.
Artifactory is another competitor but last I checked npm and Docker registries required the paid version.
Yes, I've been running smart-private-npm for years on Cloudant accounts. Works exactly like original npm, but more reliable. Glad to provide advice if you go that direction.
32 comments
[ 4.6 ms ] story [ 72.5 ms ] threadThe problem is that npm just migrated their registry to use CloudFlare. Yarn already used CloudFlare, and had https://registry.yarnpkg.com CNAME'd to https://registry.npmjs.org. CloudFlare has a security restriction that one CloudFlare account cannot have DNS records pointing to another CloudFlare account, so npm moving to CloudFlare broke yarn's existing CNAME to npm's registry (thus breaking yarn).
This is actually an issue with the CDN portion not the DNS portion.
[0]: https://github.com/yarnpkg/yarn/issues/5885#issuecomment-392...
Or, as the old sawhorse goes, "every company has a Testing environment. Some companies are fortunate enough to have a separate production environment as well"
Can it wait until the start of the next week? Almost always.
More status on the Github issue [0] than yarn status page [1]
[0]: https://github.com/yarnpkg/yarn/issues/5885#issuecomment-392...
[1]: https://status.yarnpkg.com/
https://twitter.com/ceejbot/status/1000230136900239360
(See: https://developers.cloudflare.com/workers/about/)
No, but we have had multiple failures with various tools when attempts to remove the remote dependencies came into conflict with not particularly unusual real world requirements. This is a pretty serious flaw in the JS ecosystem right now.
Unfortunately I can't offer much constructive advice here. We have found no good solution so far that doesn't have significant limitations, except in the case that all of your developers are in the same place and you are willing to literally clone the entire thing also in the same place, have everyone use that one clone, and incorporate the necessary maintenance work into your processes (which is a viable if somewhat awkward strategy if these constraints are acceptable in your situation).
Artifactory is another competitor but last I checked npm and Docker registries required the paid version.