I just read this, and having never used any internal package registry to me it's just mind-boggling how the dependencies are recorded and the default logic for picking which repo to use (i.e. the one with the highest package version).
it makes sense for a few scenarios, like a maintainer testing a new version via a private repository server before pushing this update to the public repository.
But not having a way to use a separate namespace for private packages that only looks at your internal feed is certainly a mistake. NPM at the very least least you easily map the `@MyCorp/` prefix or similar to a registry of your choice.
> it makes sense for a few scenarios, like a maintainer testing a new version via a private repository server before pushing this update to the public repository.
This is what tools like `npm link` are for, though. You really don't need this sort of fragile and dangerous version selection at all.
npm link does not behave even remotely the same as a package installed from a repository, and there are quite a few packages where this does make a difference.
Another charitable explanation for the version selection rule is to allow specifying more than one mirror of the registry. Thus if the closer mirror has the latest version it will be used, but if there is a new version in the main registry that has not been synced to the mirror yet, the latest version will be used.
> Another charitable explanation for the version selection rule is to allow specifying more than one mirror of the registry. Thus if the closer mirror has the latest version it will be used,
This is correct, but in light of this being now vulnerable to abuse, this will have to be controlled in mixed use scenarios, which are common, e.g.
> if I run an internal feed that transparently proxies and caches the public one, as well as hosting my company's internal artefacts, the rules now might need to be different between packages?
> for e.g. between Newtonsoft.Json (new versions always originate on the public feed, never locally) and "SDCo.GeneralUtils" (new versions always originate on the local feed, never upstream)
> just mind-boggling how the dependencies are recorded and the default logic for picking which repo to use
It makes some sense actually, For starters this is an immutable artefact: https://www.nuget.org/packages/Newtonsoft.Json/12.0.2 "Newtonsoft.Json 12.0.2" will always refer to the same bits. If you fix a bug in it and ship different bits, well then that's now e.g. "Newtonsoft.Json 12.0.3". There is security in immutable artefacts, you can detect tampering, with checksums.
> having never used any internal package registry
Internal package registries are useful a) for caching what can be large requests that repeatedly download the same bits and b) managing internal code, rather than huge projects or monorepos. So they are useful when codebases get large. Several vendors make products that you can deploy for this: JFrog artifactory, MyGet. Azure and Github both have hosted package feeds.
It is not mind-boggling at all that you don't want someone making an internal package called "Newtonsoft.Json" - that way lies confusion. Especially if the internal feed will "fall back" to read-through caching a public feed.
So the logic in package management is generally that a name such as "Newtonsoft.Json" is considered the same package _globally_ , if you get it from an internal feed or not. Source is typically considered irrelevant. So "Newtonsoft.Json 12.0.3" must be an update to "Newtonsoft.Json 12.0.2".
And of course that's now the problem: deliberately doing the thing that you shouldn't do is how you break systems.
The clever part is doing it the other way around by guessing an internal name and hosting it on a public feed with a later version.
I guess supply chain attacks are getting more scrutiny now after the SolarWinds thing?
The worst thing that I have seen a company to itself do regarding package naming is to fork a whole system, into call it "US" and "EU" codebases, with subcomponents, and either their own (internal) package feed. But not many things were renamed in that, so the US feed "Customer.Client" package and the EU "Customer.Client" package diverged over time, became incompatible. One shipped more iterations than the other so had a higher version number...
After the next re-org a few years later there of course was a drive to consolidate, and of course this meant using both feeds. It's the kind of chaos and confusion that can only be solved by "well, don't F**ing do that in the first place, silly".
6 comments
[ 0.19 ms ] story [ 27.5 ms ] threadBut not having a way to use a separate namespace for private packages that only looks at your internal feed is certainly a mistake. NPM at the very least least you easily map the `@MyCorp/` prefix or similar to a registry of your choice.
This is what tools like `npm link` are for, though. You really don't need this sort of fragile and dangerous version selection at all.
Another charitable explanation for the version selection rule is to allow specifying more than one mirror of the registry. Thus if the closer mirror has the latest version it will be used, but if there is a new version in the main registry that has not been synced to the mirror yet, the latest version will be used.
This is correct, but in light of this being now vulnerable to abuse, this will have to be controlled in mixed use scenarios, which are common, e.g.
as I wrote here https://news.ycombinator.com/item?id=26089387
> if I run an internal feed that transparently proxies and caches the public one, as well as hosting my company's internal artefacts, the rules now might need to be different between packages?
> for e.g. between Newtonsoft.Json (new versions always originate on the public feed, never locally) and "SDCo.GeneralUtils" (new versions always originate on the local feed, never upstream)
It makes some sense actually, For starters this is an immutable artefact: https://www.nuget.org/packages/Newtonsoft.Json/12.0.2 "Newtonsoft.Json 12.0.2" will always refer to the same bits. If you fix a bug in it and ship different bits, well then that's now e.g. "Newtonsoft.Json 12.0.3". There is security in immutable artefacts, you can detect tampering, with checksums.
> having never used any internal package registry
Internal package registries are useful a) for caching what can be large requests that repeatedly download the same bits and b) managing internal code, rather than huge projects or monorepos. So they are useful when codebases get large. Several vendors make products that you can deploy for this: JFrog artifactory, MyGet. Azure and Github both have hosted package feeds.
It is not mind-boggling at all that you don't want someone making an internal package called "Newtonsoft.Json" - that way lies confusion. Especially if the internal feed will "fall back" to read-through caching a public feed.
So the logic in package management is generally that a name such as "Newtonsoft.Json" is considered the same package _globally_ , if you get it from an internal feed or not. Source is typically considered irrelevant. So "Newtonsoft.Json 12.0.3" must be an update to "Newtonsoft.Json 12.0.2".
And of course that's now the problem: deliberately doing the thing that you shouldn't do is how you break systems.
The clever part is doing it the other way around by guessing an internal name and hosting it on a public feed with a later version.
I guess supply chain attacks are getting more scrutiny now after the SolarWinds thing?
The worst thing that I have seen a company to itself do regarding package naming is to fork a whole system, into call it "US" and "EU" codebases, with subcomponents, and either their own (internal) package feed. But not many things were renamed in that, so the US feed "Customer.Client" package and the EU "Customer.Client" package diverged over time, became incompatible. One shipped more iterations than the other so had a higher version number...
After the next re-org a few years later there of course was a drive to consolidate, and of course this meant using both feeds. It's the kind of chaos and confusion that can only be solved by "well, don't F**ing do that in the first place, silly".