57 comments

[ 2.9 ms ] story [ 104 ms ] thread
Unrelated: What ever video player they are using, it sucks. I am viewing the video on a 21:9 screen in full screen and content is cut off since the player fills out the whole screen instead of keeping the video centered with black bars.
At least there's a download-link with the video, playing it back locally show's its also scaled.
Probably depends on a video player of your browser, how it scales the video. It's just an mpeg4 video published on the website.
It also had terrible audio quality for me.
It's because they set "object-fit: cover" on the video element's CSS.
While I'm also eagerly looking for an alternative to Git submodules (an initial pit of failure is that clones and checkouts aren't recursive by default, and in the long run it's annoying that branching/forking a module doesn't fork its submodules' upstreams), and the existing alternatives are poor (subtrees fail to resolve merge conflicts, and subrepos are unmaintained, fragile, and break when you rebase a clone/pull commit), I'm instantly put off by the slick commercialized sales-funnel design of your website.
I know submodules, and I sort of know subtrees (which as I understand are similar to what this tool does, except more manual and error-prone), but what are subrepos? [this]( https://www.mankier.com/1/git-subrepo)?
git subrepos are an alternative to git submodules and git subtrees.

https://github.com/ingydotnet/git-subrepo

git subrepos work simply by copying your dependency to a subdirectory and committing the changes using one large commit that retains metadata about the update to the subrepo. For that reason, git subrepos aren't symlinks. You don't need to git clone --recursive like with git submodules, and you don't need cross-repo authentication. Updating a subrepo means performing another commit.

Even though git subrepos are the most poorly maintained, the design is simpler.

I wish someone would fork and take over maintenance.

git subrepos are the best.

Are Git submodules really that bad? Really? Maybe it's me that haven't yet discovered their true dark side. They're not perfect, but works well for mid-sized projects at least.
The most annoying problem is CI, i cannot even use a private git submodule in Github Actions without creating a personal token, passing it to secret and use some git command
Which seems more a problem with Github tooling, and less with git submodules.
That's because you haven't had all code messed up when someone pushed in the wrong order :-)
How can pushing in the wrong order mess up all code? The worst thing I can imagine is pushing the superproject with a subproject update, but not pushing the subproject. That would end up with an unbuildable commit in the superproject, as you can never check out its associated commit from the subproject, but I don't see how that's close to "messing up all code".
They're fine, so long as you do not have to fork them. Very often we have to work on multiple submodules in parallel, to make that work we use the same name for all feature branches, then our CI and tools understand that these unrelated branches in unrelated repos actually belong together, but I'd prefer not to have to do that. At least it taught me why so many big companies use monorepos.
Good question, I was wondering the same. I have never really had an issue with submodules, and I am not sure what the killer feature that this adds over submodules.
They're better now than they were years ago, but yes they're still bad. And they're still second-class citizens for git commands.

Git has been adding more config settings to try to make it less painful, such as submodule.recurse, but that slows down most git-command significantly and the truth is most repos don't actually need to recurse - they just need the top level. So instead people end up writing aliases to try to handle it, which works ok, but it's brittle and people have to know to do it.

And if you have your repo on GitHub, they don't do some simple things that could have made it easier. For example, they could let the repo owner make the git-clone copy-to-clipboard thing have --recurse-submodules in the command you copy, so that cloning will do the right thing. But they don't. I understand why git itself doesn't want to make such things automatic, but GitHub could do it for at least private enterprise account repos.

BTW, I am _not_ advocating whatever this "Git X-Modules" thing is that the link points to. I have no idea what that is, never used it, and don't plan to.

They're much easier to consume now than they used to be but at the same time they're usually broken by default. Adding them to a project is not seamless or invisible. They do not "just work" with the same commands your team was previously using.

They're still pretty annoying to actively work in.

Maybe that is the problem - you should not use them for stuff that needs to be actively changed all the time.

We keep only DTOs in sub-modules so you really update it if you change property on some communication objects.

This way you also don't want them to be invisible - you really want to see and have people notice this was updated and actively choose what to do with it.

They are implemented in a maximally safe way, which generally means they leave files / folders / etc lying around when you least expect it which goes on to cause later annoyances, because doing otherwise could mean deleting data you didn't know would be deleted.

Say you're git, and someone has a submodule that goes from having:

    sub/
      .gitignore = a/ignored.txt
      a/
        tracked.txt
        ignored.txt
to this, when they `git checkout` some other newer or older or totally-separate-history branch (possibly several layers above this submodule):

    sub/
      b/
        tracked.txt
what do you do with sub/a/ignored.txt? What if the whole submodule was removed?

Git's answer to every single question like this is to either fail to perform the submodule version change, or leave a now-untracked-and-not-ignored sub/a/ignored.txt file in the directory, which leaves you with an un-clean checkout that it'll warn and complain and possibly conflict on.

It's highly unergonomic, but reliably avoids silently doing anything that might cause you to lose data. The awful ergonomics are what people generally hate about it. They'd rather it made some consistent decisions so it Just Worked™ in nearly all cases. But that's not Git-like.

They have a reputation for being bad, which is enough for a company like the one in the OP to use as a marketing hook.
This is probably only marginally helpful given how ~monorepo-focused the pitch is, but, if:

- you're using submodules to vendor dependencies or develop multiple related projects

- you have control over your environments/toolchains

You can generally use Nix to compose the projects together from separate source repos. You can have a version-controlled specification of exactly which commits you're depending on and control of when to update them.

Fair warning: I'm speaking with the naivety of someone who occasionally needs to build projects that use submodules, and as someone who has been tempted to use submodules for my own inter-related projects (but used Nix to avoid doing so), and not as someone firsthand familiar with other use-cases like working in a bigcorp monorepo :)

Not sure why the angle is "replace Git submodules". Seems like it's simply a different architecture where the "sub repo" library requires a lot more updates than what a git submodule is designed to handle
Just like others, I've never really had a problem with Git submodules apart from forgetting to recursive clone or or twice. So I don't personally see the point of this, epsiecally since the pricing page implies it will cost money once it's out of beta.

(On that note, are there any actual reasons to not just --recurse-submodules by default?)

Yeah - recurse-submodules discards local changes without warning.
Could you provide an example or reference?
"Enterprise pricing will be revealed after the full-fledged release."
> Submodules done right

Click-bait marketing seriously?

The basic premise is true, I've heard many wiser-than-me devs nightmare stories about submodules. But I thought the OSS world had largely moved to git subtree, which is completely transparent for people who don't maintain the subtree folder and is supported in git for quite a while. Perhaps not?

In either case, I'd love to see a comparison to both git subtree and submodules, on a more "workflow-by-workflow" basis.

This sounds to me a lot like git-subtree combined with automation where commits to the subtree's native repository cause automation to add those commits (with path adjustments) to the super-repo, and vice versa.

This keeping both sides always in sync with each other can avoid some of the downsides of git-subtree, but has its own limitations like only working when both sides are owned by the same org, does not allow a subtree that maintains "local patches", etc.

If simultaneous pushes to super-repo and child repo occur at the same time, one gets applied and the other gets converted to a pull request (unclear if the server reports error back to the git client, or if the push appears to succeed, but then appears to be force pushed over by the other change, being preserved as a pull request.)

I don't understand all the negativity in this thread. Submodules can be annoying in that they require special attention and treatment all the time. You need to keep the mental model of your entire project's tree in mind so that you don't screw up or mix up something. The whole thing just feels a bit too fragile.

Submodules are usually good for frameworks and libraries that are more stable than your main codebase. However, those submodules under active development within the same organizations can be a pain to maintain.

I'm not sure there's a good solution to this under git at all, but at least there's a product that makes an attempt and I absolutely don't mind the commercial intent here. For example GitHub is commercial after all, what's wrong with that?

> Submodules are usually good for frameworks and libraries that are more stable than your main codebase. However, those submodules under active development within the same organizations can be a pain to maintain.

Except they aren't. Libraries are better in their own code base and published as a separate artifact which is auto-versioned.

From the article you linked:

> I know some programming ecosystems, such as C++ does not have a default way of handling libraries and dependencies, and in that case it might be good to look into submodules.

> Please, if there is any dependency management in your chosen platform use it.

I use C++. There is no such thing as package management in C++ (at least no unified standard that you can use with confidence with every library). So giving a blanket statement and saying that you should just ship library artifacts is a non starter in a language like C++. This basically implies either creating a package manager, or distributing binaries manually, or forcing users to find the correct versions manually and install them.

Git submodules is extremely helpful here, and if you look at the documentation this is what it sounds like it was created for:

> Large projects are often composed of smaller, self-contained modules. For example, an embedded Linux distribution’s source tree would include every piece of software in the distribution with some local modifications; a movie player might need to build against a specific, known-working version of a decompression library; several independent programs might all share the same build scripts.[0]

If this suggestion here doesn't sound like your use case, then your probably using the wrong tool.

In the same way, you wouldn't tell people who use a hammer to cut a piece of wood in half that the hammer must be a terrible tool. You would say something like, you're using the wrong tool, here's a saw. Use the correct tool for the job at hand. If you need source code from a 3rd party library that you won't modify but you depend on, a submodule is a fine tool for the job.

[0]: https://git-scm.com/docs/user-manual#submodules

This 'solves' the submodule problem by just copying everything, as far as I can tell.

A few big differences from submodules:

With X-modules, you have all the code in your repo, vs. a reference to the other repo. If you are looking at submodules to manage repo size, this solution is not for you.

With X-modules, you are tracking a branch, not a revision. If you want to pin an external dependency, it's not clear that you can do that with an X-module (the screencast doesn't show it, I don't think).

X-modules require an extra application to manage the repo relationships. There's a command-line client, but it's listed as unsupported on their website. It's not clear what the auth story is around who can make these changes. It doesn't look like adding an X-module would be subject to code review.

Personally, I'm not interested in this at all. I don't think it's a bad product, but I wonder what problems it actually solves. I don't know how common the submodule pattern is to compose a monorepo like they've demonstrated, not in an 'enterprise' setting they hint at future pricing for, anyway. And it's not clear to me whether their sync pattern is going to be OK for external third-party dependencies, unless you want to actually track their changes automatically vs. being intentional about when you pull in changes; this seems like a supply chain issue to me.

With x-modules we aim to solve the issues that monorepo introduces keeping the benefits.

Benefits: easy cross-component changes, repeatable builds Issues: huge repositories, lack of code ownership boundaries, complicated versioning for individual products/components

Of course people nowadays rarely use submodules to build a monorepo, but I believe they would like to if submodules would be transparent.

Git X-modules are at early stage and only syncs a ref with a directory now. We'll add an option to pin a particular commit and most interesting - an option to receive module changes as PRs queue that could be individually accepted (and applied alongside with the necessary call site refactorings).

Git X-modules is available for GitHub users now, and we plan to add GitLab and Bitbucket support - both cloud and on-premises. So, we reuse platform's user management to only let repository admins/devs set up the modules - currently via web UI, and via .xmodules files in the future versions.

I don't like them either, but things are a lot more manageable if:

- you require that the submodule always move forward (reverts are not allowed; you must fix the submodule and get something there first) - the submodule commit must always be reachable from the configured branch (`submodule.NAME.branch` in `.gitmodules`) - you require that commit be in the first parent history of the branch

This makes conflict resolution trivial (the last prevents MR heads conflicting because the first-parent history linearizes and requires that "A contains B" be true in one pairing).

Without these requirements, it is really a wild west of insanity IME. Much better just to use the dependency mechanisms of whatever tools you use (assuming they exist).

What's the comparison between this and Repo?
I assume the parent comment refers to this, to save others googling "git repo"..

https://github.com/GerritCodeReview/git-repo

repo is Google's tool that they use for at least their public code, and it's also used by a number of other sprawling projects (eg, Couchbase). It's certainly the first tool I would think of when you need to manage a ton of repositories under one project.
Fair. In hindsight, I've actually used it. Probably with Chromium and perhaps AOSP earlier. But I didn't recognize the name immediately. Thought I'd add the link for others.
Some aspects of git submodules are bad, especially for beginners. When I used them the first time had the strange experience that checking out or pulling left the submodule out of sync and I had to go to the directory and check out the right reference. Now I've learned to throw --recurse-submodules after "git checkout", "git pull", and "git clone" and you can mostly forget submodules then unless you start to interact with it explicitly.
I haven't had a chance to try it at scale yet, but I've been looking at josh (Just One Single History): https://github.com/josh-project/josh

which goes at more-or-less the same problem from the other direction. Josh helps with treating a monorepo as separate repos, as opposed to git x-modules' approach of treating separate repos as one repo. Either way, the idea is to try and get the benefits of both monorepo and multi-repo styles, while avoiding/mitigating the disadvantages.

HN talked about it 11 months ago: https://news.ycombinator.com/item?id=27844363

Anyone have experience/updates to share since then, or a comparison with git x-modules?

In every respect, versioned libraries are far superior to submodules.

Every time I've encountered submodules in any work projects, I've personally replaced them with auto-incremented semver libraries (usually jars) and received copious thanks for it.

When I see this done, it ends up causing headaches as well. You now have to deal with a second authentication flow that is usually a lot more opaque. I work in games with less technical people such as artists. Bootstrapping their workspace is an effort. "Setup your maven credentials" might as well be ancient latin.

Submodules aren't ideal but I do prefer that you can pull down everything you need in a one liner instead of multi-stage pull, build, auth, download, build etc.

So I wouldn't say its better in every respect.

In most aspects submodules are far superior to bad versioning. I understand the Amazon workflow, but never found a company were it worked properly.

In my current job, which uses semver and conan and not submodules, submodules would have solved most of the problems. But unfortunately they are all git newbies (switching from svn just lately), so you cannot throw submodules at them yet.

My personal setup is: /home/eric/repos/

All git repos end up in that folder - it's basically my project folder. Before explaining more, I should specify that this setup works well for me, but I could see how others may not agree with it.

Let's say I have 2 projects called prj1 and prj2. In that case, I would have the following directories:

/home/eric/repos/prj1 /home/eric/repos/prj2

Both of those would be git repos. Now, let's say that both of those repos rely on a shared repo, which I'll called shared-repo. In that case, I would have this:

/home/eric/repos/shared-repo

Then (and here's the part that probably won't sit well with folks), I just reference the parent directory from either prj1 or prj2 to access shared-repo.

Down sides: I'm forcing a hard coded directory hierarchy and directory naming. Up side: it's simple. If symlinks worked just as well on Windows as they do on Linux, I could just create a symlink in both prj1 and prj2 that points to my shared-repo directory and that could solve this whole mess too.

Having said all that, submodules have worked for me in the past, but I use them so rarely that I always need to look them up - so I've gotten in the habit of keeping things simple for myself and just doing things the way I mentioned above.

I've used a similar system, and tried to formalize this approach at work. The directory containing the projects is the "workspace" or "deployment" (depending if you're talking about your dev machine or a server, but same structure), and it contains "projects". Take whatever set of projects at whatever versions they need, check them out to the workspace, maybe set some environment variables (or symlink apache config files) as applicable, and hit the run button.

The biggest downsides of this approach that I discovered are that (A) projects are not self-contained, which meant some expert (i.e. me) always had to go and fix the deployments, because nobody else understood the system, and (B) a lot of the frameworks we used had their own system for organizing dependencies, which meant that we'd end up with projects inside projects inside projects anyway, and this system, whose intended purpose was to _flatten_ the directory tree, effectively just added another level to it.

Edit: (C), which is a variation of (B): if some projects are in monorepos, that throws off the system, too. Now instead of workspace/project you have workspace/monorepo-project/subproject, so now in some cases you have to reference "../../something" instead of just "../something". Not everything fits into the nice flattened-out system where relative paths are easy to guess, so you end up having to either merge things together with symlinks, which gets confusing because now you have two parallel structures going on, or just configuring dependencies with environment variables after all (at which point you wish you had just formalized the environment variables from the beginning to make everything explicit).

The more important downside I see is code traceability and verification. You would completely be void of both with that setup.

I'm not saying you shouldn't, as it works for you, and that's of course fine.

But, the key functionality of submodules is that the sumodule can exist on its own, as a git repo. The degree of dependency is then entirely controlled by the parent git repo.

I feel like submodules are an extension of SVN externals which are also in my opinion a bad idea. I rather run a mono repo or bring dependencies I need at build time using something like tag versus having to manage submodules.
Looking forward for GitLab version!!
Main thing I wish for is to have a submodule be easier to set to track a (version) tag. Now I have to lookup what the checksum corresponds to.

Maybe also I could set their dirs to be 'read only'; git should refuse any local changes and only accept a direct change of checkout (of hash or tag).