I've been playing with a homegrown centralized VCS client and server project, aimed at basically being the VCS host for monorepos, and this looks like a much better foundation for the VCS layer.
Absolutely, good for the author. However, given the numerous examples of Google pulling the plug on nonessential projects, I don’t see this as strictly positive development.
Based on this https://devboard.gitsense.com/martinvonz/jj it looks like there are two people working on it full-time, with 7 people contributing at least 500+ lines of code churn and 14 contributing code changes. Overall the project is fairly healthy and there does appear to be enough people engaged with the project that it should continue, should Google pull the plug.
I had to choose a VCS for a project a year ago. At that point jujutsu looked extremely promising but immature. It's great to see that so much progress has been made. I can't wait to give it another go, the next time I get to choose a VCS for something. Innovation in this space is desperately needed.
I would say "What a weird name for a VCS. Whether that will work ...", but then I have to remind myself of the dictionary meaning of "git". So who knows. Maybe we will be adopting all kinds of martial arts terminology. For example: "I use Karate to manage my code. I divide everything using chops. When a kata is done, ..."
Did you know that the English way to pronounce "jujutsu" isn't how the Japanese pronounce it? The Japanese way to pronounce it could actually be a lot easier for many non-English speakers (I'd anyway argue it's a common enough word that people are familiar enough with it to pronounce it in a way that's comfortable to them).
That's funny, I thought the name means 呪術, but the person in that audio clip is (probably) pronouncing 柔術. I prefer my interpretation though (git with magic).
Ambiguous romanization is weird, I wish romanization rules would just follow IME rules (except in the case of consecutive n's, that'd look weird).
> Did you know that the English way to pronounce "jujutsu" isn't how the Japanese pronounce it?
That's kind of tautological ...
However, yes, that is certainly true. Japanese has a pitch accent rather than a stress accent and all syllables are roughly the same length. English speakers don't tend to talk that way.
Looks really cool! One thing I'm not clear on from the docs: does it support ignoring changes to some files for "real" commits? For example, a repo at work has a file used for mocking up feature flags. The file is tracked but it's encouraged to edit it when testing things, just don't commit your changes. If I'm not mistaken, I'd have to remember to undo changes to that file before "describing" the commit. Is that right?
The commit will indeed be created immediately, there's no way to prevent that except for .gitignore I'm aware of. Until you run `jj describe`, it won't have a description.
However, if you don't manually put a branch on it, it'll never get pushed and will stay on your machine only.
You can sit on this personal commit and rebase it on top of any other commit to move around the repo, again and again if you like.
This would ignore changes while you're testing - but you have to remember to turn it off or, iiuc, you won't pull intentional changes either.
You might find hooks useful too. Not to assume your knowledge, these are shell scripts placed in .git/hooks that are invoked, e.g., before commit or before push. You could have it parse git status, detect changes to <file>, prompt for confirmation if changed and remove from working set if the change is unintentional.
README says that the git backend is the recommended backend, as the "native" one has no additional features, so I imagine: it handles them the same as git (ie. they are just objects in the .git repo data, and each time you change them you add a new one, and they are poorly compressible and optimizable) -- which is, I imagine, the problem you're referring to.
It won't be completely the same as git, because the client fetches stuff more lazily (according to the linked presentation). So the backend will still fill up with stuff and need lots of storage, but the clients then won't necessarily slow down in the same way.
That said, any git users will still get slowed down - only jj users would see the benefit. Git does have better features these days for shallow clones though and even git is on the way to killing off the need for LFS.
I am certainly no expert in version control systems, but I've gotta say that it's really wonderful to see a project that builds on the algorithmic and cultural successes of Git but with a simplified and modernized approach. The reason that Git took over the open-source world is two-fold: first, it was adopted by Linux, which ended up being the most influential OSS project of all time. Second, the Git model, which is distributed and egalitarian at its core, is a natural model for a fast-paced, globally distributed community of developers. These are both reasons to appreciate Git, but they do not imply that Git is the final word in version control. I'm excited to see innovation in this space!
> which is distributed and egalitarian at its core, is a better model for a fast-paced, globally distributed community of developers than previous monorepo systems like Mercurial
I'm a bit confused by this. I don't think that's what monorepo means, is it? Monorepo is what you choose to put in a repo? And I thought Mercurial was extremely similar to Git as it's also a DVCS?
Git and Mercurial are fairly similar, at least compared to what OP is apparently hinting at. Different tech and different internal architectures, but both of them are DVCS offering roughly equivalent workflows.
It is. The OP is wrong, probably because he has never used Mercurial, which wouldn’t be that surprising given it doesn’t have a lot of users outside of the few remaining companies still using it.
Mercurial is indeed extremely similar to Git. Also, it's two weeks younger, contrary to OP's (apparently now-deleted) assertion that git was replacing hg. (I find it amusing that bzr, git, and hg all came out within a month of each other.)
This Git-compatibility-first approach makes Jujutsu seem like a stronger contender to replace Git than I've seen so far.
I'm curious about its management of conflicts. I know that pmeunier has taken a lot of care to formally work out a theory of patches to drive Pijul, and that unsound or problematic notions of patches/conflicts can lead to serious problems— they say that's what led to all the performance problems with Darcs, right? I'd love if the comparison page on the repo wiki gave a little more detail than that Pijul's handling of conflicts seems 'similar'.
Storing the conflicts symbolically in this way lets you reproduce the conflicts later and even auto-resolve certain conflicts, but it doesn't address resolving the actual contents of conflicts. You could probably use Pijul as a jj backend and get the best of both worlds (if someone were to implement it).
This looks promising. One question I had after reading about its git compatibility is that they seem mostly focused on the use case where a Jujutsu user accesses a git repository (hosted by e.g. GitHub) with jj. But does it support the converse way of working, i.e. accessing a native Jujutsu repository with git?
I ask this because most developers are already quite familiar with the git CLI so in production use one would probably see developers co-working with jj and git in the same codebase. Or would the realistic production scenario be always using git (as opposed to native Jujutsu database) as the backing storage to allow accessing both with git and jj CLIs?
I would assume there would always be the expectation that you either use Jujitsu as a frontend to a git repo, or have a complete Jujitsu based remotes.
If you're going to work on and contribute to a project that is already using Jujitsu, it is reasonable to expect that you'd adapt your workflow to the project itself and not the other way around.
At this time, there's practically no reason to use the native backend. The backend exists mainly to make sure that it's possible to eventually add functionality that cannot easily be added to the Git backend.
Nice to see this posted here. I switched over to it about 2-3 weeks ago, and I haven't looked back. It took a lot of mental rewiring, but I really enjoy the workflow `jj` provides. There's no longer a time to think about what's being committed, because all file changes automatically amend the working copy commit. Of course, sometimes you don't want this, so you have things like the ability to split a commit into two, moving commits, etc. But having _everything_ operate on commits is really nice! Other niceities that I like:
- `jj log` is awesome for getting an overview of all your branches. If, like me, you have a lot of work in progress at once, this provides a great map
- Conflict resolution is really cool, as you can partially resolve conflicts, and then switch branches. Conflicts are also tracked specially, but I haven't done too much with this yet.
- The abbreviated changeset ids are really handy. I often will just `jj log` (or in my case just `jj` as that's the default), notice a changeset I want to rebase, then run `jj rebase -s qr -d master`. `qr` here is an abbreviated changeset id for a branch/commit, and usually much quicker than typing the branch name out! This will probably change when clap gets updated to support dynamic tab-completion though.
"As a Git power-user, you may think that you need the power of the index to commit only part of the working copy. However, Jujutsu provides commands for more directly achieving most use cases you're used to using Git's index for. For example, to create a commit from part of the changes in the working copy, you might be used to using git add -p; git commit. With Jujutsu, you'd instead use jj split to split the working-copy commit into two commits. To add more changes into the parent commit, which you might normally use git add -p; git commit --amend for, you can instead use jj squash -i to choose which changes to move into the parent commit."
Ugh, I was baited by myself here. I’m he scripts I use at the day git (which are the only way I tolerate hit, honestly. It does this, and also recursively merges from parent branches in commit.
We’re on an old school feature branch/master is production topology though.
Sometimes you have changes that are permanent to your repo (ie local workflow), that you always want to keep locally, but never push to the remote.
In git you would always leave the changes unstage, does that mean with jj you would always have to remove them before pushing? I haven’t found an answer on the linked page.
Side note: I really wish git had a way to mark commit has ‘no-push’ so they never leave your local copy, as an upgrade of the unstaged workflow.
Ooh, that's me! As in the comment, please don't do this.
It's been a while since I worked at a place doing this, and I'm on my phone, so some details may be fuzzy or wrong, but when I was figuring all this out, I remember this SO comment being really helpful:
Basically, there's two different APIs, and neither of them are designed for ignoring config files (but both of them happen to do things that look like ignoring config files, therefore get misused).
`assume-unchanged` is an optimisation tool that tells git that it shouldn't both checking files and folders that are expensive to check but never changed. If changes do ever happen, and git realises this, then git will remove the assume-unchanged flag - because clearly it wasn't true!
`skip-worktree` is a more aggressive version of this that tells git never to touch certain files or folders at all. But now it changes do happen to those files, it's not clear what git should do. Overwriting the files would cause data loss, but ignoring the files completely means that you miss out on upstream changes. And because you're telling git that these files should never be checked, there's no good way to handle merging and conflicts.
What typically happens with both of these flags is that they work well about 80% of the time, and then cause a lot of confusion that last 20% of the time.
The alternative is almost always some sort of configuration file that is directly referenced in .gitignore, and that therefore never gets checked in. (In addition, it's often useful to have a (e.g.) config.json.default file for getting new developers up and running quickly.) Any system that needs to be configured (database connections, API URLs, IP addresses, etc) should use this configuration file. Alternatively, I find environment variables, along with .env files for local development, to be really effective, because most things can already be configured that way.
This typically takes sightly longer to set up the first time, but will work very reliably from then on.
See also these changes in git's official documentation, and the commit message that explains why it was necessary:
For a long time I have been wishing that Git had a capability of "track this file locally, but don't transfer it when pushing or cloning". Such files would be listed using the same syntax as .gitignore, but in a different file, say .gitlocal or something.
Git kind of has some "local tracking" already: if I am not mistaken, lightweight tags behave like this. It would be cool if it could track files in the same way.
Under the hood it could be done via a separate database, stored not in .git but in .gitl directory or some such. The database in .git would behave as if both .gitignore and .gitlocal contribute to ignoring, and .gitl as if .gitlocal was .gitignore in reverse: it would ignore anything not covered in .gitlocal (and also ignore what's in .gitignore). Or something along these lines.
That won't work for local changes to files that are legitimately committed in the repo. Like if you need to set your local database, or your own dev hostname in config, for example.
gitignores are usually committed (they're treated like a normal file), so yes, in this context that would delete it from everyone.
There's .git/info/exclude, but that has some kinda large surprises if it excludes a tracked file and I don't recommend anyone use it unless they know what to look for and can always remember what they've excluded.
I have a $HOME/.gitignore that I use for this. (You can configure git to use that globally.) It's not a panacea, and I think other commenters are right that you should instead endeavor to organize things so that the project's own gitignore results in a sane workflow. But I think having permanently unstaged changes is worse.
In my experience, there is almost always a simpler way to handle these cases that involves being able to put a file in gitignore. This is one of the big advantages of environment variables, for example - they can typically be stored in a .env file (with a .env.default file checked in for new developers to copy from), and loaded dynamically in different configuration systems.
If my local setup requires me to ignore changes in checked-in files, I usually find that I need to handle configuration more cleanly.
(I did work on a project that made use of git update-index - this was a terrible mistake and caused pain every time we needed to update the config file in the repository. Please never go down this route!)
dotenv and direnv are more than enough for this task, I think, but you can make it as complicated as you care to with similar tooling that pulls secrets/config from the cloud (Doppler, parameter store, etc.)
Most places I've worked at have created tooling that more or less merges the two - sane defaults and non-sensitive values go into a `.env` file of some kind (.env, .env.development, whatever), and then a tool merges a subset of that config from a remote store which contains the stuff you don't want committed in the repo.
Usually used for connecting to a remote dev or staging instance if you can't or don't want to run the entire stack locally.
This is a smell that you should refactor how your configuration is done. What happens when there's a legit change to the config? If it's in git with `update-index`, you're going to have a hard time actually changing that file and getting the changes to the team. There are other reasons, but this is why things like 12 Factor recommend putting config in environment variables. It's made my life much easier.
https://12factor.net/config
So in your workflow you never "git commit -a"? So you have to always manually mark what you stage. Which is probably more work than always manually removing the changes you don't want to commit.
The ability to rewrite older commits easily in jj also looks like it would help with this usecase if you get it wrong once.
Concretely I think you would do is: Instead of staging part of your changes and then committing as in git, you would call jj split and split the commit into the part you want to keep local and the part you want to push. This way the local changes always stay in your working copy commit.
Even better, just commit the local changes once when you start. Work locally and before you push you call jj diffedit on your initial commit of the local changes and remove them. Now all the work you did since then will be automatically rebased on the edited initial commit and you can now push up. Instead of excluding your local edits every single time you just have to do it once before pushing.
I like seeing what I'm about to commit, so I always do `git commit -p` or `git add -i`. Most people where I work do the same, so I don't think this workflow is uncommon.
if I don't do this (actually I use magit to select each hunk to stage, but that's just add -p with a fancy interface) then I'll accidentally commit testing log lines and that sort of experimental code
I never, ever commit -a. That flag horrifies me. I want to choose, specifically, each line of code that I am going to publish.
I never `git commit -a`, because I'm paranoid that I've done something like named a variable "foo" as I'm just working through the logic and not caring about naming.
I'll then go back through, tidy and add all my changes.
"add -p" is great for showing you all the chunks of code you've written one-by-one and then you do "y" or "n" for whether you're adding them. Doing it like this means that you are reviewing what you've changed as a final check that you haven't left a debug line in, or forgotten part of what you meant to do. It's also a natural way of splitting up what you've done into two separate commits.
Actually, maybe I'm just a complete git, but I couldn't figure out how to `git reset HEAD~` my accidental commits, `git rebase -i HEAD~6`, format `jj log` more like `git log --color --oneline --graph --full-history` (which shows one-line-per-commit), `git checkout -p` (and obviously, `git add -p`), `git show HEAD~`, refer to N-th parents, e.g. `master~5`, and a bunch of other things...
It also feels a bit weird that new files are automatically committed without being manually approved, but I suppose this might theoretically help with some of git's annoyances.
I never do `git commit -a` precisely because I don’t want to randomly add files that I have in the repository. If I’m in a hurry, I’ll do `git add -u && git commit` to add changed files that were already in the repository. But, more typically, I use magit to stage exactly what I want to commit.
> Sometimes you have changes that are permanent to your repo (ie local workflow), that you always want to keep locally, but never push to the remote.
I appreciate that there are times when this has to be in the middle of an otherwise committed file, but it's worth avoiding that if at all possible and putting the locally-changed bit in a different file because, as others have pointed out, this is error prone. It feels like the equivalent of keeping your important files in the recycle bin because it's easily accessible.
For 90% of git users 90% of the time, the staging area is an inconvenience that adds an extra step.
Git should really steal the only thing I like about perforce, many uncommited change lists. It should be an easy workflow change to add 'named stages' along side the default stage. That way you can just leave changes in a different stage, tracked but uncommitted.
A safer way to keep temporary changes is to create a new local-only branch with a commit of those temporary changes. Then, whenever you need them, you rebase the branch on top of your working branch.
Usually these temporary changes are to config files, so rebasing shouldn’t create conflicts if you don’t otherwise modify the config file.
An alternative is to stash only those temporary changes but I find branches to be easier to work with.
Assuming in git you would keep it as untracked file in the working copy, I find jj’s automatic anonymous working copy commit better.
For once, it avoids the downsides of carrying untracked files around. Ever accidentally committed an untracked file somewhere deep in history during a long rebase?
Also I find it clearer what files are committed: every file - except if it is in .gitignore. Meanwhile in git you scourge through your untracked files before each commit to decide which ones you don’t want to add. Ever accidentally committed all files and forgot that there were untracked files you didn’t want?
It's not really different than using Git to work on the same branch. If you and the teammate commit to the same branch, then you'll need to resolve the divergence somehow, usually a merge or rebase, or choose to forcibly overwrite the other's changes.
To be clear, I am describing my actual usage of jj. It’s worth noting that there’s currently not a `jj pull` command, so divergences typically involve manually setting a branch pointer for me.
It mentions that it can be used with backends like Dropbox, but it would be wonderful if we finally had a system that could easily be used with IPFS. This is especially important for large data, since you can't store 1TB on github (and no, I don't count lfs, since you have to pay for it).
IPFS is the natural solution here, since everyone that wants to use the dataset has it locally anyway, and having thousands of sources to download from is better than just one.
So if this uses IPFS for the data repo, I'm switching immediately. If it doesn't, it's not worth looking into.
In the author’s presentation [0], the Google roadmap includes “custom working copy implementation for our internal distributed VFS”. The related graphic shows a “working copy” block connected to a “distributed file system block”.
This work might be extensible to include IPFS and other distributed virtual file systems.
1. When we ingest files or make new commits, how are these additions to the object store persisted?
2. When operations modify the working copy, how should these changes be reflected in the user's view of that working copy?
Ordinary git handles (2) by directly modifying the files on the filesystem. If you `git checkout` a branch, git will `rm` nonexistent files, `open()` and `write()` new ones, and adjust modification timestamps etc as needed. As you make changes to these files, some commands will occasionally "notice" that the file changed after the fact, and some may choose to modify the index to match.
the jj on github also does this, but inside Google, our concept of "working copy" needs to be disconnected from local files. Developers don't have their own local "working copy" backed by files on the ordinary filesystem; instead, we do all development inside a FUSE-mounted virtual FS called "client in the cloud" (CitC), so working anywhere inside our giant monorepo doesn't take any disk space (except caching). I think that's what the "Distributed file system" refers to - instead of modifying the local filesystem, jj would need to talk to whatever remote service provides the user's FUSE-backed view whenever the user uses `jj checkout` or some other jj operation that modifies the working copy.
When you speak of implementing IPFS storage, I think instead you want to keep the object store and operation log on IPFS while keeping the local working copy right on the ordinary file system, similar to how git-LFS keeps local files untouched while modifying the way they're persisted to the git object store.
Alternatively, perhaps we could imagine an IPFS backend similar to `jj git` and `jj native`, perhaps `jj ipfs push/pull`. Then, a completely local repository could push/pull to and from IPFS, completely agnostic of how the user's repository is stored on disk.
In any case, jujitsu's API surface is flexible enough to support any of these use cases since the author designed it from the ground up to smoothly support very different needs for internal and external users. Most users outside Google just want a familiar working copy containing ordinary files, and the fact that the repository structure happens to be backed by a git-like object store (linus' "git is just a merkle tree of files" philosophy) is incidental under the hood. That's just fine, even though most internal users will be interacting with a very different way of using jj when everything's said and done. Ideally, nobody needs to notice or care about the difference.
[2]: Linus Torvalds on git: “In many ways you can just see git as a filesystem — it’s content addressable, and it has a notion of versioning, but I really designed it coming at the problem from the viewpoint of a filesystem person (hey, kernels is what I do), and I actually have absolutely zero interest in creating a traditional SCM system.”
If you're storing 1TB of binary files in git, you're just doing it wrong anyways. You have a bunch of other tools and capabilities for doing this in a way that doesn't make your repository nightmarishly stupid to deal with because of its size.
Nexus, Artifactory, Packages (deb, rpm, nix), Cache, GitHub Releases... There are so many places you can grab a signed binary from that are just outright better for the health of your repo, and will respect your developers time.
The issue is not limited to archives, artifacts, and packaging. Game projects, for example, have large directories with many binary assets which need to be change-controlled. Artifact repositories address distribution, in a way, but don't generally support change control much if at all.
(And yeah, git's historically a poor choice for this – so you may see companies sticking with Perforce or other non-distributed solutions.)
I didn't exactly intend it to operate precisely the same way that git does, but rather to have extensions of git that unify the system into one easy to use version control for data and code.
In most projects today, the code is (or generates, anyway) the data. This is true for materials science in physics, neural networks, and creation of databases via ETL. So, it would make sense to remove the requirement of making users of some software to regenerate this data, which may take 2 months on a supercomputer. Downloading that would be much faster. You can put it on a university server, or AWS, but now the data is in some system that is not guaranteed to be there. In fact, it's almost guaranteed to *not* be there in a very short period of time (people move positions and lose their access to these servers constantly).
So the very obvious best solution is IPFS for distribution of the data, but it does need to be linked to the git repo somehow. Of course, the data may not be simple or textual and play well with simple text based diffs for version control, so using something like borg can solve the issue of both data privacy, if needed, and block based diffs.
So this isn't to suggest "just git everything", but rather to say, 'if there's a new version control system for data and code, it's probably added some improvements to fit, and this could be a direction that makes sense'.
So I was checking to see if it had gone that direction yet.
Maybe I came across as accusative, but I'm genuinely curious. Do you have 1Tb text files or this is some kind of media management for video production, something like that?
Because it's a source control system, which means it's intended to store source code, not the artifacts generated from the source code. It seems far-fetched that anyone would manage to author 1 TB of source code.
The point is to have an easy way to distribute code as data. This is important for many areas, such as training neural networks (code with proper seeds can ensure the weights output by training), various applications in basic physics, database creation via ETL, etc.
If the choice is "run this code in the repo, wait 10 weeks while it's running, and retrieve the 50GB file", vs "download this file", of course, the latter is better. But many of these processes exist in academia, wherein you are essentially guaranteed to lose access to the server and maintenance of that file for download, it can get pretty annoying. Additionally, there's no seamless way of distributing it (it's in the docs, point somewhere else that may or may not exist, etc).
Since essentially all big data is really just code, it would make much more sense to tie these directly at the hip. So, a git/repo commit hash that is a key directly to the IPFS data hash would fix this problem directly.
So it's not "wanting big files in a git repo" (an obvious no-no, since central servers shouldn't be used for storing large data, and github centralized repos only should store single digit MB or so), it's wanting to relieve the cost of running processes that may require supercomputers weeks of processing for QM calculations, etc by providing a guaranteed hash pairing of the output of the code.
IPFS didn't always seem to have pinning services at BitTorrent prices($3-$5 a month with 1TB bandwidth, no crypto wallet needed) and the client usew tens to hundreds of kb per second idle.
Unfortunately, the model of putting every block in the DHT instead of having roots mode be the default, and then spamming your wantlist to tons of peers seems to still be at least partly around.
Right now IPFS looks pretty good thanks to the gateways and services, so I would imagine well see more of it in the future, but I can see why it took so long.
Pijul needs a 1.0 release if it wants wide adoption. I don't understand why they wait.
Meanwhile, if rebasing on git is an issue, you should probably try stacked-git (https://stacked-git.github.io/). It manages commits as a stack of patches - like quilt, but on top of git.
git-branchless also fits in the same vein as stacked-git with everyone’s favorite commands like `record`, `unrecord`, & `smartlog`.
That said, I’ve been using darcs this past month & the performance issue seem overblown (/ largely fixed, tho not fully addressed) while the stability & usability are there. Email support is a bonus.
"working copy is automatically committed" seems like a good idea at first glance, but there are many situations where this is not a good idea:
- when new artefact files are added and you have not yet added them to .gitignore, they'll be automatically committed
- when you have added ignored files in one branch and switch to another branch, the files will still be in your working copy but not listed in your .gitignore file, and would then be automatically committed
- staging only some files and comitting is much easier than splitting a commit after the fact
Constantly having to split after the auto-commits feels like I have to keep fighting this automated commit system. It's a bit like the autosave feature of text editors, which some people find useful, but I never felt like it's needed.
A big downside I can see is that sensitive information might end up in the repo forever because I forgot to revert the auto commit.
I always end up reading through my commit diffs while committing and then maybe like a third of the time I edit something about them. So for me this seems like it could be a natural acknowledgement of that; just make that editing after the fact be the primary / only workflow.
Haven't used this yet so no idea if it's actually great or terrible, but I like to see ingrained conventional wisdom challenged like this. We're always in some local maximum, so I think it's often interesting to be pushed in some new direction on the gradient.
I think you are misunderstanding the autocommit, but maybe I am.
I think you don't need to do this after every autocommit but only before the manual ones? Isn't that how amending commits in git works? The unamended commit is no longer around, right?
Also it seems to me that you can rewrite old commits in jj to get rid of accidentally committed information a lot more easily than you can remove it from when you accidentally commit it to git...
Meta comment: Bit frustrating to see so many downvotes in these threads when people are just trying to grok what the jj model means concretely for important workflows.
I most definitely agree. To be honest I know I go against the current here, but so far there is nothing I really like from what I’ve seen in jj.
I should try it for real, see how it feels when using it to get a better sense of it.
I'm of the same opinion as you, here. I generally have 10+ "extra" files in my project directory (output files, notes, one-off scripts for certain things, etc). When I add files to a commit, I do it by filename, never "everything that's new/changed". I don't have a use case for "everything I've created/changed goes into the commit, always".
> switch to another branch, the files will still be in your working copy but not listed in your .gitignore file
This is a failing of git, imo. There should be a .local.gitignore to somesuch, that is "added to" .gitignore. It's VERY common for me to have files that I want ignore, but are specific to me; they don't belong in the project's .gitignore. I know there are ways to do this, but all of them are clunky. There should be a simple, out of the box way to do it.
i always have a large pile of temp files and such that should never be committed but it's never been a problem because i never use `git add .` i use `git add -u` which is "only add things that are already being tracked"
there's a newer flag for the same thing but my brain is hardcoded to the old "-u" option from "back in the day". It feels like a "simple, out of the box way to do it" to me.
I do the same, but in all the mess that git status outputs, I occasionally forget to git add some required source code - which leaves entire chains of commits in my repository in a broken, impossible to compile or run state. Worse, I don’t notice until I try to run the code on another computer - and find I’m missing data.
Having a “local gitignore” makes a lot more sense. I’d like to be more explicit about which local files should be ignored.
I've used that file before but... it always felt sort of hacky. And every time I want to use it, I need to figure out where it is. It really feels like there should be a more obvious, straight forward way to do it. Admittedly, "user friendly" is not git's strong suit.
The .git directory seems like the obvious location for it; I agree that info/exclude isn't the most obvious path (I had to look up what the exact path was again for my previous comment), but all things considered that seems like a minor issue. I suppose you could symlink .git/ignore to .git/info/exclude or something in the git template.
These might be the clunky way your talking about but you can have “private” git ignored either in ‘.git/info/exclude’ in the repo or in the ‘core.excludesfile’ set in your git config. The later is quite nice because you could actually version control that elsewhere if you version control your system config.
> - staging only some files and comitting is much easier than splitting a commit after the fact
I see this project as a challenge to that conventional wisdom. This view is certainly the one I have embedded in my mind. But is it right? I end up fixing up the index and amending commits post facto quite often. I can also do it pre facto. But in a world where you can't fully avoid editing after the fact, mightn't it be better to have a single workflow for this kind of editing? That is, if you can't totally get rid of post facto commit editing (which I think is reality), can you actually get rid of pre facto editing, and be left with just one editing workflow? If so, maybe that's good!
I haven't used this yet, but this strikes me as a very plausible attack on a conventional wisdom that we take for granted but may not actually be doing us any favors.
Obviously it depends on your workflow. If you're working on one feature at a time and only saving things out of your editor that you want to go to the project you cloned, it probably is.
That's not my world at all. I have logging and instrumentation all over the place when debugging. I have multiple features in flight. I have tweak patches that I maintain externally I don't want committed. I'm trying things for someone else, etc...
The index is the tool that allows you to manage all that without hating the process. That's why it was invented, in fact. Now, if you don't need it it seems needless, like someone is making you type "git add" for no reason. And jj seems to be aimed at that demographic, and that's fine.
But jj then needs to jump through some odd hoops to get back the "partial commit" workflow that is natural in git. Meh. Not a win for me personally.
There's also the problem of "patch hygiene", which flows like this tend to do pretty badly with. It's routine in some projects (Linux is the flagship here, obviously) to demand clean commit messages and perfect bisectability. Pull requests in that world routinely need to be split/squashed/reworked during review and when moving across branches. You need tooling to do that.
This, to me, reads like you're incredibly used to one workflow so any different one seems "odd". But I don't think it's odd, I think it's just different. If we'd all been doing it the jj way for a decade and a half, I think the "index" and "staging" would be the thing that seems odd to us.
So for instance, to work with the index with all your logging and instrumentation and debugging that you describe, you must be doing a lot of `git add -i` over the course of your workflow to partially stage things. Maybe you're also stashing the unstaged changes to make sure what you have in the index works on its own.
Well, I don't think that interactive add workflow actually seems any different fundamentally from copious use of `split`ing changes out of the working copy commit with jj. And I don't think stashing seems fundamentally different from rebasing the changes you split onto the branch you want to apply them to, leaving behind all the logging and instrumentation.
Yes, it seems like a lot of commit editing. But this is downstream of your workflow that requires some step to differentiate between what to check in and what not to, at a fine-grained level. You would have to split things up manually with jj, but you're already splitting things up manually with git, it's just a different workflow with a different command, but I'm not convinced it's any harder. Just different.
> Pull requests in that world routinely need to be split/squashed/reworked during review and when moving across branches. You need tooling to do that.
Yes, but jj seems to come with all that tooling! And I think its model of implementing all that on top of commits, rather than a combination of the concepts of unstaged+staged+stashed+commits, may actually be a better fundamental model.
But again, I still haven't used jj. My workflow is a lot like yours, at least oftentimes. I split stuff up ahead of time, I use the index a lot, I stash stuff a lot, I edit commits, I rebase interactively, I squash and split. But by my reading of jj, I think I can accomplish all that stuff, and just maybe in a better way.
I agree. We've gotten used to pick what should go into the commit after we've done the work.
There's a very serious and glaring error in doing that if you don't commit everything. When someone checks out your commit in a clean repo, his codebase is in a state that has never existed before, not even on your computer.
Some people (I think we've all done it at least once) split their work into multiple commits after a few days of hacking. Like "Adding data services for X", "UI features for X". This is a bad example, but you get the point.
Normally this doesn't cause a problem since all commits get pushed at the same time. But it can easily break things like git-bisect (which is just an amazing tool for finding astonishing bugs).
> staging only some files and comitting is much easier than splitting a commit after the fact
Re this point, how is it any different? "Staging" the files is essentially the same as splitting the commit, anyways — it's just that the newly-split contents go into a "staging area" vs a commit. Do you mean that the tooling to accomplish this is not good?
I am working on a TUI tool for Git/jj to handle staging/splitting. Could you tell me more about what workflow would be problematic for you? For example, if you had to select lines that go in the first commit/staging area, does that work? Would it be better or worse if, for each line, you had to decide whether it would go into the first commit/staged vs second commit/unstaged area? Do you ever need to invert the first commit/staging area vs second commit/unstaged area?
> - when new artefact files are added and you have not yet added them to .gitignore, they'll be automatically committed
From only reading the README, my understanding is that you would just add the files to .gitignore and at the next auto commit they are removed again from the anonymous working copy commit. I agree it feels unintuitive that files are added and removed from the anonymous working copy commit, instead of not added in the first place. But since the anonymous working copy commit is largely invisible to you it shouldn’t make a difference.
For point 1, they could easily diff the already tracked to the new ones right ?
Interesting nonetheless, I never liked staging in git but I never hit the case you mention so now I realize there's a need for a phase between a file system and a commit
This looks incredibly good to me. The working copy as "commit being worked on right now" is a much simpler data structure than the "working copy + stash + staging area" of git.
I really hope this succeeds... VSC support would be a great start...
This project is a great example of subliminal marketing.
It is less apparent now but still, the repeated flex of “Google”, 20% project etc when no typical reader would assume them is classic corporate charlatanry.
The plan for how the add signed commits is there, and the work isn't that hard (especially as gitoxide continues to add functionality), it just has to be pushed over the line and I've been a bit slack on getting that going.
There's definitely nothing foundational blocking it though and it will happen one day if you'd like to give it a go in the meantime.
No daemon, it happens 'whenever you run a command'.
> Commands snapshot the working copy before they start, then the update the repo, and then the working copy is updated (if the working-copy commit was modified). Almost all commands (even checkout!) operate on the commits in the repo, leaving the common functionality of snapshotting and updating of the working copy to centralized code.
I think your question is getting into the field of martial arts lineage. People might have their own narratives/ mythologies around this, but here's the most neutral way I can explain it: As techniques and styles evolve over the years, people come up with new names to describe those styles. Name similarities will often imply closer ties in lineage.
As a Brazilian Jiu Jitsu practitioner, I cringe when I see it spelled any other way, but also I have to recognize that I only feel that way because I have more exposure to that specific martial art/spelling.
There are multiple romanisation systems for Japanese, but the most common one is Hepburn. In Japan, Kunrei-shiki is sometimes used (especially by the government), which is designed with Japanese speakers in mind (vs Hepburn which was designed with English speakers in mind).
It's jujutsu in both, but there are varying ways of representing the long vowel on the first "u" -- either omitting it (jujutsu), using a macron or circumflex (jūjutsu), or repeating the vowel (juujutsu).
Ju-jitsu and jiu-jitsu are not correct in any romanisation system that I know of, I'm not really sure how they came about. Probably historical accident.
So is it pronounced like it looks here? I've never seen this spelling before, and IME (and I assume GP's) everyone says 'jew-jit-sue'. But it's more like 'jew-jut-sue' (as in to jut out)?
To use your phonetic spelling juujutsu would be more like jew-jew-tsue, with the first jew being stretched out a little. That is, both ju syllables would be pronounced the same way. The tsu on the end is its own syllable.
Unless we're talking about the English word, which I'm not sure how to pronounce. If we're talking about the American word then it's really just like a lot of other murkin words: butcher it any way you like.
The base assumption, though, is that we're talking about 柔術 (with a long u), not 呪術 (without a long u). Without a clarification from the author, it's hard to tell which one it's supposed to be.
Just to add clarification: 柔術 is the martial art, while 呪術 means sorcery.
The first kanji in 呪術 means curse, so while "sorcery" could be a cool name for a software project in English, a Japanese speaker would probably have apprehensions about using 呪術.
I cringe whenever I see it spelt with an "i." It would "jujutsu" or similar in either of the typical romanization schemes (Hepburn, etc). "Ji" would be pronounced "jee" in any of the standard romanizations.
Pronunciation is more like joo-juh-tsu. "Tsu" is its own syllable.
Realistically, Jujutsu is a pretty new project and there are many features to add and lots and lots of code that needs to be written. There is a lot of talking that needs to happen, in other words. So, most of the developers and committers actively hang out and discuss things in Discord, yes, because it's convenient, and fast, and allows all the interested parties to be alerted. I say this as someone who started hacking/working on Jujutsu recently; having Discord has been nice for such a new project.
The reality is that the project is in a part of its life where active discussion and feedback with the userbase is pretty valuable. So, you just go where the users are, and Discord is a big place for that.
GitHub Issues and GitHub Discussions are also actively used by a lot of people, and you can just post things there. Every major committer watches those venues as well, AFAIK, I know my notifications are set up for it.
Over time, assuming things are wildly successful, you'd probably have a lot of different venues to discuss things. I would view Discord mostly in the context of a new project that wants feedback, in that regard.
270 comments
[ 2.0 ms ] story [ 261 ms ] threadI've been playing with a homegrown centralized VCS client and server project, aimed at basically being the VCS host for monorepos, and this looks like a much better foundation for the VCS layer.
Jujutsu: A Git-Compatible VCS - Git Merge 2022:
Video:
https://youtu.be/bx_LGilOuE4
Slides:
https://docs.google.com/presentation/d/1F8j9_UOOSGUN9MvHxPZX...
That's got to feel good!
Full disclosure: This is my tool
According to the readme, “jj” was chosen because it was easy to type, and a name was found to fit that.
Really?
The wiki audio sample[1] seems fairly close to how people in the US pronounce the word, just with a more emphasized first syllable.
---
1. https://upload.wikimedia.org/wikipedia/commons/f/f1/Ja-Jujut...
Ambiguous romanization is weird, I wish romanization rules would just follow IME rules (except in the case of consecutive n's, that'd look weird).
That's kind of tautological ...
However, yes, that is certainly true. Japanese has a pitch accent rather than a stress accent and all syllables are roughly the same length. English speakers don't tend to talk that way.
Looks very interesting, will definitely have to give this a deeper try.
However, if you don't manually put a branch on it, it'll never get pushed and will stay on your machine only.
You can sit on this personal commit and rebase it on top of any other commit to move around the repo, again and again if you like.
In case it helps your use case:
This would ignore changes while you're testing - but you have to remember to turn it off or, iiuc, you won't pull intentional changes either.You might find hooks useful too. Not to assume your knowledge, these are shell scripts placed in .git/hooks that are invoked, e.g., before commit or before push. You could have it parse git status, detect changes to <file>, prompt for confirmation if changed and remove from working set if the change is unintentional.
(the UX around this is the shortcoming of all current DVCS..)
* Disclaimer: I'm one of the creators/maintainers of the project.
That said, any git users will still get slowed down - only jj users would see the benefit. Git does have better features these days for shallow clones though and even git is on the way to killing off the need for LFS.
JJ doesn't seem to do that though, right?
I'm a bit confused by this. I don't think that's what monorepo means, is it? Monorepo is what you choose to put in a repo? And I thought Mercurial was extremely similar to Git as it's also a DVCS?
[0] https://hginit.com
SQLite uses a custom vcs called Fossil, but doesn't make much effort to push broader adoption (afaics) so it remains academic at this point.
Jujutsu keeping git compatibility looks like a differentiator that reduces cost of adoption. I'm excited!
I'm curious about its management of conflicts. I know that pmeunier has taken a lot of care to formally work out a theory of patches to drive Pijul, and that unsound or problematic notions of patches/conflicts can lead to serious problems— they say that's what led to all the performance problems with Darcs, right? I'd love if the comparison page on the repo wiki gave a little more detail than that Pijul's handling of conflicts seems 'similar'.
Storing the conflicts symbolically in this way lets you reproduce the conflicts later and even auto-resolve certain conflicts, but it doesn't address resolving the actual contents of conflicts. You could probably use Pijul as a jj backend and get the best of both worlds (if someone were to implement it).
I ask this because most developers are already quite familiar with the git CLI so in production use one would probably see developers co-working with jj and git in the same codebase. Or would the realistic production scenario be always using git (as opposed to native Jujutsu database) as the backing storage to allow accessing both with git and jj CLIs?
If you're going to work on and contribute to a project that is already using Jujitsu, it is reasonable to expect that you'd adapt your workflow to the project itself and not the other way around.
At this time, there's practically no reason to use the native backend. The backend exists mainly to make sure that it's possible to eventually add functionality that cannot easily be added to the Git backend.
Thank goodness. This has been needed for a long time.
Props to the author Martin for investing himself in creating elegance!
- `jj log` is awesome for getting an overview of all your branches. If, like me, you have a lot of work in progress at once, this provides a great map
- Conflict resolution is really cool, as you can partially resolve conflicts, and then switch branches. Conflicts are also tracked specially, but I haven't done too much with this yet.
- The abbreviated changeset ids are really handy. I often will just `jj log` (or in my case just `jj` as that's the default), notice a changeset I want to rebase, then run `jj rebase -s qr -d master`. `qr` here is an abbreviated changeset id for a branch/commit, and usually much quicker than typing the branch name out! This will probably change when clap gets updated to support dynamic tab-completion though.
Also from the documentation:
https://github.com/martinvonz/jj/blob/main/docs/git-comparis...
"As a Git power-user, you may think that you need the power of the index to commit only part of the working copy. However, Jujutsu provides commands for more directly achieving most use cases you're used to using Git's index for. For example, to create a commit from part of the changes in the working copy, you might be used to using git add -p; git commit. With Jujutsu, you'd instead use jj split to split the working-copy commit into two commits. To add more changes into the parent commit, which you might normally use git add -p; git commit --amend for, you can instead use jj squash -i to choose which changes to move into the parent commit."
We’re on an old school feature branch/master is production topology though.
In git you would always leave the changes unstage, does that mean with jj you would always have to remove them before pushing? I haven’t found an answer on the linked page.
Side note: I really wish git had a way to mark commit has ‘no-push’ so they never leave your local copy, as an upgrade of the unstaged workflow.
I do this too, but I quite frequently forget that I've done it and "git commit -am" and end up pushing my private changes anyway.
i have these 2 aliases assume = update-index --skip-worktree unassume = update-index --no-skip-worktree
"assume" as in "assume it's unchanged / not wanted"
which lets me say "git assume path/to/file" and then "unassume" it when/if i want to commit it.
https://news.ycombinator.com/item?id=36954723
It's been a while since I worked at a place doing this, and I'm on my phone, so some details may be fuzzy or wrong, but when I was figuring all this out, I remember this SO comment being really helpful:
https://stackoverflow.com/a/23806990
Basically, there's two different APIs, and neither of them are designed for ignoring config files (but both of them happen to do things that look like ignoring config files, therefore get misused).
`assume-unchanged` is an optimisation tool that tells git that it shouldn't both checking files and folders that are expensive to check but never changed. If changes do ever happen, and git realises this, then git will remove the assume-unchanged flag - because clearly it wasn't true!
`skip-worktree` is a more aggressive version of this that tells git never to touch certain files or folders at all. But now it changes do happen to those files, it's not clear what git should do. Overwriting the files would cause data loss, but ignoring the files completely means that you miss out on upstream changes. And because you're telling git that these files should never be checked, there's no good way to handle merging and conflicts.
What typically happens with both of these flags is that they work well about 80% of the time, and then cause a lot of confusion that last 20% of the time.
The alternative is almost always some sort of configuration file that is directly referenced in .gitignore, and that therefore never gets checked in. (In addition, it's often useful to have a (e.g.) config.json.default file for getting new developers up and running quickly.) Any system that needs to be configured (database connections, API URLs, IP addresses, etc) should use this configuration file. Alternatively, I find environment variables, along with .env files for local development, to be really effective, because most things can already be configured that way.
This typically takes sightly longer to set up the first time, but will work very reliably from then on.
See also these changes in git's official documentation, and the commit message that explains why it was necessary:
https://github.com/git/git/commit/1b13e9032f039c8cdb1994dd09...
Git kind of has some "local tracking" already: if I am not mistaken, lightweight tags behave like this. It would be cool if it could track files in the same way.
Under the hood it could be done via a separate database, stored not in .git but in .gitl directory or some such. The database in .git would behave as if both .gitignore and .gitlocal contribute to ignoring, and .gitl as if .gitlocal was .gitignore in reverse: it would ignore anything not covered in .gitlocal (and also ignore what's in .gitignore). Or something along these lines.
There's .git/info/exclude, but that has some kinda large surprises if it excludes a tracked file and I don't recommend anyone use it unless they know what to look for and can always remember what they've excluded.
Git will fight you if you ignore .config files which are actually used.
If my local setup requires me to ignore changes in checked-in files, I usually find that I need to handle configuration more cleanly.
(I did work on a project that made use of git update-index - this was a terrible mistake and caused pain every time we needed to update the config file in the repository. Please never go down this route!)
Most places I've worked at have created tooling that more or less merges the two - sane defaults and non-sensitive values go into a `.env` file of some kind (.env, .env.development, whatever), and then a tool merges a subset of that config from a remote store which contains the stuff you don't want committed in the repo.
Usually used for connecting to a remote dev or staging instance if you can't or don't want to run the entire stack locally.
The ability to rewrite older commits easily in jj also looks like it would help with this usecase if you get it wrong once.
Concretely I think you would do is: Instead of staging part of your changes and then committing as in git, you would call jj split and split the commit into the part you want to keep local and the part you want to push. This way the local changes always stay in your working copy commit.
Even better, just commit the local changes once when you start. Work locally and before you push you call jj diffedit on your initial commit of the local changes and remove them. Now all the work you did since then will be automatically rebased on the edited initial commit and you can now push up. Instead of excluding your local edits every single time you just have to do it once before pushing.
I like seeing what I'm about to commit, so I always do `git commit -p` or `git add -i`. Most people where I work do the same, so I don't think this workflow is uncommon.
I never, ever commit -a. That flag horrifies me. I want to choose, specifically, each line of code that I am going to publish.
I'll then go back through, tidy and add all my changes.
"add -p" is great for showing you all the chunks of code you've written one-by-one and then you do "y" or "n" for whether you're adding them. Doing it like this means that you are reviewing what you've changed as a final check that you haven't left a debug line in, or forgotten part of what you meant to do. It's also a natural way of splitting up what you've done into two separate commits.
Also learning about progress git workflows though, so that's cool.
> error: unexpected argument '-i' found
Actually, maybe I'm just a complete git, but I couldn't figure out how to `git reset HEAD~` my accidental commits, `git rebase -i HEAD~6`, format `jj log` more like `git log --color --oneline --graph --full-history` (which shows one-line-per-commit), `git checkout -p` (and obviously, `git add -p`), `git show HEAD~`, refer to N-th parents, e.g. `master~5`, and a bunch of other things...
It also feels a bit weird that new files are automatically committed without being manually approved, but I suppose this might theoretically help with some of git's annoyances.
I appreciate that there are times when this has to be in the middle of an otherwise committed file, but it's worth avoiding that if at all possible and putting the locally-changed bit in a different file because, as others have pointed out, this is error prone. It feels like the equivalent of keeping your important files in the recycle bin because it's easily accessible.
For 90% of git users 90% of the time, the staging area is an inconvenience that adds an extra step.
Worktrees allow you to have multiple branches open potentially with dirty state on any or all of them
https://git-scm.com/docs/git-worktree
My feature request is about having n stages instead of one so uncommitted work can be organized better while staying in the usual working copy.
How would I accomplish this with worktrees?
Disclaimer : I'm not on my computer, I know a file like that exists, it may not be at this exact location
Usually these temporary changes are to config files, so rebasing shouldn’t create conflicts if you don’t otherwise modify the config file.
An alternative is to stash only those temporary changes but I find branches to be easier to work with.
For once, it avoids the downsides of carrying untracked files around. Ever accidentally committed an untracked file somewhere deep in history during a long rebase?
Also I find it clearer what files are committed: every file - except if it is in .gitignore. Meanwhile in git you scourge through your untracked files before each commit to decide which ones you don’t want to add. Ever accidentally committed all files and forgot that there were untracked files you didn’t want?
It mentions that it can be used with backends like Dropbox, but it would be wonderful if we finally had a system that could easily be used with IPFS. This is especially important for large data, since you can't store 1TB on github (and no, I don't count lfs, since you have to pay for it).
IPFS is the natural solution here, since everyone that wants to use the dataset has it locally anyway, and having thousands of sources to download from is better than just one.
So if this uses IPFS for the data repo, I'm switching immediately. If it doesn't, it's not worth looking into.
This work might be extensible to include IPFS and other distributed virtual file systems.
[0] https://docs.google.com/presentation/d/1F8j9_UOOSGUN9MvHxPZX...
1. When we ingest files or make new commits, how are these additions to the object store persisted?
2. When operations modify the working copy, how should these changes be reflected in the user's view of that working copy?
Ordinary git handles (2) by directly modifying the files on the filesystem. If you `git checkout` a branch, git will `rm` nonexistent files, `open()` and `write()` new ones, and adjust modification timestamps etc as needed. As you make changes to these files, some commands will occasionally "notice" that the file changed after the fact, and some may choose to modify the index to match.
the jj on github also does this, but inside Google, our concept of "working copy" needs to be disconnected from local files. Developers don't have their own local "working copy" backed by files on the ordinary filesystem; instead, we do all development inside a FUSE-mounted virtual FS called "client in the cloud" (CitC), so working anywhere inside our giant monorepo doesn't take any disk space (except caching). I think that's what the "Distributed file system" refers to - instead of modifying the local filesystem, jj would need to talk to whatever remote service provides the user's FUSE-backed view whenever the user uses `jj checkout` or some other jj operation that modifies the working copy.
When you speak of implementing IPFS storage, I think instead you want to keep the object store and operation log on IPFS while keeping the local working copy right on the ordinary file system, similar to how git-LFS keeps local files untouched while modifying the way they're persisted to the git object store.
Alternatively, perhaps we could imagine an IPFS backend similar to `jj git` and `jj native`, perhaps `jj ipfs push/pull`. Then, a completely local repository could push/pull to and from IPFS, completely agnostic of how the user's repository is stored on disk.
In any case, jujitsu's API surface is flexible enough to support any of these use cases since the author designed it from the ground up to smoothly support very different needs for internal and external users. Most users outside Google just want a familiar working copy containing ordinary files, and the fact that the repository structure happens to be backed by a git-like object store (linus' "git is just a merkle tree of files" philosophy) is incidental under the hood. That's just fine, even though most internal users will be interacting with a very different way of using jj when everything's said and done. Ideally, nobody needs to notice or care about the difference.
[1]: More about Google's internal VCS needs: https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
[2]: Linus Torvalds on git: “In many ways you can just see git as a filesystem — it’s content addressable, and it has a notion of versioning, but I really designed it coming at the problem from the viewpoint of a filesystem person (hey, kernels is what I do), and I actually have absolutely zero interest in creating a traditional SCM system.”
(And yeah, git's historically a poor choice for this – so you may see companies sticking with Perforce or other non-distributed solutions.)
In most projects today, the code is (or generates, anyway) the data. This is true for materials science in physics, neural networks, and creation of databases via ETL. So, it would make sense to remove the requirement of making users of some software to regenerate this data, which may take 2 months on a supercomputer. Downloading that would be much faster. You can put it on a university server, or AWS, but now the data is in some system that is not guaranteed to be there. In fact, it's almost guaranteed to *not* be there in a very short period of time (people move positions and lose their access to these servers constantly).
So the very obvious best solution is IPFS for distribution of the data, but it does need to be linked to the git repo somehow. Of course, the data may not be simple or textual and play well with simple text based diffs for version control, so using something like borg can solve the issue of both data privacy, if needed, and block based diffs.
So this isn't to suggest "just git everything", but rather to say, 'if there's a new version control system for data and code, it's probably added some improvements to fit, and this could be a direction that makes sense'.
So I was checking to see if it had gone that direction yet.
This was done with SVN projects. Keeping the blobs out of your source repos has been the preferred way for a long time.
[Edit] The only folks who seem to want to do this are game developers, and they are generally not people you would want to emulate.
> Keeping the blobs out of your source repos has been the preferred way for a long time.
This is just appeal to tradition.
It might be, but the arguement was that we don't do it because of git.
We haven't been doing it for a long time, but that's not because of git.
If the choice is "run this code in the repo, wait 10 weeks while it's running, and retrieve the 50GB file", vs "download this file", of course, the latter is better. But many of these processes exist in academia, wherein you are essentially guaranteed to lose access to the server and maintenance of that file for download, it can get pretty annoying. Additionally, there's no seamless way of distributing it (it's in the docs, point somewhere else that may or may not exist, etc).
Since essentially all big data is really just code, it would make much more sense to tie these directly at the hip. So, a git/repo commit hash that is a key directly to the IPFS data hash would fix this problem directly.
So it's not "wanting big files in a git repo" (an obvious no-no, since central servers shouldn't be used for storing large data, and github centralized repos only should store single digit MB or so), it's wanting to relieve the cost of running processes that may require supercomputers weeks of processing for QM calculations, etc by providing a guaranteed hash pairing of the output of the code.
Unfortunately, the model of putting every block in the DHT instead of having roots mode be the default, and then spamming your wantlist to tons of peers seems to still be at least partly around.
Right now IPFS looks pretty good thanks to the gateways and services, so I would imagine well see more of it in the future, but I can see why it took so long.
I need to try it out at $WORK since constant rebasing on a busy repo with a hundred or so committers is not fun.
Meanwhile, if rebasing on git is an issue, you should probably try stacked-git (https://stacked-git.github.io/). It manages commits as a stack of patches - like quilt, but on top of git.
That said, I’ve been using darcs this past month & the performance issue seem overblown (/ largely fixed, tho not fully addressed) while the stability & usability are there. Email support is a bonus.
- when new artefact files are added and you have not yet added them to .gitignore, they'll be automatically committed
- when you have added ignored files in one branch and switch to another branch, the files will still be in your working copy but not listed in your .gitignore file, and would then be automatically committed
- staging only some files and comitting is much easier than splitting a commit after the fact
Check out the documentation, many of the cases you are concerned about are explicitly mentioned:
https://github.com/martinvonz/jj/blob/main/docs/git-comparis...
A big downside I can see is that sensitive information might end up in the repo forever because I forgot to revert the auto commit.
Haven't used this yet so no idea if it's actually great or terrible, but I like to see ingrained conventional wisdom challenged like this. We're always in some local maximum, so I think it's often interesting to be pushed in some new direction on the gradient.
I think you don't need to do this after every autocommit but only before the manual ones? Isn't that how amending commits in git works? The unamended commit is no longer around, right?
Also it seems to me that you can rewrite old commits in jj to get rid of accidentally committed information a lot more easily than you can remove it from when you accidentally commit it to git...
Meta comment: Bit frustrating to see so many downvotes in these threads when people are just trying to grok what the jj model means concretely for important workflows.
> switch to another branch, the files will still be in your working copy but not listed in your .gitignore file
This is a failing of git, imo. There should be a .local.gitignore to somesuch, that is "added to" .gitignore. It's VERY common for me to have files that I want ignore, but are specific to me; they don't belong in the project's .gitignore. I know there are ways to do this, but all of them are clunky. There should be a simple, out of the box way to do it.
there's a newer flag for the same thing but my brain is hardcoded to the old "-u" option from "back in the day". It feels like a "simple, out of the box way to do it" to me.
Having a “local gitignore” makes a lot more sense. I’d like to be more explicit about which local files should be ignored.
You can even ignoring changes in files that are already tracked with `git update-index --assume-unchanged <file>`
I think .git/info/exclude is what you're looking for? (and also the global ~/.config/git/ignore)
The same location as .gitignore seems like the obvious location for it, to me.
I see this project as a challenge to that conventional wisdom. This view is certainly the one I have embedded in my mind. But is it right? I end up fixing up the index and amending commits post facto quite often. I can also do it pre facto. But in a world where you can't fully avoid editing after the fact, mightn't it be better to have a single workflow for this kind of editing? That is, if you can't totally get rid of post facto commit editing (which I think is reality), can you actually get rid of pre facto editing, and be left with just one editing workflow? If so, maybe that's good!
I haven't used this yet, but this strikes me as a very plausible attack on a conventional wisdom that we take for granted but may not actually be doing us any favors.
Obviously it depends on your workflow. If you're working on one feature at a time and only saving things out of your editor that you want to go to the project you cloned, it probably is.
That's not my world at all. I have logging and instrumentation all over the place when debugging. I have multiple features in flight. I have tweak patches that I maintain externally I don't want committed. I'm trying things for someone else, etc...
The index is the tool that allows you to manage all that without hating the process. That's why it was invented, in fact. Now, if you don't need it it seems needless, like someone is making you type "git add" for no reason. And jj seems to be aimed at that demographic, and that's fine.
But jj then needs to jump through some odd hoops to get back the "partial commit" workflow that is natural in git. Meh. Not a win for me personally.
There's also the problem of "patch hygiene", which flows like this tend to do pretty badly with. It's routine in some projects (Linux is the flagship here, obviously) to demand clean commit messages and perfect bisectability. Pull requests in that world routinely need to be split/squashed/reworked during review and when moving across branches. You need tooling to do that.
So for instance, to work with the index with all your logging and instrumentation and debugging that you describe, you must be doing a lot of `git add -i` over the course of your workflow to partially stage things. Maybe you're also stashing the unstaged changes to make sure what you have in the index works on its own.
Well, I don't think that interactive add workflow actually seems any different fundamentally from copious use of `split`ing changes out of the working copy commit with jj. And I don't think stashing seems fundamentally different from rebasing the changes you split onto the branch you want to apply them to, leaving behind all the logging and instrumentation.
Yes, it seems like a lot of commit editing. But this is downstream of your workflow that requires some step to differentiate between what to check in and what not to, at a fine-grained level. You would have to split things up manually with jj, but you're already splitting things up manually with git, it's just a different workflow with a different command, but I'm not convinced it's any harder. Just different.
> Pull requests in that world routinely need to be split/squashed/reworked during review and when moving across branches. You need tooling to do that.
Yes, but jj seems to come with all that tooling! And I think its model of implementing all that on top of commits, rather than a combination of the concepts of unstaged+staged+stashed+commits, may actually be a better fundamental model.
But again, I still haven't used jj. My workflow is a lot like yours, at least oftentimes. I split stuff up ahead of time, I use the index a lot, I stash stuff a lot, I edit commits, I rebase interactively, I squash and split. But by my reading of jj, I think I can accomplish all that stuff, and just maybe in a better way.
There's a very serious and glaring error in doing that if you don't commit everything. When someone checks out your commit in a clean repo, his codebase is in a state that has never existed before, not even on your computer.
Some people (I think we've all done it at least once) split their work into multiple commits after a few days of hacking. Like "Adding data services for X", "UI features for X". This is a bad example, but you get the point.
Normally this doesn't cause a problem since all commits get pushed at the same time. But it can easily break things like git-bisect (which is just an amazing tool for finding astonishing bugs).
Re this point, how is it any different? "Staging" the files is essentially the same as splitting the commit, anyways — it's just that the newly-split contents go into a "staging area" vs a commit. Do you mean that the tooling to accomplish this is not good?
From only reading the README, my understanding is that you would just add the files to .gitignore and at the next auto commit they are removed again from the anonymous working copy commit. I agree it feels unintuitive that files are added and removed from the anonymous working copy commit, instead of not added in the first place. But since the anonymous working copy commit is largely invisible to you it shouldn’t make a difference.
Interesting nonetheless, I never liked staging in git but I never hit the case you mention so now I realize there's a need for a phase between a file system and a commit
I really hope this succeeds... VSC support would be a great start...
It is less apparent now but still, the repeated flex of “Google”, 20% project etc when no typical reader would assume them is classic corporate charlatanry.
Shame because I like the project otherwise
https://github.com/martinvonz/jj/issues/58#issuecomment-1247...
There's definitely nothing foundational blocking it though and it will happen one day if you'd like to give it a go in the meantime.
This is some kind of background process that automatically commits any changes you make.
You can use the CLI to check what it did and if you want to modify the auto commits.
> Commands snapshot the working copy before they start, then the update the repo, and then the working copy is updated (if the working-copy commit was modified). Almost all commands (even checkout!) operate on the commits in the repo, leaving the common functionality of snapshotting and updating of the working copy to centralized code.
weird choice, isnt explicit always better than implicit
I'm long accustomed to spelling it, in English, as Jujitsu. I've also seen Jiu-jitsu. "jutsu" is much less common, IME.
Is there such thing as canonical Romanisation of Nipponese? I can deal with a project being "wrong" better than not knowing which of us is wrong.
As a Brazilian Jiu Jitsu practitioner, I cringe when I see it spelled any other way, but also I have to recognize that I only feel that way because I have more exposure to that specific martial art/spelling.
It's jujutsu in both, but there are varying ways of representing the long vowel on the first "u" -- either omitting it (jujutsu), using a macron or circumflex (jūjutsu), or repeating the vowel (juujutsu).
Ju-jitsu and jiu-jitsu are not correct in any romanisation system that I know of, I'm not really sure how they came about. Probably historical accident.
Unless we're talking about the English word, which I'm not sure how to pronounce. If we're talking about the American word then it's really just like a lot of other murkin words: butcher it any way you like.
The first kanji in 呪術 means curse, so while "sorcery" could be a cool name for a software project in English, a Japanese speaker would probably have apprehensions about using 呪術.
Pronunciation is more like joo-juh-tsu. "Tsu" is its own syllable.
Too bad support/discussion happens entirely on Discord.
The reality is that the project is in a part of its life where active discussion and feedback with the userbase is pretty valuable. So, you just go where the users are, and Discord is a big place for that.
GitHub Issues and GitHub Discussions are also actively used by a lot of people, and you can just post things there. Every major committer watches those venues as well, AFAIK, I know my notifications are set up for it.
Over time, assuming things are wildly successful, you'd probably have a lot of different venues to discuss things. I would view Discord mostly in the context of a new project that wants feedback, in that regard.