I agree with this post. I’ve never found the GitHub wiki experience to be particularly ergonomic. I don’t have any issues with it, but it’s no more convenient than a simple /docs folder. And from there, it’s almost trivial to turn /docs into GitHub pages. Similar effort for a much better end product.
Wikis typically connote distributed, anonymous edits. This feature is partially covered by git already.
Fossil (https://fossil-scm.org/home/doc/trunk/www/index.wiki) solves this pretty nicely. You can have documentation as files or in a special wiki namespace and it's versioned both ways, and every repository clone gets everything. Even better than that, your in-tree documentation files are rendered and browseable in exactly the same way as the dedicated wiki namespace.
The linked URL to the home page there can even serve as an example: the "trunk" is a check-in name (https://fossil-scm.org/home/doc/trunk/www/checkin_names.wiki) that points to the newest check-in on the "trunk" branch. You can replace it with any other reference to get the old version; eg, version-2.20 would work to get the version 2.20 of the docs, 2015-03-14 would work to get the version from 14 March 2015, etc.
You absolutely can use "just a /docs" directory in Fossil. You can even point the web server to /doc/trunk/docs/index.md or whatever other file names you want. :-)
Fossil was written for Sqlite in the same way that git was written for Linux. It's really a shame that more projects don't use it. I think that a github competitor (with social features, PRs, CI, etc) with a fossil backend would be very popular.
In fossil's case, every instance includes those features, this makes the social platform "the web itself"
Having said that, there are central hosting projects, but where that has some value with git, git proper is distributed and does not need it but all the auxiliary stuff does. with fossil it provides almost no value. All the auxiliary stuff is built in so all it provides is a place to host. Which is fair but hosting is not hard with fossil.
With everything included in fossil I abuse it as a personal social platform(think discord) super easy to host and it gets me chat, forums, wiki, and file storage. None of them great, but it is so easy I don't really care. voice and video do require an additional service, so there is that. Now I just need to find some friends...
Yep, I use Fossil for all my side projects. Super easy to host, tiny, includes everything I need for a project, all in one file. Great piece of software.
I've been gradually migrating my Git repos to Fossil as I've been touching them. Been quite happy with it.
There are only a couple things that I miss:
- Grouping repos together into a combined project. I'd love to be able to have a single Fossil server/instance with a single set of users, wiki pages, tickets, etc. but multiple independent codebases. Closest I've gotten to that is to simply have multiple independent branches instead of a single trunk (example: https://fsl.yellowapple.us/avorion/home), and it's worked surprisingly well, but it's clear Fossil wasn't designed with this workflow in mind, so there are some rough edges with it (albeit minor and easy to work around).
- Compatibility with things that assume you're using Git. Being able to export to Git helps a lot here, but it's still extra steps. Ideal solution here would be for the Fossil server to be able to double as a Git forge and present repos accordingly, such that I can point things like CI/CD pipelines or Terragrunt module calls or whatever directly to the Fossil repo itself over the Git interface those things expect instead of having to setup a Git forge manually and somehow synchronize everything w.r.t. access controls.
- An equivalent to Git's submodules. This would IMO help address the “grouping repos together into a combined project” case as well.
There are login-group options which lets you have a single sign on with a group of repos, To unify the other bits it looks like it would be possible to change the main menu links to the central repo(in the web settings).
Never wanted to do what you want but it sounds possible
set up the fossil server on a directory of repos probably with --repolist to get a list
fossil server --repolist /var/fossil/
get all repos in the same login-group
change the menus in each repo to point to the core repo. Based on some quick tests I suspect a full http:// url is required.
- Forum /forum {@2 3 4 5 6}
+ Forum https://mydomain.org/core/forum {@2 3 4 5 6}
Interesting because I just added a wiki for one of my projects. I'm not using it for docs, since the project already has in-tree docs. I'm using it more as a public scratchpad of ideas / experiments to try that aren't well-defined enough (or known to be worth) opening as an issue yet.
In my experience, the docs for something like setting up a dev env are typically greatly improved by the second person who sets up the dev env, not the personal who originally wrote the docs.
In that case, when the docs are not associated with a code change, you want to make getting those improvements into the docs as frictionless as possible, otherwise the changes aren't going to get made.
Personally, I've found that making docs updates incredibly fast + easy to be far more valuable than anything you get from forcing doc changes through the full SDLC process. If someone has feedback on your docs changes they would have shared in a review, they can just update the docs instead.
> In my experience, the docs for something like setting up a dev env are typically greatly improved by the second person who sets up the dev env, not the personal who originally wrote the docs.
In my experience, this is also true of a lot of code as well. Your dev scripts should probably have much more relaxed standards than your service source or CI/CD. Ideally I could define merge requirements by directory without doing some weird shenanigans with the CODEOWNERS file and a bot.
It sounds like the whole argument here is that the commits in the docs repo aren't identical commits in the source code repo. But you can ameliorate that with something like tags or submodules.
The Github wiki is pretty bad. However had, it is easier to use than issues and coordinating them. Github issues require too much cross-communication and not everyone has the time to meta-coordinate many different issues in many different projects. A wiki lowers the entry-barrier too, so the idea of a wiki is, in principle, good.
If I were Github I would improve the wiki, a bit stylistically, to make it
more visually pleasing to use (but not much, those designers always go overboard when making changes in my experience), but much more importantly so, to make the wiki a more flexible addition, including API-wise, usage examples, documentation and so forth. People can, in principle, do so on their own, but also from experience, most people stop doing so after a while, and then the wiki decays into outdated information. That's bad too. Spawning more issues to manage the wiki also does not work well.
I really wanted the wiki to at minimum have directories so they would be easier to structure and allow even for major versions that are maintained in parallel.
The second paragraph neatly triggered my confirmation bias:
The initial version of this post opened with “You can use the wiki or a docs folder for your GitHub project, both are valid choices” but as I wrote more, I realised that there is a single reason to use a wiki, and many more reasons not to use the wiki. So many in fact, that I consider using the wiki on GitHub is an anti-pattern.
A very straightforward example of McCulloch's quote that "Writing is thinking."
Yes, and the rest of it is too; there's another outage today and my CI is blocked. I guess I can read the wikis while I wait, eh?? https://www.githubstatus.com/
I've never understood why people even _generate_ from the docs folder. If you've written it in markdown (which they nearly always do) then it's already rendered properly in Github. Or is it because they then publish those docs somewhere else?
> Or is it because they then publish those docs somewhere else?
Exactly, I use Sphinx to make a nice website from the markdown sources (although it doesn't have to be markdown). That includes generating a full API reference page from all the docstrings.
A lot of GitHub’s secondary features are like this. The Issues tab and Discussions tab are so similar, with slightly different feature sets. And users will happily use both for feature requests and bug reports, with varying degrees of quality, so then I just have yet another thing to stay on top of. I think Discussions were made to reduce noise in very busy repos, but I generally find something useful in the noise. It’s really just a way to ignore users. As a result, I always turn off Discussions and just let people file issues when they feel it’s appropriate.
I agree, but my only gripe is I hate the ceremony required for doc-only updates. It needs a review and CI. The review is a good thing in most cases (want your docs to be correct), but that often takes my team like two days (I'm realizing this is likely our fault now that I'm typing this). For CI, I've just added in a step to all our GitHub actions to skip Markdown only changes. Anyone have any better ideas?
If your process is bothering you, change your process - it's in your control. Write a new bullet point: "doc-only updates can skip review and CI". When you make a doc-only update, you click the "skip checks and merge anyway" button. Some people are so preoccupied with making sure things can only be done a certain way, they don't stop to think about whether it's the right way.
Of course someone will object to doc-only updates not getting review. That person is now on the hook for reviewing all doc updates after you merge them. It's the exact same thing you were doing before - they only objected when it got formalised.
The last paragraph says:
> At some point your docs will outgrow a single folder, and then all bets are off. You’ll want a separate repo with its own build process...
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
I also put everything in one repo nowadays, no matter the usage, the language etc. All is synced, and all is accessible by my LLM. There are a lot of tools to manage monorepos, and frankly most of the time you don't even need them.
It makes releasing software a lot more complicated.
Not a big deal if you're basically the only developer and handroll the process - but poly repositories make release and dependency management a lot more straightforward to wrangle
Most of the time your repo is so small that you won't run into the problems of a large repo and so you don't need those tools. Don't confuse that for monorepos have no problems when they get large.
You have the wrong measure. Large is number of parts. That is partially source files, partially projects/teams, and partially things that are conceptually not related. Likely other things as well.
It is unlikely GB/TB is ever a measure, though if your repo is that big and some people only need a subset of the repo it would be.
Worked at a place (as a tech writing manager) where the monorepo was several TB and company-issued laptops all had 500 GB hard drives. It was like a rite of passage that new writers or people with new laptops would inevitably not RTFM or learn about sparse checkout and try to clone the entire repo, which was not good.
We had managers and even non-engineers check-in documentation at Google: each page had an "Edit me" button that spawned an editor in a new tab with a CL(PR) ready. It worked very well.
On a large project you will have problems. You can maintain a monorepo anyway as many people do, and deal with the problems of a large monorepo. Or you can go to multirepo and deal with the issues of multirepo. Both have been done successfully, and both have significant problems that you need to work with.
Most people advocating a monorepo have never worked on a project large enough to see the issues with a monorepo and so are arguing for a monorepo without understanding the problems with them. For most people a monorepo is the correct answer because their project is small.
Seems like if you're small enough, a monorepo is the right way to go because it doesn't matter at that scale, and if you're big enough, you'll have the resources to throw at making monorepos scale.
Mono vs poly at scale needs resources. You have different compromises with each and so the resources go to different places. However there is no clear cut winner despite a few mono repo at scale advocates trying to claim otherwise - they are always completely ignoring the issues with a monorepo setup.
Multirepo at scale also needs resources, there's an enormous amount of work required for version bumping and synchronizing everything. People always completely ignore all those chores. Generally, you have either a massive amount of tech debt or you have one person doing nothing but running around doing all that work for everyone else (works great if all that work seems to magically appear for you). You can also be furiously working at automating all those chores, but that's the same level of effort you'd have to throw at scaling out a monorepo--just different.
Exactly because monorepos have least overhead when they’re small, monorepos generally win because you need to be small for a long while until you get big.
By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
So far, I’ve started a new job a handful of times by collapsing a premature polyrepo strategy: people were not experienced enough to merge two git repos without a common root.
I’ve only once went the other way, and it incurred so much overhead, it decreased developer productivity by some small but not insignificant percentage.
To be clear: I’m not a maximalist. All of my open-source work is exceedingly compartmentalised. My DNS library is separate from my external-dns webhook is separate from my fork of external-dns. They could all live in one repo. But FOSS encourages reusability, commercial software encourages clumping and vendoring.
> By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
Conversely to your experience, I have worked at a handful of places who have a monorepo that has been creaking under its own weight for years, but its structure as a monorepo now underpins the business, and so migration to a polyrepo simply never happens, and developers are now checking out a 50GB repo in its entirety periodically.
I'm on the other side of this problem, with a company that went multi repo for bad reasons (political, not technical) and I would give you serious money if you could solve my problems by just forcing me to check out 50gb every now and then...
Instead I deal with a 30+ repo clusterfuck (technically we have 60+ services, but I only have to run half...) that is held together by hopes and prayers, takes literal hours of actual effort to bring everything up to date on master, and has become a fractured hellscape where people are afraid to leave their tightly constrained silos of service combinations.
Long story short... I will take a bad monorepo over bad multirepo any day of the week.
I suspect that your hours of effort to get everything up to date would exist in a monorevel, too. It would exist in a different form, and so it would be harder to measure, but a large part of the work has to be done either way. It's just that certain parts of the work become very visible.
Yeah, compared with the maintenance effort of running around to 100 different repos, keeping everything in sync and the deps all updated, I'd always take the pain of 50GB checkouts.
It looks like these days a combination of git-scalar, git-lfs, and bazel will work for a small 50GB monorepo like that. The largest cost would be porting over the existing build system to bazel and then integrating it with CI. Once that is done, though, the monorepo will just scale, and all the constant ongoing cost of multirepo sync will be gone.
Google is an advocate of monorepo. However a lot of people are seem to be regurgitating what google wrote about them, but they are not Google scale and have no idea what the problems Google faces are.
Google also is very much in the yell loudly and ignore anyone who points out the problems of a monorepo.
Maybe it is just because I know a lot of Googlers/ex-Googlers, but almost all of the advocates for monorepos that I have spoken to are basing it from their experience at Google. I won't disagree that they tend to gloss over the extensive tooling they have to make it work, though...
Do you have only those two dirs at the top level? If so how are you finding it? I tend to have a docs dir at the top level, along with other build stuff
> I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
I have not taken full advantage yet, but every source sub-directory can have its own CLAUDE.md or AGENTS.md file, with instructions for a given directory, whenever something like Claude opens a file in a folder, if there's an appropriate agent doc in the housing folder, it should read / apply its contents when working. Not sure how this happens with Sub-Agents on that note.
I think if you want both, you might as well add a ./docs/ directory, and put your md documents in there however you want, this has the upside of letting you have docs with your code always, as well as letting you link to direct source code files.
I think there is a missing pro here on the wiki side: trivial edits are trivial. Even fixing a typo in the docs directory requires PR + approvals + CI. Effectively limiting your docs contributors to folks who are comfortable with a code editor, and git, is a decision
I'm no fan of GitHub add-ons and I agree with the premise here but...
I can think of one other possibility. It's easier to write in a wiki via the browser. I can open that on my phone and edit docs. I can open it in my browser and edit docs.
On desktop it's a tiny bit more to pull the repo and open it in your editor (and you might already be there) but that tiny bit can be enough to stop you from writing documentation. For me, writing documentation must be totally painless so that I'll actually do it.
Why am I not a fan of the add-ons like PR's, wiki's, discussions, projects, and issues? Because they each introduce vendor lock-in to varying degrees.
GitHub is pretty good about editing on the web. Markdown files can be edited straight from github.com. On desktop you can hit the period key to directly open the repo in vscode.dev. Technically on mobile you can change github.com to github.dev to do the same, though the editing experience is worse than directly editing on GitHub.
105 comments
[ 0.31 ms ] story [ 13.6 ms ] threadWikis typically connote distributed, anonymous edits. This feature is partially covered by git already.
The linked URL to the home page there can even serve as an example: the "trunk" is a check-in name (https://fossil-scm.org/home/doc/trunk/www/checkin_names.wiki) that points to the newest check-in on the "trunk" branch. You can replace it with any other reference to get the old version; eg, version-2.20 would work to get the version 2.20 of the docs, 2015-03-14 would work to get the version from 14 March 2015, etc.
Having said that, there are central hosting projects, but where that has some value with git, git proper is distributed and does not need it but all the auxiliary stuff does. with fossil it provides almost no value. All the auxiliary stuff is built in so all it provides is a place to host. Which is fair but hosting is not hard with fossil.
https://chiselapp.com/
With everything included in fossil I abuse it as a personal social platform(think discord) super easy to host and it gets me chat, forums, wiki, and file storage. None of them great, but it is so easy I don't really care. voice and video do require an additional service, so there is that. Now I just need to find some friends...
There are only a couple things that I miss:
- Grouping repos together into a combined project. I'd love to be able to have a single Fossil server/instance with a single set of users, wiki pages, tickets, etc. but multiple independent codebases. Closest I've gotten to that is to simply have multiple independent branches instead of a single trunk (example: https://fsl.yellowapple.us/avorion/home), and it's worked surprisingly well, but it's clear Fossil wasn't designed with this workflow in mind, so there are some rough edges with it (albeit minor and easy to work around).
- Compatibility with things that assume you're using Git. Being able to export to Git helps a lot here, but it's still extra steps. Ideal solution here would be for the Fossil server to be able to double as a Git forge and present repos accordingly, such that I can point things like CI/CD pipelines or Terragrunt module calls or whatever directly to the Fossil repo itself over the Git interface those things expect instead of having to setup a Git forge manually and somehow synchronize everything w.r.t. access controls.
- An equivalent to Git's submodules. This would IMO help address the “grouping repos together into a combined project” case as well.
Never wanted to do what you want but it sounds possible
set up the fossil server on a directory of repos probably with --repolist to get a list
get all repos in the same login-groupchange the menus in each repo to point to the core repo. Based on some quick tests I suspect a full http:// url is required.
In that case, when the docs are not associated with a code change, you want to make getting those improvements into the docs as frictionless as possible, otherwise the changes aren't going to get made.
Personally, I've found that making docs updates incredibly fast + easy to be far more valuable than anything you get from forcing doc changes through the full SDLC process. If someone has feedback on your docs changes they would have shared in a review, they can just update the docs instead.
You can even do live edits on the web if you don't want to use a command line.
In my experience, this is also true of a lot of code as well. Your dev scripts should probably have much more relaxed standards than your service source or CI/CD. Ideally I could define merge requirements by directory without doing some weird shenanigans with the CODEOWNERS file and a bot.
however I think maybe this has changed? I don't see it in https://github.com/robots.txt now
and on GitLab: https://docs.gitlab.com/user/project/wiki/
It sounds like the whole argument here is that the commits in the docs repo aren't identical commits in the source code repo. But you can ameliorate that with something like tags or submodules.
If I were Github I would improve the wiki, a bit stylistically, to make it more visually pleasing to use (but not much, those designers always go overboard when making changes in my experience), but much more importantly so, to make the wiki a more flexible addition, including API-wise, usage examples, documentation and so forth. People can, in principle, do so on their own, but also from experience, most people stop doing so after a while, and then the wiki decays into outdated information. That's bad too. Spawning more issues to manage the wiki also does not work well.
The initial version of this post opened with “You can use the wiki or a docs folder for your GitHub project, both are valid choices” but as I wrote more, I realised that there is a single reason to use a wiki, and many more reasons not to use the wiki. So many in fact, that I consider using the wiki on GitHub is an anti-pattern.
A very straightforward example of McCulloch's quote that "Writing is thinking."
An example:
- source: https://github.com/eslint/eslint/blob/main/docs/src/rules/no...
- rendered: https://eslint.org/docs/latest/rules/no-control-regex
This is a very simple example, you can see how the code blocks are highlighted better and slightly interactive.
Exactly, I use Sphinx to make a nice website from the markdown sources (although it doesn't have to be markdown). That includes generating a full API reference page from all the docstrings.
Of course someone will object to doc-only updates not getting review. That person is now on the hook for reviewing all doc updates after you merge them. It's the exact same thing you were doing before - they only objected when it got formalised.
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
Not a big deal if you're basically the only developer and handroll the process - but poly repositories make release and dependency management a lot more straightforward to wrangle
It is unlikely GB/TB is ever a measure, though if your repo is that big and some people only need a subset of the repo it would be.
Managers and others won't touch the repo. (Sometimes it's better the don't...)
Most people advocating a monorepo have never worked on a project large enough to see the issues with a monorepo and so are arguing for a monorepo without understanding the problems with them. For most people a monorepo is the correct answer because their project is small.
By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
So far, I’ve started a new job a handful of times by collapsing a premature polyrepo strategy: people were not experienced enough to merge two git repos without a common root.
I’ve only once went the other way, and it incurred so much overhead, it decreased developer productivity by some small but not insignificant percentage.
To be clear: I’m not a maximalist. All of my open-source work is exceedingly compartmentalised. My DNS library is separate from my external-dns webhook is separate from my fork of external-dns. They could all live in one repo. But FOSS encourages reusability, commercial software encourages clumping and vendoring.
Conversely to your experience, I have worked at a handful of places who have a monorepo that has been creaking under its own weight for years, but its structure as a monorepo now underpins the business, and so migration to a polyrepo simply never happens, and developers are now checking out a 50GB repo in its entirety periodically.
Instead I deal with a 30+ repo clusterfuck (technically we have 60+ services, but I only have to run half...) that is held together by hopes and prayers, takes literal hours of actual effort to bring everything up to date on master, and has become a fractured hellscape where people are afraid to leave their tightly constrained silos of service combinations.
Long story short... I will take a bad monorepo over bad multirepo any day of the week.
Really? I feel like most of the stuff I have read advocating monorepos are from people at Google, which is a HUGE monorepo.
Google also is very much in the yell loudly and ignore anyone who points out the problems of a monorepo.
I always initialize my projects with a src, and docs, directory, for exactly this reason.
My reasoning is that I shouldn't have to go hunting for the docs for the code, or vice versa.
I have not taken full advantage yet, but every source sub-directory can have its own CLAUDE.md or AGENTS.md file, with instructions for a given directory, whenever something like Claude opens a file in a folder, if there's an appropriate agent doc in the housing folder, it should read / apply its contents when working. Not sure how this happens with Sub-Agents on that note.
I think if you want both, you might as well add a ./docs/ directory, and put your md documents in there however you want, this has the upside of letting you have docs with your code always, as well as letting you link to direct source code files.
I can think of one other possibility. It's easier to write in a wiki via the browser. I can open that on my phone and edit docs. I can open it in my browser and edit docs.
On desktop it's a tiny bit more to pull the repo and open it in your editor (and you might already be there) but that tiny bit can be enough to stop you from writing documentation. For me, writing documentation must be totally painless so that I'll actually do it.
Why am I not a fan of the add-ons like PR's, wiki's, discussions, projects, and issues? Because they each introduce vendor lock-in to varying degrees.
[0]: https://docs.github.com/en/communities/documenting-your-proj...
in repo (can be wiki format):
- dev docs
- user manual
maybe different repo:
- community usage wiki