Depends on how your architecture, github as a whole likely has volume of writes greater than a single server instance, however either components could be split in separate dbs ( seems to be their approach) or shard / partition the data
Bit late, but replying to this because my comment does have missing context. Most of the repos we have are large enough that downloading three individual files is faster than cloning the whole repo. Even with a shallow clone with a depth of 1. I've tried the combinations and even tried downloading a zip file of the repo. Still faster to download the three files. And I haven't even added parallel downloads because it's fine as is.
But given the issues, why continue to use the API? Because those files are related to the build pipeline logic and they change maybe once in 3-4 months. I can live with an error once in 3-4 months. Recently I was working on changing some of the logic of the pipeline and that's when I ran into it enough to say that there was something odd from Github's end.
Even in cases like Cassandra it's far from flawless, mostly due to the massive complexity. Failover works great, compaction works great, schema changes work great, repairs work... but what happens if two of those happen at once? Or all of them? There have been quite a few bugs over the years that involve corner cases when the various coordination and deferred-work systems interact.
3 comments
[ 4.4 ms ] story [ 13.0 ms ] threadBut given the issues, why continue to use the API? Because those files are related to the build pipeline logic and they change maybe once in 3-4 months. I can live with an error once in 3-4 months. Recently I was working on changing some of the logic of the pipeline and that's when I ran into it enough to say that there was something odd from Github's end.