73 comments

[ 175 ms ] story [ 390 ms ] thread
I am intrigued by the level of sophistication this Google's private git offers. But I'm not probably in their intended audience as I'm not interested in providing my full personal and credit card information just to use git... I am probably just too ingrained in GitHub and AWS to even consider other options. But I'm curious, still. Is it ... good? I hope somebody is going to share their experience with me.
Regardless of Git you should check out Google Cloud. Massive improvement in developer experience in comparison to AWS.
It's just git so there's not much sophistication, nor is one service "better" than another in that regard.

Github/Gitlab are still far more useful as they are more than just repos. The issues, PRs, wikis, and general management features are what most projects need. Azure DevOps is probably the best competitor from the clouds.

Sure git is just git. I was referring more to the other features this PR piece is promising: Fast Code Search, Debug in production, Detailed audit logs.

I am curious how they relate to say Github or other services.

Also as a sidenote, the Quickstart link for Debug in production and Detailed audit logs is the same. Is it so for a purpose?

Cloud Source Repositories PM here.

>Also as a sidenote, the Quickstart link for Debug in production and Detailed audit logs is the same. Is it so for a purpose?

Thanks for pointing this out. It's not intentional. I'll make sure it gets fixed.

Cloud Repos and Cloud Build are very lacking. I wouldn't recommend for anything other than side projects.

Github, Gitlab, Azure DevOps are also either free or very cheap and have 100x more features and functionality.

Google Cloud is worth checking out. CSR is just one of their offerings in the GCP ecosystem.
Well bitbucket is free regardless the number of repo so why would I choose Google repos?
C'mon, man, click the link before arguing with the premise. Had you done so, you'd have seen in big bold type "Unlimited private Git repositories for free"
Title is wrong: they have unlimited private repositories for free, but for a maximum of 5 users per project.
That is also wrong. Not five users per repo but five active users per account, across all repos.
Is Cloud Source Repositories using the same web app as the internal Cider tool?
(comment deleted)
And so the count down to this services death begins. /Sarcasm
For some reason, I don't trust google at all for anything...
It wasn't all that long ago that Google shut down Google Code. They said: https://opensource.googleblog.com/2015/03/farewell-to-google...

> After profiling non-abusive activity on Google Code, it has become clear to us that the service simply isn’t needed anymore.

And now they've created a new product which does pretty much the same thing... Wonder how long this one will last.

AFAIK, GSR is really intended to support cloud deployment. So not a replacement for Google code.

I think the assumption is that most folks will use github / gitlab , etc. as their primary, and mirror to GSR for deployment.

I disagree that CSR serves the same purpose as code.google.com, and is likely to meet a similar end.

code.google.com hosted public repos, issues, wiki, downloads, etc., and was generally just ripe for abuse. Google created it largely because Sourceforge was the only thing that existed at the time, and it wasn't doing a great job. Today there's GitHub, Bitbucket, Gitlab, and more.

CSR is primarily for hosting private code, shared among teams, and adding integrations with other Cloud services, in the Cloud Console. It also adds code search, which is pretty great. Hosting private repos, it's much less easily abused (it's not impossible). There are plenty of private repo hosts, and this one aims to be the one best integrated with GCP. It has, I think, a much better business reason to continue existing. It might not make money (I legitimately do not know) but it helps drive GCP adoption/usage, unlike code.google.com which didn't.

Disclosure: I worked on code.google.com, and helped turn it down, and helped create CSR from its ashes. I don't work on either anymore.

> It might not make money (I legitimately do not know) but it helps drive GCP adoption/usage, unlike code.google.com which didn't.

Understandable considering that AWS has codecommit and MSFT owns github now.

Proper code search could be a pretty killer feature on its own. Half the time I need to find something on github I just clone the repo and search from my own computer.
Heh, yes. Github's search is case insensitive and ignores sigils, braces, etc.
I've been waiting years for GitHub to put some real effort into code search and cross-linking. I had given up on it, but I suspect this will be a high priority for Microsoft.
Hi, Jason! Thank you for the awesome work with Cloud Build :)
Well, there was at least one legitimate reason to shut it down. It was used to host malware/phishing disguised as official google software under the official umbrella of google.com. What could go wrong with downloading chromebeta from google itself?

That being said, the tool to migrate to GitHub worked well. It was an easy migration.

Well that was better than SourceForge. SourceForge actively added crapware to open source code.
My biggest complaint with Google Cloud's UX, which applies to their new Cloud Source Repositories opt-in UI: They shard out so many of their user experiences into different websites and domains.

Under the umbrella of Google Cloud you've got Google Cloud Console, Firebase, Stackdriver, Source Repositories, Dialog Flow, probably a couple others I'm forgetting. Almost all of these have entries on the Google Cloud Console which open a new tab and link you to a new website.

It makes keeping track of where things actually are extremely difficult. For example: Firebase functions on the Firebase UI are also Google Cloud Console functions. The old Google Cloud Datastore product, which lived on the Console, is now Cloud Firestore on the Firebase console. Stackdriver logging used to be on the Stackdriver UI, now its on the Cloud Console, but there are still references to Stackdriver all over that product, and they continue to invest to Stackdriver (Istio service maps).

AWS does this a little bit, in four or five of their hundreds of products. Lightsail is one example. Azure does this a little bit as well (DevOps is a separate page even though its essentially an Azure product).

GCP is the worst offender. I want to see a better unifying vision for what being a Google Cloud customer actually means, instead of all these confusing and isolated experiences.

> I want to see a better unifying vision for what being a Google Cloud customer actually means

The thing you've got to understand is that none of these products are "Google Cloud" products. They're Google services (or, even moreso, services of random third parties which Google has acquired) that you happen to be able to hook into from your GCP project and access through the GCP CLI.

The closest comparison I can think of is Heroku's "Elements", which are third-party offerings (usually of their own services, with their own service dashboards and everything!) that happen to be subscribe-to-able and high-level-configurable through Heroku's dashboard/CLI.

Except, in this case, many of the third party "elements" that are subscribe-to-able and high-level-configurable in GCP, happen to be created by other departments/sub-companies of Google.

Still nothing to do with GCP, though, any more than a random Heroku Element has to do with Heroku.

I can tell you that your view does not match that of the engineers working on these products. If something is namespaced under a project and you use the cloud console or CLI to manage it, it is very much a GCP product.

(I am one such engineer, and have worked on the CLI, cloud build, and container services)

You’re speaking of design intent, I think. I’m speaking of point-in-time experience from a user perspective. It’s clear what GCP wants to transition toward—but:

1. that’s not where many GCP-aegis’ed services are right now, and

2. newly-absorbed Google properties will always start out looking like this, so there’ll never be a time when GCP won’t look at least somewhat like this (unless the post-acquisition integration process is changed drastically.)

It is true that a lot of GCP services were once external, like Firebase and Stackdriver, and it is true that there are numerous different domains and UIs that you manage GCP products under, but I believe the future direction is definitely towards unifying things, and the split right now is probably mostly technical. This isn't an official statement by any means, though.
That's not quite right. Google Cloud is basically everything, including the Google Cloud Platform (cloud computing), G-Suite, Maps, Chrome, Android, etc.

Firebase and Stackdriver are underneath GCP.

One day some part of not all will shutdown and disappear and where is your sources again.
Interesting that they're going this way.

We used Google's source repository functionality for a while because there is no real choice if you want to use Google Cloud Builder, which we were testing out for cloud-based Docker builds. GCB was pretty disappointing, and we moved off it.

We also found the source repository service annoying. GCB can't pull code from Github, so you first have to set up a Google source repo and turn on Github sync; the source repo is, in other words, only there to be a mirror. Unfortunately, this introduces problems. You get latency -- your Google repo is always behind. Worse, we frequently experienced a situation where the sync was broken, either due to Github being flaky, or the syncing itself being flaky. In such cases, it was impossible to tell what the status of the sync was; all you could do was go into the UI and see that it was not up to date; no way to force a sync. So this setup became a liability. At the time (not sure if this is still the case), there was also no API to set up new mirrors, it was all manual. The whole UI around GCB and source repos seemed to have been created in a rush, very bare bones. A lot of Google serviced start out like this.

I suppose this is a strategic play. For developers, you'd get a better UX if Google simply had tighter integration with Github and Gitlab, but Google probably doesn't want that. However, the low quality of their offerings so far (especially, cough, StackDriver) mean that this stuff will have an uphill battle in proving itself to developers, most of whom already have an allegiance to better tools like Github. If you're already on Github, why would you use this stuff?

> Worse, we frequently experienced a situation where the sync was broken, either due to Github being flaky, or the syncing itself being flaky. In such cases, it was impossible to tell what the status of the sync was; all you could do was go into the UI and see that it was not up to date; no way to force a sync.

Having worked for some of the big 5 American tech conglomerates, I would say this is the exact reason why they did not support it initially nor fully. The bigger the company, the more risk their is running/depending on someone else's software SLA/scaling/dependencies/security/privacy/laws... You just had a bad experience with Google cloud but you don't really know if it was google's fault or Githubs.

Plus, they have so many software developers, they can build their own version that conforms to their requirements themselves for practically the same costs.

Cloud Source Repositories PM here. I appreciate your feedback and will share it with the team.

>it was impossible to tell what the status of the sync was; all you could do was go into the UI and see that it was not up to date; no way to force a sync.

We've heard this was a problem for users so we're adding the ability to force a sync and to view the current sync status. These features will be live in the next few weeks.

I would strongly advise you to make the mirroring system completely hidden. It's weird to me that the UI even exposes this.

First, have people register their origin repos. In fact, don't let people register individual repos; just their accounts or organizations. So instead of "Source Repositories", it's just a list of repos that GCP has access to.

Then, behind the scenes, mirror everything you have access to. All projects that you can read. Keep it internally. If you offer a UI to browse code and so on, pretend the code is remote. All references need to show https://github.com/someorg/myrepo.git as the remote, for example. Pretend your mirror doesn't exist.

Only add Github hooks for projects where real-time syncing is needed, i.e. build triggers.

Then, use Github hooks like you usually do, but have it fall back to pull-based polling when hooks aren't working (something I bet you guys can detect simply by noticing that hook activity for all projects is below average!). Anything that needs a build trigger could also have a manual button that checks the upstream.

Consider the developer's perspective. I already have my code on GH. I don't want to also store it in GCP. Why should I? It's right there on GH for any API consumer to read. Keeping it on GCP is just more work.

Being able to build with GCB without involving the indirection of a mirror would be even better.

> Consider the developer's perspective. I already have my code on GH.

What if you don't? The fact that Google needs to acknowledge that CSR isn't where existing projects are primarily hosted (and thus needs to accommodate projects primarily hosted elsewhere) doesn't mean they shouldn't try to be a primary host.

That's fine. I'm only talking about that particular requirement that you use SR/CSR in order to use services like GCB.

Obviously you have to point GCB at something. Today, you can only point GCB at SR/CSR. If you go to GCB and try to add a trigger, and select Github, it will go and set up a mirror on SR/CSR for you.

Hi there, looking to move away from another hosting provider. I think what GSR really needs for that to happen is the PR feature, along with integration with Cloud Build at point of PR.
Will there be pull request support anytime soon?
I was surprised to find this is missing. I do hope they add this.
Has anyone noticed how they use what appears to be the GitHub logo on this page...? Surely that isn't okay.

https://cloud.google.com/images/products/source-repositories...

It says right underneath it "Mirror code from GitHub" There's nothing wrong with using imagery to draw your attention to related features.
I'd tend to agree if it weren't listed right next to the feature "Unlimited private Git repositories for free", which is a feature GitHub charges for, but seeing as this is a somewhat competitive product, it feels a little suspect.
I am a GCP user and we use Gitlab. I understand that CSR provides better integration with other services in GCP. However, I am achieving most of those using gitlab-ci.

Can someone explain (not sure, if I am missing anything) why should I move to CSR? If not, I will better stay with gitlab. TIA.

Code search might be the killer feature here.

I used to work at google many eons ago and I spend 90% of my day wading through billions LOCs with their internal code search tool.

I believe it's the same one used by chromium: https://cs.chromium.org/

Without all the nice bells and whistles github provides for collaboration like wikis and issue tracking I don't think google will be successful at convincing people to maintain their code here. I don't find the outliner useful. That's a common IDE feature and a repo is not an IDE.

I've always viewed source control as a productivity tool more than infrastructure. With that mindset, it will take a lot more to convince people to do more than mirror their code here. Maybe that's enough, but mirroring has its headaches...

(comment deleted)
tl;dr: this isn't replacing github, it's solving a more specific problem

I think everyone here is missing the point. The purpose here seems to be similar to github actions in providing workflows around repository events (though also including some basic ability to view the code as well). Having a clean way to cut a release that updates app servers on app engine and serverless functions in tandem is needed right now.

I don't think this is intended to be used by anyone that doesn't have their platform on GCP.

For highly sensitive codebases within an organization, they may not want to host it on microsoft's servers as well—but this seems secondary to the workflow functionality.

It's possible I'm incorrect though and I do think the messaging here should be improved. Chiefly that it shouldn't lead with "Unlimited private Git repositories for free"

>> Having a clean way to cut a release that updates app servers on app engine and serverless functions in tandem is needed right now.

This is the entire field of CI/CD with dozens of providers, including Github and Gitlab too.

I'm not claiming they're the only ones (I called out github's offering explicitly). I'm just explaining why this exists because a lot of people in this thread seem to think it's a github replacement.
I was responding to the "needed right now" part in your first comment. Unless I'm misunderstanding, do you think it's not covered by the other CI/CD providers?
I don't mean to imply that other products are inferior. A problem can simultaneously exist and have solutions.
Ah, so it's Google's Visual Studio Online/VSTS/Azure DevOps[1]

[1] Microsoft name shuffling is so much fun...

Xoogler here.

Internally as Google, you have a substantially power powerful version of Codesearch, which is semantic, i.e. it relies on actual binary artifacts produced by builds to index code. All of the code base uses Blaze ( open sourced as Bazel) to express the rules.

The version of code search here is cute, but it's still not a full blown structured semantic code search, which I really miss. Searching for and browsing code on Github feels underwhelming in comparison, because you wan't really find or explore code the way you think of it.

You can see a demo of semantic Codesearch tech externalized here at:

https://blog.bazel.build/2017/12/14/introducing-bazel-code-s... http://cs.chromium.org

For example if you click on a header or a symbol, you can get proper cross references to a symbol.

https://cs.chromium.org/chromium/src/pdf/document_loader_imp...

I really wish they add structured code search to Cloud Repos, even if it comes with some heavy restrictions such you must use Bazel as a build system.

It would be a killer feature, and to my knowledge there is no equivalent to such a product. IDEs offer some similar overlapping functionality, though other code search engines still operate in terms of strings rather than actually real binary symbols ( I think Sourcegraph is trying here but that still does not seem to use some kind of accurate and structured information like you have in the code, it still seems not use build rules).

How do you deal with things like system headers then? Force everyone to check in their OS (bye BSDs)? Assume everyone runs the same OS?

How do you deal with compiler provided internals, where someone might be using clang which has behavior X, but gcc / icc / msvc doesn't?

These problems are more prominent in C/C++, but are everywhere... Not easy...

Google can probably check in everything for themselves... Probably not so easy for other companies? Maybe? Can Google solve this without sinking too much resources into it?

My understanding of this is the Google approach is to have all builds in the cloud using Forge i.e. infrastructure that lets you cloud builds.

This means you can do hermetic builds, and thousands of other engineers can reuse the build artifacts you produce.

And in the particular context of tools such as code searching, your build artifacts update the code searching index pretty quickly post committing a change.

My point wasn't how do you build these things. It's how you index them.
> I think Sourcegraph is trying here but that still does not seem to use some kind of accurate and structured information like you have in the code, it still seems not use build rules

Sourcegraph does use language servers (same as your editor would use) to get precise definitions and references.

For example ("find references" on some open-source code):

Go: https://sourcegraph.com/github.com/theupdateframework/notary...

TypeScript: https://sourcegraph.com/github.com/ReactiveX/rxjs/-/blob/src...

It understands many build tools, but not all. Go, JavaScript, TypeScript, Java, Python, and PHP support is all pretty solid. If you use the most common build tools for your language, it usually works well. It degrades un-gracefully if you use custom build tools or scripts, unfortunately, but we're working on making it support more build tools and customization. Sourcegraph development is open source; if you notice any problems, let us know in the issue tracker at https://github.com/sourcegraph/sourcegraph.

Thank you for the reply, appreciate you reading the comments so thoroughly.

My top piece of feedback is considering some form of a strict mode, where your vision is not best effort support support for some legacy tool for a language, but fabulously amazing knock your socks off good support for users who take the time to invest in making their build friendly with your search index.

It might seem tempting to go after the longest common denominator audience here, but personally I think it's far more valuable to show users how code search can be amazing if they do their part to invest in it, vs. code search being "kind of sort of useful" if you have a fallback.

And maybe my Xoogler experiences are showing here, though Bazel / Kythe are in my opinion the perfect vehicles to realize that vision.

Very good points, thanks! We will be exposing a lot more configurability and extensibility in Sourcegraph in the next release (3.0-preview, and 3.0 more generally). I will add Kythe support to our roadmap and will think about how we can offer a strict mode soon. (We will need to offer that when Sourcegraph supports "write" functionality like automated refactoring, but it would be great to have it sooner.)
It's honestly one of the greatest advantages of working in google3. GitHub search is honestly pitiful next to it.
I find the pricing of this service to be incomprehensible, as well as contradictory to “Unlimited private Git repositories for free”. Can anyone explain it?
> I find the pricing of this service to be incomprehensible, as well as contradictory to “Unlimited private Git repositories for free”.

You get unlimited private repositories, 5 project users, 50 GB of storage, and 50 GB egress per monthly for free.

Monthly overage charges beyond the free quota are $1 per project-user, $0.10 per GB of storage, and $0.10 per GB of egress.

So for 10 project-users, 100 GB of storage, and 200 GB of egress in a month, you pay $5 for users, $5 for storage, and $10 for egress, for a total of $20.

I still don’t get it. What is a “project” and how does it differ from a “repo”? If they are the same thing, isn’t every project/repo going to cost at least a $1?
Cloud Source Repositories PM here.

A project as described here refers to a "Cloud Project" which is the basis for creating, enabling, and using all GCP services including managing APIs, enabling billing, grouping resources, etc. Within a project, one could have tens or hundreds of repositories. With 6 users on a single project, you would pay $1/month total (because the first 5 users are free) and all those users would have access to every repository in that project.

I appreciate your feedback that you find the pricing model and project concept difficult to understand. I'll bring this back to the team and we'll look at different ways to improve this.

Thank you - this explanation is much more clear and I understand now. I’m generally familiar with AWS but haven’t used GCP, so I didn’t have the “Cloud Project” concept.
Projects are the fundamental organization unit of GCP. Every resource you create in GCP lives in a project, and users are granted access to projects.

https://cloud.google.com/resource-manager/docs/creating-mana...

"Project-user" here means that access is counted per project. So if you (x@) use GCR for projects A and B, meaning you check in/out code as x@A and x@B, that's two "project-users".

Disclaimer: I work for GCP.