I'm the tech lead and primary software engineer on the Go runtime. If you have any questions, please don't hesitate to ask!
I'm super thrilled to announce that Go 1.11 is now available on App Engine! We now support...
If you're a current Go-on-App Engine customer, you should check out our migration guide at https://cloud.google.com/appengine/docs/standard/go111/go-di... to learn how to migrate from the Go 1.9 runtime to the new Go 1.11 runtime. For the time being, you can still use the legacy App Engine APIs with the Go 1.11 runtime, but you should start migrating to the Google Cloud client libraries.
Yes! Thanks for releasing this and making my life easier :)
I have an old app deployed to App Engine. The app itself is rock solid and chugging along fine (I touch it once a year or so), but I dreaded having to deploy it due to the lack of vendoring support and other... peculiarities of the Go runtime. Glad to see this is no longer an issue!
I take it the datastore change is just an api change? (There isn't any data migration needed, is there?)
It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?
> I take it the datastore change is just an api change? (There isn't any data migration needed, is there?)
Yup!
> It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?
I'm not sure about the performance impact. But I can tell you that we're working with the Cloud Memorystore team to have a better memcache story for the App Engine second generation runtimes
Both google.golang.org/appengine/datastore and cloud.google.com/go/datastore connect to the same data? Is there a list of incompatibilities somewhere? I remember seing some serialization differences between these two datastore libraries.
Hi (App Engine PM here).
Yes both libraries connect to the same database (Cloud Datastore). cloud.google.com/go/datastore is preferred as it uses the Cloud Datastore API instead of the App Engine specific API, so your code will be portable.
You are right that the devil is in the details, and there might be some slight serialization differences. I cannot find an exact list.
> the cloud datastore package will default to writing your nested structs as entity values, while the appengine datastore packages will only write your nested structs as flattened sets of attributes.
For cloudsql in particular, you can connect via the `/cloudsql` Unix socket. But anything that needs raw socket access can now be used from App Engine.
when will appengine support setting environment variables with gcloud command or inside the gcloud console instead of in the app.yaml, i.e. so that each environment can have their own variables without copying app.yaml's?!
(App Engine PM here) At this time, app.yaml is the only way to set env vars. If you want to separate environments, you could use different .yaml files, I personally use app.staging.yaml and app.prod.yaml
You definitely don't want to check secrets into the repo for the same reason you don't set them as environment variables: It's not secure. The solution is to use Cloud Key Management Service. More info here: https://cloud.google.com/kms/ We use that to either store secrets directly, or to decrypt encrypted entities in a DB (e.g. Datastore).
Neat, looks like they're using a solution similar to what we're doing, except instead of using Cloud KMS to encrypt information stored in Google Cloud Storage, we're putting our information in Datastore.
The reason being that these secrets are absolutely essential for many tasks that our registry needs to perform, and the past six years of experience have shown us that Datastore has better availability than GCS. We haven't seen Datastore ever go down unless all of Cloud is down too, whereas we have seen outages isolated to just GCS. Datastore also has lower latency (since it's a DB, not bulk file storage).
One caveat is that the maximum size of a single entity in Datastore is 1 MB -- if you're encrypting stuff larger than that then you'll need to shard (ugly) or just use GCS. Since none of our secrets are anything close to that large, it works just fine.
> I'm the tech lead and primary software engineer on the Go runtime.
Hey thanks a lot for your efforts! Your direct communication about the Go runtime progress has been a breath of fresh air compared to the years preceding you. I wonder though, how many developers are there on the App Engine Standard Go team?
You cant run a GRPC server though still right? Thats all I really want to be able to do especially now that the app engine endpoints libraries are no longer supported on python
Inbound GRPC will be limited to HTTP1.1; that's a limitation of the platform rather than the language runtime. We're working on HTTP2 support, but I can't promise any timelines.
By inbound GRPC you mean when running an app engine instance as a client right? So running a GRPC server is completely out of question still I would presume?
It's always been my impression that Go on GAE is the most lightweight compared to Java/Python, as it not "memory hungry" like Java nor "slow" like Python. Am I correct?
I actually looked at this before making my other comment. You’re absolutely right about the salt. One of the go benchmarks is basically pure cgo using vector intrinsics. It naturally blows idiomatic JS out of the water. So you might get those kind of results if that’s the kind of code you’re going to write, but it’s far from what I consider idiomatic. Another language’s proponent would be fair to point out that most languages have C FFIs.
That Go pi-digits program is 50% faster than a Java program that also uses the GMP library, just like the Go program!
That Go pi-digits program is only 4% faster than a PHP program that uses the GMP library.
It's not enough to "point out that most languages have C FFIs" when for this arbitrary precision arithmetic task GMP is allowed and others step-up and contribute those programs.
I see. I had only looked at the source code of the Nodejs competitor. To be fair the point still stands that you need to read the source to know what is really being measured.
No, there's no reason to take the benchmarks game with even a pinch of salt!
The benchmarks game website shows what it says it shows --
"Which programs are faster? Will your toy benchmark program be faster if you write it in a different programming language? It depends how you write it!"
Any plans to support the cloud endpoints with the standard env?
Wondering if the new runtime would be able to be embed that. That would be only reason for me not to switch from flex env.
Cloud Endpoints currently runs on GAE Standard Gen 1 and GAE Flex. We're investigating a solution for GAE Standard Gen 2 (which go 1.11 is, along with python 3.7, node 8, etc.) and GCF.
As for EAP: unfortunately not. Personally, I'd like to see a managed version as opposed to the current framework or sidecar approach, as it'll be easier to implement across products, but that means it'll likely take a little longer.
How long will the legacy App Engine APIs continue to work? Will they be removed before the runtime is GA?
These new runtimes seem amazing (and fix a lot of problems), but we also lose a ton of functionality that made app engine so desirable (easy users auth, images api, built in email sending, search, cron, and others).
It kind of feels like we've thrown the baby out with the bathwater.
(App Engine PM here) The App Engine APIs will be present in the Go 1.11 runtime when it goes GA and until it is turned down.
We do not know exactly which future version of the runtime will stop supporting them.
We are working on Google Cloud standalone products to replace most missing features (e.g. Cloud Scheduler for cron jobs). Not all of them are ready yet.
App Engine APIs were great when there was no alternative (when GCP did not exist, or when App Engine could not use arbitrary packages), but at the same time, contributed to the "lock-in" of App Engine, which was one of the main criticism. We believe it is in the long term benefit of our users to use standalone Google Cloud or third party services, instead of replying on APIs only accessible in App Engine.
How do you feel about the distinction between Google and Go? It sometimes feels like they're joined at the hip, something which historically had made people queasy (Java and Oracle, .NET and Microsoft, etc). Though Go is a project which came from Google, I personally feel that it better serves the interests of the Go community to treat them separately. Google receives special treatment from Go - the announcement that AWS Lambda would support Go did not receive similar fanfare on golang.org.
Side note: there is some activity on the Go blog which is more mutual cooperation and less Google hivemind:
I just tried to use go111, but I get the following message:
>[7] Access Not Configured. Cloud Build has not been used in project <project> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbuild.go... then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
So go111 cannot be used without Cloud Build, for which I have to activate billing?
Correct, Go's runtime is compiled into the program's binary.
However, there's still an environment needed to run that binary. For Go on App Engine, this includes the sandbox (backed by gVisor) and the operating system (Ubuntu).
We also tend to call everything in the toolchain the "runtime" - this includes the CLI for uploading and staging your app, and the builder used to compile your program. Not the strictest definition :)
Yes, with App Engine, you upload source code, and it's compiled for you remotely. For this newer runtime, it happens inside a Cloud Build invocation.
For App Engine flexible, you can skip that build step and provide your own container, but that isn't possible for App Engine standard. (If you're interested in something like that, see https://g.co/serverlesscontainers)
Am I understanding this correctly, App Engine has made it so "go run goapp.go" on my local environment is basically accomplished with "runtime: goapp" in the app.yaml? No other changes necessary in the code?
This actually seemed to be live a week or two ago, I managed to run Miniflux [1] on App Engine with no changes though to get it working properly it would need it's scheduling changing.
Ah I hadn't seen that, I was quite surprised when I used goapp111 and it deployed successfully. It's great to be able to take a normal Go application and run it on Google Cloud. Thanks!
Does this new classic+ environment now supersede the Flex environment?
I’m curious because now it’s seems like there are 3 solutions. Although I really like the simplicity of this solution and the modernizing from the constraints of true classic mode.
Eh, not exactly supersede, but the runtimes are definitely slowly converging. It's a goal of mine to make them easy to switch between -- sometimes you'll want flex because you need a VM, most times you wont. But your code should be easily portable between them. With this release, they are nearly identical already.
So two of the solutions are flex and go1.11 on standard. Is the third solution the older runtime versions (eg 1.9 and earlier)? If so, then there's really just two solutions, since 1.11 replaces 1.9 on standard.
If you can run within the standard environment, I'd suggest you do (but you should check pricing, etc to see if it suits). Especially if you want 0-1 scaling.
I migrated golang.org from standard to flexible a couple weeks ago, primarily because of increased control over the runtime (custom Docker image) and increased instance sizes (configurable memory, CPU past the App Engine limits, which caps out at 2 GB). See https://golang.org/issues/27205 if you want to read more.
I don't, sorry. While the app was unhealthy, it was spinning up dozens of instances. While it was healthy (on standard), I recall it ran on less than a handful of instances.
It currently runs on two fat flexible instances (GCE VM), but could easily run on one. We only run on two for some redundancy.
I've measured throughput of ~thousands of requests per second on a F4_1G on apps I've worked on previously. They were CPU-constrained, but not doing very much (JSON serialization).
The new runtime might be even better, since GOMAXPROCS is no longer 1.
Is this a GA product or still "beta"? Google's consistent announcement of products as available while in beta is getting annoying, because their SLAs for "beta" products are still 0 and thus impossible to be able to use in actual business applications.
> Beta is the point at which we are ready to open a release for any customer to use. There are no SLA or technical support obligations in a Beta release, and charges may be waived in some cases. Products will be complete from a feature perspective, but may have some open outstanding issues. Beta releases are suitable for limited production use cases.
Wonder if it's still possible to get answers here :) giving it a shot although it's 13h old by now.
We use Go for a production server for multiple of our games, it's used to sync gamestates between devices and allows us to remotely inspect and change player's gamestate. Super handy and it's worked flawlessly! We used to use python, but Go is much better suited as it's less error prone.
One question though, we've been having connectivity issues in China, the GAE is deployed in the US, and as it seems there's no multiregional support. What can we do? Ideally keeping as much of the convenience of the GAE as possible.
100 comments
[ 5.0 ms ] story [ 195 ms ] thread* vendoring
* regular best-practice package structures
* go modules
* the regular Google Cloud client libraries: https://github.com/GoogleCloudPlatform/google-cloud-go
This is a "second-generation" runtime (https://cloud.google.com/blog/products/gcp/introducing-app-e...), meaning that we're now running stock Go in the gVisor sandbox (https://github.com/google/gvisor). We've removed all of the restrictions present in the old runtime, like limited socket and file access. You can even import "unsafe"!
If you're a current Go-on-App Engine customer, you should check out our migration guide at https://cloud.google.com/appengine/docs/standard/go111/go-di... to learn how to migrate from the Go 1.9 runtime to the new Go 1.11 runtime. For the time being, you can still use the legacy App Engine APIs with the Go 1.11 runtime, but you should start migrating to the Google Cloud client libraries.
I have an old app deployed to App Engine. The app itself is rock solid and chugging along fine (I touch it once a year or so), but I dreaded having to deploy it due to the lack of vendoring support and other... peculiarities of the Go runtime. Glad to see this is no longer an issue!
It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?
Yup!
> It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?
I'm not sure about the performance impact. But I can tell you that we're working with the Cloud Memorystore team to have a better memcache story for the App Engine second generation runtimes
You are right that the devil is in the details, and there might be some slight serialization differences. I cannot find an exact list.
Feel free to post here if you need more help.
> the cloud datastore package will default to writing your nested structs as entity values, while the appengine datastore packages will only write your nested structs as flattened sets of attributes.
Other than that, there are no differences.
(GCP Gopher)
either you whitelisted your ips or you used the proxy (which go appengine could already use).
so with go you just needed to use https://github.com/GoogleCloudPlatform/cloudsql-proxy which means you just need to change the query string.
however a new feature is emerging where you can have a private ip that is connected to your internal services, however it is still a beta feature.
Source: myself, but not a googler, just a regular user
(GCP Gopher)
See https://cloud.google.com/appengine/docs/standard/go111/using... and scroll way down to the sample code.
You definitely don't want to check secrets into the repo for the same reason you don't set them as environment variables: It's not secure. The solution is to use Cloud Key Management Service. More info here: https://cloud.google.com/kms/ We use that to either store secrets directly, or to decrypt encrypted entities in a DB (e.g. Datastore).
You can see how we use it here (our project is open source): https://github.com/google/nomulus/tree/master/java/google/re...
Shows how to encrypt env vars with Cloud KMS then decrypt them in memory.
The reason being that these secrets are absolutely essential for many tasks that our registry needs to perform, and the past six years of experience have shown us that Datastore has better availability than GCS. We haven't seen Datastore ever go down unless all of Cloud is down too, whereas we have seen outages isolated to just GCS. Datastore also has lower latency (since it's a DB, not bulk file storage).
One caveat is that the maximum size of a single entity in Datastore is 1 MB -- if you're encrypting stuff larger than that then you'll need to shard (ugly) or just use GCS. Since none of our secrets are anything close to that large, it works just fine.
P.s typo on your link
Hey thanks a lot for your efforts! Your direct communication about the Go runtime progress has been a breath of fresh air compared to the years preceding you. I wonder though, how many developers are there on the App Engine Standard Go team?
Though...I wouldn't necessarily call Python slow! Python keeps getting faster, and we now support Python 3.7 on our second gen runtimes: https://cloud.google.com/blog/products/gcp/introducing-app-e...
Python might be perfectly acceptable of course.
You can go to the filter, turn off all languages but Go and Python, and get a general idea.
There's also the old Computer Benchmark Game, which should be taken with a whole barrel of salt: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Which program? URL?
It’s fifty percent faster than the competing java program.
That Go pi-digits program is only 4% faster than a PHP program that uses the GMP library.
It's not enough to "point out that most languages have C FFIs" when for this arbitrary precision arithmetic task GMP is allowed and others step-up and contribute those programs.
That's true of every comparison -- basic due diligence.
And read how the measurements were made --
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
And read some background --
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
And, of course, look for a task that might be appropriate --
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
The benchmarks game website shows what it says it shows --
"Which programs are faster? Will your toy benchmark program be faster if you write it in a different programming language? It depends how you write it!"
GOMAXPROCS is not 1 for this new runtime. As far as I know, it's an entirely vanilla Go runtime.
Cloud Endpoints currently runs on GAE Standard Gen 1 and GAE Flex. We're investigating a solution for GAE Standard Gen 2 (which go 1.11 is, along with python 3.7, node 8, etc.) and GCF.
Any ETA for Gen 2 or alpha signup?
Flex uses ESP (https://github.com/cloudendpoints/esp), which is deployed as a sidecar, and works with any runtime.
As for EAP: unfortunately not. Personally, I'd like to see a managed version as opposed to the current framework or sidecar approach, as it'll be easier to implement across products, but that means it'll likely take a little longer.
These new runtimes seem amazing (and fix a lot of problems), but we also lose a ton of functionality that made app engine so desirable (easy users auth, images api, built in email sending, search, cron, and others).
It kind of feels like we've thrown the baby out with the bathwater.
We are working on Google Cloud standalone products to replace most missing features (e.g. Cloud Scheduler for cron jobs). Not all of them are ready yet.
App Engine APIs were great when there was no alternative (when GCP did not exist, or when App Engine could not use arbitrary packages), but at the same time, contributed to the "lock-in" of App Engine, which was one of the main criticism. We believe it is in the long term benefit of our users to use standalone Google Cloud or third party services, instead of replying on APIs only accessible in App Engine.
How do you feel about the distinction between Google and Go? It sometimes feels like they're joined at the hip, something which historically had made people queasy (Java and Oracle, .NET and Microsoft, etc). Though Go is a project which came from Google, I personally feel that it better serves the interests of the Go community to treat them separately. Google receives special treatment from Go - the announcement that AWS Lambda would support Go did not receive similar fanfare on golang.org.
Side note: there is some activity on the Go blog which is more mutual cooperation and less Google hivemind:
https://blog.golang.org/go-cloud
But I still wonder why this stuff belongs here.
>[7] Access Not Configured. Cloud Build has not been used in project <project> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbuild.go... then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
So go111 cannot be used without Cloud Build, for which I have to activate billing?
Does anyone have any elucidation?
However, there's still an environment needed to run that binary. For Go on App Engine, this includes the sandbox (backed by gVisor) and the operating system (Ubuntu).
We also tend to call everything in the toolchain the "runtime" - this includes the CLI for uploading and staging your app, and the builder used to compile your program. Not the strictest definition :)
(GCP Gopher)
For App Engine flexible, you can skip that build step and provide your own container, but that isn't possible for App Engine standard. (If you're interested in something like that, see https://g.co/serverlesscontainers)
Note: you need to have go1.11, and by default you need to be in a directory NOT in your $GOPATH for modules to work in go1.11
Copy paste error I guess.
Yes, you don't need to use the appengine context for anything (except to access the services at google.golang.org/appengine/...)
You can use the standard net/http package to make HTTP requests, for example.
Would be great if that was solved.
[1] - https://github.com/miniflux/miniflux
I’m assuming urlfetcher is deprecated now too? I’ll go read the migration guide now ...
Awesome work! Love Go and GAE! Very happy so far.
Glad you're enjoying both Go and GAE!
I’m curious because now it’s seems like there are 3 solutions. Although I really like the simplicity of this solution and the modernizing from the constraints of true classic mode.
So two of the solutions are flex and go1.11 on standard. Is the third solution the older runtime versions (eg 1.9 and earlier)? If so, then there's really just two solutions, since 1.11 replaces 1.9 on standard.
If you can run within the standard environment, I'd suggest you do (but you should check pricing, etc to see if it suits). Especially if you want 0-1 scaling.
I migrated golang.org from standard to flexible a couple weeks ago, primarily because of increased control over the runtime (custom Docker image) and increased instance sizes (configurable memory, CPU past the App Engine limits, which caps out at 2 GB). See https://golang.org/issues/27205 if you want to read more.
It currently runs on two fat flexible instances (GCE VM), but could easily run on one. We only run on two for some redundancy.
I've measured throughput of ~thousands of requests per second on a F4_1G on apps I've worked on previously. They were CPU-constrained, but not doing very much (JSON serialization).
The new runtime might be even better, since GOMAXPROCS is no longer 1.
correction to my post above: 1 GB is the max memory for standard: https://cloud.google.com/appengine/docs/standard/#instance_c...
I say this as a huge fan of Google Cloud and the experience of using Go on App Engine.
"Announcing [some cloud]’s New Go 1.11 Runtime"
That being said, I have no idea how similar submissions of Go content from competing vendors would er... go. ;)
We support custom images for the flexible runtime, but not for the standard runtime.
If you're as keen for that as I am, sign up here: http://g.co/serverlesscontainers
Yes, it's beta.
You're right - if you require an SLA, don't use beta products.
As per https://cloud.google.com/terms/launch-stages:
> Beta is the point at which we are ready to open a release for any customer to use. There are no SLA or technical support obligations in a Beta release, and charges may be waived in some cases. Products will be complete from a feature perspective, but may have some open outstanding issues. Beta releases are suitable for limited production use cases.
We use Go for a production server for multiple of our games, it's used to sync gamestates between devices and allows us to remotely inspect and change player's gamestate. Super handy and it's worked flawlessly! We used to use python, but Go is much better suited as it's less error prone.
One question though, we've been having connectivity issues in China, the GAE is deployed in the US, and as it seems there's no multiregional support. What can we do? Ideally keeping as much of the convenience of the GAE as possible.