Firebase cloud functions are more AWS Lambda competition, where you write JS functions that follow a particular interface and can do certain things.
This is being able to deploy regular NodeJS servers with no special changes or modifications, and have them run an autoscaling sandbox that does a really good job of keeping the server alive (and charging you) only when absolutely necessary. Also scales up infinitely (up to your budget, anyway) in a pinch.
Very nice. Now, maybe allow it to connect to a typical SQL database via Cloud SQL? It seems like the only hosted database you can use with this is Cloud Datastore, a proprietary NoSQL database.
At this point I believe Google has a deep aversion to storing secrets in environment variables. The best way I have found so far is to use Cloud KMS to encrypt the credentials and have the deploy process be able to fetch and decrypt them on the fly. Some folks also store that and other config in Datastore, although I think that's really clunky. You can also use Google Cloud KMS-backed Hashicorp Vault: https://cloud.google.com/solutions/using-vault-for-secret-ma...
the lack of availability of node.js on GAE standard was one of the main reasons I decided to deploy my next.js app on Heroku instead. I may have to try it again to compare head to head...
Is there anything in particular that you'd like to know? E.g. performance, ootb tooling, etc?
No. The real issue is that these APIs/platforms are closed source. Sooner or later you may need to move a service from appengine to a non-gae environment and you will find that nothing works. If you invest in tools/libs (i.e wrappers/ORM around the GAE APIs) the loss is even bigger.
As far as I can tell, Flexible also only supports Node v8 unless you customize and host your own docker container in the registry. OOTB, deploying with v9+ generates an error
Flex feels like a lower level of abstraction. You provide a docker file and what you want your machines to look like (vcpu, ram, storage) and it handles everything else. Standard is much closer to Heroku with the added benefit of being able to call APIs for managed services. If you want control over your environment (including what dependencies are available), you want to use Flex. In every other case, I think Standard is the way to go.
I'd presume the most common are probably other experience with GCP, other applications on GCP, or prefer GCP tooling to Azure. I haven't compared pricing, so no comment there. Lastly, anti-ms pov.
For me there's main reasons are 1. Free tier 2. Features in that order
It is perfect for "hobby" projects that have no revenue today. If a "revenue strategy" emerges, it isofcourseeasy to enable billing but there is no 12 month $300 trial.
Awesome questions.
We do understand from the company standpoint that its difficult to give the roadmap and the timelines of the product. However, a lot of the user's decision is completely based on the road map of the product. Without being clear on this, how would you expect users to use the product?!
I agree, lack of python 3 support is what pushed me to seek alternatives (e.g. heroku) when wanting to build out a project with minimal server infrastructure overhead.
Hi lazypenguin -- in case you're still interested in Python 3 support on App Engine -- we're working on it. If you'd like to be included in a private alpha test, please complete this form: https://docs.google.com/forms/d/e/1FAIpQLSfLXViTUmtY6Ed_Gm9H...
Can someone from GCP explain why the App Engine standard environment has such long lead-times in terms of language support? What is required, infrastructurally, to ship a new standard-environment language?
(PM on the team) To be able to deliver Node.js on App Engine Standard, we had to develop an entirely new sandbox environment. You can learn more about it by watching this I/O talk (starting at 15min): https://youtu.be/ogexnfng_hE?t=14m48s
Historically supporting a language on App Engine required modifying the languages libraries to work on proprietary parts of Google infrastructure. This entailed a decent bit of effort on our part, and also meant that some apps & code wouldn't execute correctly.
We now use technology based on gVisor (https://github.com/google/gvisor) in production for App Engine standard. This allows us to release new language runtimes unmodified. Node.js support is the first runtime based on this new stack.
We completely agree that we should be releasing support for new languages & major versions more frequently...
This surprises me. I had assumed incorrectly that flexible environments were the future of App Engine.
I'm pretty happy with using flexible environment for nodejs. However, the fact that this supports scaling to zero, and has headless chrome built-in, I could see using this for a low-cost headless Chrome environment.
I've had several projects where I've wanted a node server separate from the main app server on which I could use headless Chrome for pdf generation. It's never easy to justify the cost of an extra instance for that purpose. But with a zero scale capability, this could presumably be cheaper and easier to justify.
App Engine is a PaaS (platform as a service), as is PCF. Docker is a containerization platform, though, which is entirely something else. e.g. a PaaS might use Docker (many do).
Standard, in my opinion, is still the best bet of you want most of the things out of the box from GCP. Flex does allow a lot of customization but you miss out on Memcache, Task Queues, Auto tracing in Stackdriver etc which are sometimes really backbone of your highly scalable architecture given that you don't want to manage your own infrastructure.
Not to mention that standard is enough to run a pretty significant free-tier site.
I'd way rather program in JS than Python [edited: thanks yathern], especially now that async/await is a thing in Javascript. Very cool work by the Appengine team.
Yeah but the "standard" env is always behind(at least on the Go version) and on top of that you are even more locked in. The only real advantage of the standard environment is that you pay only for what you use. On the "fkexible" environment you must keep at least one instance live all the time. IMHO you should avoid GAE.
Perhaps you should avoid GAE for hobby projects, but if you can afford it, it can save you some money in not needing any ops. For hobby sites you probably barely need ops.
Standard GAE is great for hobby projects due to the free tier. It's nice to still have a running server that I haven't touched in years.
By contrast, I deployed an app on Digital Ocean and, though they have a solid product, I eventually shut the instance down to avoid paying the monthly fee on an app I rarely used.
The new runtime is 100% idiomatic without any special apis. We do offer certain native libraries pre compiled system libraries (to support headless chrome among other things) and certain ENV Variables for gcp API tokens.
You should be able to lift and shift a GAE Standard Node.js app to any other provider with minimal overhead.
I can't make direct promises regarding the runtime update cadence, but I can promise that I work closely with the people responsible for this and they care a lot about getting updates out in a timely fashion. The new runtime stack makes this easier to do than previous GAE Standard runtimes.
Gigalixir has been wooing me. I've been writing phoenix apps lately and Gigalixir include 10k rows of postgres for free, plus no sleeps and no daily restarts. Definitely worth looking at.
Founder of Gigalixir here in case you have any questions. Also, happy to give out free credits for the standard tier. Gigalixir also works with many languages other than Elixir.
I was looking into it a bit today, but was discouraged after hearing some comments about having to specialize on the application source level to accommodate the GAE environment, and that you have to use their proprietary non-relational database.
Are those things true, or as significant as they sound? (Not being able to directly port my source to another service sounds quite risky...)
That’s ancient history, particularly with these recent launches, as Google Cloud has become a real platform.
This is vanilla node. You need not use Datastore, and can happily talk to a managed MySQL or Postgres instance via Cloud SQL (albeit sadly still via public IP and/or a proxy for now). The same is true of the recent Java 8 support, and will be true of all the new runtimes going forward (as mentioned downthread, the Google I/O talk is the best reference here).
Google Cloud Functions is similar, fwiw. You can even package up arbitrary binaries and exec them (ffmpeg is a favorite, as is imagemagick’s convert).
(PM on the team) App Engine is designed for stateless HTTP workloads. for an email or DNS server, I recommend using Google Kubernetes Engine or Google Compute Engine.
(PM on the team) Migrating from Node.js flexible to Node.js standard should be pretty straightforward. Chances are you only need to perform a new deployment and remove `env: flex` in your app.yaml.
I'm debating between GAE standard or Firebase now for my next project :/ I've always been confortable with nodejs but didn't want to bother with GCP (GAE flex). One part that I liked about Firebase is how it's easy to support multiple login providers (twitter,fb...etc). Is there something similar in GAE standard nodejs ?
I don't quite understand how GAE would affect using Firebase or not. Even on standard you would still use Firebase Auth, there is no GAE specific authentication.
We use Firebase (and Firebase Functions :fire:) for most of our stuff. Then have a few things running on App Engine. What are you building? I'm a big fan of firebase if it makes sense for what you are building!
Firebase doesn't support Access Management with a limit to one project. So if you generate login:ci token it'll create token with access to all your projects and there are no other ways to manage it. I don't know why it still not implemented. Be careful.
Are there differences in the scaling characteristics of the nodejs standard environment and the python 2 standard environment? Does one environment scale faster than the other?
This is helpful to people starting new projects. Using the Standard environment would cost significantly less than the Flexible environment which is one of the reasons I avoided GAE before.
I have a NodeJS app running on flex that I always thought was a little pricey, so I'll probably move it to standard.
However, I do have one question.
My app server currently polls another external endpoint periodically and does something based on the results. If I move it to standard will this still work? I assume not since it can scale to zero.
For handling incoming long-lived connections, you could pair with Fanout [1] (disclosure: founder). Basically the client sees a raw WebSocket, and your backend gets woken up with HTTP requests when things happen.
I haven't seen anything like this for outbound connections though.
Well, the out of the box experience for me is pretty bad. Just created a TypeScript based React App, using npx, which runs locally just fine and when deployed to app engine it just 500s with nothing in the log as to why.
So as a coding n00b I tend to put my stuff on Heroku if only because at the moment that's what I know... and it is free. Any advantage to jumping into Google's App Engine?
App Engine autoscales. To be sure, this isn't a key feature for a lot of sites which don't achieve a ton of scale, but Google can keep spinning up new frontend instances and the Data Store essentially "just scales" if you use it properly.
Heroku is no doubt the easiest to set up and use. I find their free tier too restrictive, however, since (1) apps are put to sleep pretty aggressively and take time to wake up, resulting in failed requests and (2) they have a set quota for total free hours/month across all apps on your account.
GCP free tier provides a lot more in comparison (app engine hours, cloud storage, memcache, pub/sub, cron, egress, cloud functions and more), but you have to be willing to tolerate the vendor lock-in.
take a look at gigalixir.com. I'm moving an app over there right now and they don't do the annoying sleep thing, or the daily restarts. You also get a free 10k row db. Pricing is reasonable if/when you want to scale later too. More expensive than rolling it on GCE yourself, but you get all the convenience of the heroku workflow.
Founder of Gigalixir here in case you have any questions. Also happy to give out free credits for the standard tier. Gigalixir also works with many languages other than Elixir.
118 comments
[ 0.20 ms ] story [ 166 ms ] threadThis is being able to deploy regular NodeJS servers with no special changes or modifications, and have them run an autoscaling sandbox that does a really good job of keeping the server alive (and charging you) only when absolutely necessary. Also scales up infinitely (up to your budget, anyway) in a pinch.
App Engine is great for full HTTPS servers, and is billed on an instance basis (it does scale to zero when unused)
GCF is great for responding to events in the cloud (Pub/Sub, Storage, webhook, Firebase database) and is billed on a per function execution basis.
The actual runtimes are quite similar, though GCF is still Node 6 based for now.
(I work for GCP)
It's really unclear from this what the security best practice is for where to put the Cloud SQL password.
Do we just assume the password is public info and rely on IAM?
This discusses really clunky methods to use the legacy datastore or a bucket to store a password.
https://stackoverflow.com/questions/22669528/securely-storin...
Seems odd when competitors (Heroku, Beanstalk) have a special system for keeping environment variables with the password in.
Edit: For those also interested, here is the relevant issue https://github.com/GoogleCloudPlatform/google-cloud-node/iss...
Is there anything in particular that you'd like to know? E.g. performance, ootb tooling, etc?
Would you like to be an alpha tester? Here's the sign-up form: https://docs.google.com/forms/d/e/1FAIpQLSfLXViTUmtY6Ed_Gm9H...
It is perfect for "hobby" projects that have no revenue today. If a "revenue strategy" emerges, it isofcourseeasy to enable billing but there is no 12 month $300 trial.
From various comments over the years here and elsewhere, I’ve been led to the understanding that Python 3 was “coming soon” several times.
1. Why now?
2. Is it true this time?
3. Will Python 3 keep up to date with new releases, or 5 years from now will we be stuck on 3.6?
4. What is the future and roadmap of AppEngine?
The feeling in the community is that AppEngine is another Google Reader, and death may be imminent. Can you reassure us?
5. What is the future of Standard and Flex?
Should all new projects be started on Flex?
Is standard ever going to keep up with language releases?
Thanks
Historically supporting a language on App Engine required modifying the languages libraries to work on proprietary parts of Google infrastructure. This entailed a decent bit of effort on our part, and also meant that some apps & code wouldn't execute correctly.
We now use technology based on gVisor (https://github.com/google/gvisor) in production for App Engine standard. This allows us to release new language runtimes unmodified. Node.js support is the first runtime based on this new stack.
We completely agree that we should be releasing support for new languages & major versions more frequently...
I'm pretty happy with using flexible environment for nodejs. However, the fact that this supports scaling to zero, and has headless chrome built-in, I could see using this for a low-cost headless Chrome environment.
I've had several projects where I've wanted a node server separate from the main app server on which I could use headless Chrome for pdf generation. It's never easy to justify the cost of an extra instance for that purpose. But with a zero scale capability, this could presumably be cheaper and easier to justify.
I'd way rather program in JS than Python [edited: thanks yathern], especially now that async/await is a thing in Javascript. Very cool work by the Appengine team.
Node is JS, of course (with a different standard library). Did you perhaps mean to compare Node to python or something?
By contrast, I deployed an app on Digital Ocean and, though they have a solid product, I eventually shut the instance down to avoid paying the monthly fee on an app I rarely used.
https://www.youtube.com/watch?v=ogexnfng_hE&t=515s
The new runtime is 100% idiomatic without any special apis. We do offer certain native libraries pre compiled system libraries (to support headless chrome among other things) and certain ENV Variables for gcp API tokens.
You should be able to lift and shift a GAE Standard Node.js app to any other provider with minimal overhead.
I can't make direct promises regarding the runtime update cadence, but I can promise that I work closely with the people responsible for this and they care a lot about getting updates out in a timely fashion. The new runtime stack makes this easier to do than previous GAE Standard runtimes.
(Disclaimer: I work for GCP)
And you should stop telling others what works best for them without knowing anything about what their projects and needs are.
Are those things true, or as significant as they sound? (Not being able to directly port my source to another service sounds quite risky...)
That’s ancient history, particularly with these recent launches, as Google Cloud has become a real platform.
This is vanilla node. You need not use Datastore, and can happily talk to a managed MySQL or Postgres instance via Cloud SQL (albeit sadly still via public IP and/or a proxy for now). The same is true of the recent Java 8 support, and will be true of all the new runtimes going forward (as mentioned downthread, the Google I/O talk is the best reference here).
Google Cloud Functions is similar, fwiw. You can even package up arbitrary binaries and exec them (ffmpeg is a favorite, as is imagemagick’s convert).
https://cloud.google.com/appengine/docs/standard/python/conf...
But it has nowhere near the features of Firebase Auth.
This is helpful to people starting new projects. Using the Standard environment would cost significantly less than the Flexible environment which is one of the reasons I avoided GAE before.
Does cloud fumctions support node 8 yet? Will make life easier
However, I do have one question.
My app server currently polls another external endpoint periodically and does something based on the results. If I move it to standard will this still work? I assume not since it can scale to zero.
https://cloud.google.com/appengine/docs/standard/nodejs/sche...
https://github.com/GoogleCloudPlatform/nodejs-microservices-...
(I work for GCP)
https://cloud.google.com/appengine/docs/standard/java/how-re...
I haven't seen anything like this for outbound connections though.
[1] https://fanout.io
Be sure to perform the build step before deploying.
GCP free tier provides a lot more in comparison (app engine hours, cloud storage, memcache, pub/sub, cron, egress, cloud functions and more), but you have to be willing to tolerate the vendor lock-in.