Most of our stuff is plain old net/http w/some gorilla doing things we didn't want to code at the outset and never bothered to replace later on.
While our main datastore is postgres we have 3 custom datastores that are starting to take a very active role. We have a lot of time-series data, very little relational, a bit of 'document' style and micro-service traces are living in their own.
Not sure if this answers your question but maybe that gets you going in the right direction.
1. Use Godeps to vendor your deps
2. Use Logrus for logging
3. Figure out a deployment script early one. I have a Rake script that uses chef-api to lookup current production nodes, cross compiles locally and scp's the resulting binary out and restarts the process
I'm using Go on App Engine, decently large scale site. It's a pretty good setup. Includes a task queue, database, file storage, almost everything you need. Not having to manage those things yourself is pretty fantastic.
No need to use godeps, setup logging, create a deployment system, or manage linux machines with a bunch of scripts. You can even pipe your logs into BigQuery for serious analysis (and copy your database there too, though this might be expensive for large databases).
Main downside of App Engine is that network stuff has to go over urlfetch/socket api unless you use Managed VM, which is not really production ready yet.
As for frameworks, the builtin http package is fine. Just make some http.Handler structs or wrappers and you're basically done.
Using standard net/http with the gorilla mux and sessions libraries on a Linode 2GB VPS. Serving both HTTPS and HTTP. Sites are low traffic but perform well. One uses the pq library for postgres database access.
From Hailo. Running 200 microservices in production. 199 of which are Go. We run on AWS with our own system that orchestrates Go binaries but are also moving to containers.
That's a lot of microservices. How do you handle changing the interface of a service in a backward incompatible way? Or updating a dependency across all services?
We use protobuf rpc which allows us to maintain some form of backwards compatibility. New fields are ignored by old version of a service. We can also run multiple versions of a service in parallel and send a percentage of traffic to each. Also have the ability to do label based routing so new features are only enabled where a v1.1 header or something of that nature is seen. Updating dependencies across all services is somewhat of an anti pattern in microservices, this chain of dependencies shouldn't exist. Microservices are loosely coupled with a bounded context. In the case of bug fixes based on core libraries we have roll out procedures to update services over time but in a microservice world there's never a point where you look to update a block of services all at once. You should be able to deploy new versions of a service without breaking the system.
Tradeoffs are made for a microservice architecture patterns but when organisations scale to 100+ people we deem these tradeoffs necessary as what we lose in convenience in monolithic tradition, we gain in speed of execution and a highly availably fault tolerant globally scaling system.
Very early on we had a lot of critical updates in core libraries which meant rebuilding everything and releasing but at the time the number of services were quite low and we were still in the R&D phase. Overtime that maturity meant changes going out were mostly feature based and we could afford to lag those things out. You're right though, certain things require rebuilding everything but we try to approach that pragmatically. On the platform team we'd have ownership of say 20-30 services so over a period of a week we can push those through a staging and load test environment then to production. We have 4 or 5 other teams that do the same. There are tools to ensure we can keep track of which libraries are in production and if anything is out dated.
Like I mentioned before. There are definitely tradeoffs to microservices and they don't make sense for every use case but if you look at the companies that adopted this architecture pattern you'll see the common journey they all went on. Monolithic architecture for the first few years, scaled by brute force, money and people. Eventually stalling in development and organisational speed of execution. Taking a step back to reevaluate and then determining a migration path to a new decomposed service oriented architecture.
15 comments
[ 3.3 ms ] story [ 48.7 ms ] threadmailing list - https://groups.google.com/forum/#!forum/golang-nuts
subreddit - https://www.reddit.com/r/golang
While our main datastore is postgres we have 3 custom datastores that are starting to take a very active role. We have a lot of time-series data, very little relational, a bit of 'document' style and micro-service traces are living in their own.
Not sure if this answers your question but maybe that gets you going in the right direction.
Go with Google App Engine. Works well both with SDK and on AppSpot.
1. Use Godeps to vendor your deps 2. Use Logrus for logging 3. Figure out a deployment script early one. I have a Rake script that uses chef-api to lookup current production nodes, cross compiles locally and scp's the resulting binary out and restarts the process
No need to use godeps, setup logging, create a deployment system, or manage linux machines with a bunch of scripts. You can even pipe your logs into BigQuery for serious analysis (and copy your database there too, though this might be expensive for large databases).
Main downside of App Engine is that network stuff has to go over urlfetch/socket api unless you use Managed VM, which is not really production ready yet.
As for frameworks, the builtin http package is fine. Just make some http.Handler structs or wrappers and you're basically done.
supervisord: http://supervisord.org/ is working great so far.
Tradeoffs are made for a microservice architecture patterns but when organisations scale to 100+ people we deem these tradeoffs necessary as what we lose in convenience in monolithic tradition, we gain in speed of execution and a highly availably fault tolerant globally scaling system.
You can learn more about our journey on our blog https://sudo.hailoapp.com/services/2015/03/09/journey-into-a...
And slides from former platform tech lead Matt Heath https://speakerdeck.com/mattheath
Also from current platform automation lead Boyan Dimitrov http://www.slideshare.net/nathariel
Various talks can also be found on youtube.
"Updating dependencies across all services is somewhat of an anti pattern in microservices, this chain of dependencies shouldn't exist."
But what about, say, your tracing library. It seems like that would be shared across all your services, right?
Like I mentioned before. There are definitely tradeoffs to microservices and they don't make sense for every use case but if you look at the companies that adopted this architecture pattern you'll see the common journey they all went on. Monolithic architecture for the first few years, scaled by brute force, money and people. Eventually stalling in development and organisational speed of execution. Taking a step back to reevaluate and then determining a migration path to a new decomposed service oriented architecture.