I've come to terms with the fact that every tech trend is cyclical. Just look at the comeback functional programming is making right now. Or the way we went from thin clients (terminals) to thick clients (PCs) to thin clients (early web pages) to thick clients (progressive web apps). All of this has happened before, and it will all happen again.
How do you deal with the people who emphatically insist that this isn’t a cycle, that this is new, and that there are no cautionary tales from the last cycle that they have to contend with?
These people make me tired, and it feels like they are everywhere.
I agree with your point, but I often think the attitude you're complaining about occurs on both sides. "That's the same thing that we did X years ago" shows a similar level of ignorance.
Most changes over time are evolutionary vs. revolutionary. This often means at an abstract level, new things look very similar to old things because they are. But, there is change (sometimes minor) that makes the new thing new. That minor change can also be the difference between something being used or going away into history.
The broad strokes ideals (read: dogmatisms) are cyclical, but technology still progresses. i.e., we shouldn't all ditch Clojure for K just because functional programming has had a heyday before; Clojure is a legitimate evolution of the paradigm. But it's also not unlikely that the present hype will die down in a few years and some new form of OOP will make a resurgence, ad infinitum.
It's hard to know actual progress from fads until roughly a decade down the road. After being burned, I try to let others be the guinea pig now (unless I'm paid to do R&D). OOP was supposed to be the cat's meow. It turned out fairly good for certain infrastructure utilities/API's, but lousy for domain modeling, but was originally sold on domain modelling. It took people a while to realize this. OOP has use, but not the way the hypesters sold it.
I just take everything with a spoonful of salt. I enjoy hype to a certain point; it's fun and exciting, but I no longer buy into any one thing as the end-all, and when the party gets tiring I don't feel bad chilling out about it.
"first as tragedy, then as farce" has never been as true as in the age of node.js hello world "microservices" with 136 MB of low quality dependencies in node_modules (I just checked mine). I can run Java servlets with less dependencies, including the JVM.
SOA never required an ESB, this was simply vendors trying to shoehorn in a saleable unit of software - which they succeeded at better in the 90s. Likewise I've seen people implement microservices with an ESB in the middle.
Long an short of it is the ESB is an anti-pattern, always has been and SOA and microservices are the same thing.
Without an ESB SOA services and microservices are just remote procedure calls. Unless there is a real need like a client-server architecture then remote procedure calls are just an extremely slow version of normal procedure calls.
Not trying to be pedantic or anything, but it's also extremely insecure, extremely unreliable and extremely hard to implement correctly when compared to a normal procedure call.
SOA is more than just RPC it's about the decomposition of your organisation in to cohesive (from a functional(ity) perspective) units of software, to reduce the need to make the remote calls, but more importantly allow you to organise your engineers into small enough groups of people to be effective (7 give or take one person).
ESB simply add more complexity in the middle that needs yet another team to manage.
I think abstracting the differences away between the 2 seemingly equal technologies is not a good process for understanding them better. Yes, we seem to go back on ideas but this is because the market is constantly evolving, not because we forgot about the past.
SOA was not that big of a hit because the time wasn't right. Just like MySpace before Facebook, the market wasn't ready. And to say they are the same is to ignore the advances each tech made.
You could push this argument to the extremes and say "well, Microservices are just Processes on different computers talking through a network IPC and this $unknownProject has done that back when Ethernet first started".
Sorry for nitpicking, but I think the central value proposition of microservices/SOA is that you communicate with you backend via well-defined network protocols rather than eg. procedure calls so that the client and server technology stack doesn't matter.
- you can keep a service running long after the project that created it has disbanded and/or the runtime and build environment is deprecated on a given O/S
- you don't necessarily need to deep-dive into the business logic of legacy services
- you can replace a service implementation with another one running on another stack without having to adapt service clients
- you can change auth mechanism outside of the service implementations
- you can scale out (CPU-bound) services onto multiple hosts
Mind you, these benefits are not achieved by using microservices/SOA in itself; the contribution of microservices/SOA, however, is that these design goals aren't impossible to achieve because of architected reasons.
One day I had an epiphany: Microservices (or SOA if you will) are just Unix-mindset applied to web applications. Do one thing and do it well, then use all these stand-alone complexes to get an actual result you want.
"Those who don't understand Unix are condemned to reinvent it, poorly." -- Henry Spencer
>Microservices are also lighter than SOA. While SOA services are deployed to servers and virtual machines (VMs), microservices are deployed in containers. The protocols are also lighter. This makes microservices more flexible than SOA
Both of these are wrong.
A SOA service of yore would be much lighter (in CPU, if not memory) than a microservice of today. Heck, it had to be -- in the 90s they couldn't afford to run a JS runtime to power their service. It was either C++ or Java.
The protocols were also lighter, then, not now. Except if the author means simpler and easier to work with. Else, CORBA or DCOC are lighter than serializing to JSON and back.
In the eighties we used turbo pascal, 20 years later we found out we were writing microservices. For us it was needed because you could only write programs under 720Kb, So send a message to another program over a 5Mb network en let that program handle a specific task and port the results back to the main program.
Is a program running in the early versions of the JVM really faster than JS on the current version of V8? Current JS engines are very fast.
By the way, Server Side JS was a thing in the 90s, though little used. On the other hand, Perl was quite common on websites, despite being much slower than C/C++.
Perl could be nearly as fast as C/C++, depending on what your service does and how you wrote it.
The early days of running a cgi script for every request had awful performance, caused by the overhead of loading and compiling the Perl code before executing it. But there were ways to avoid that by pre-loading the code. One way was to use Mod_Perl on Apache, for example. I wrote a number of services that way, and also wrote my own pure-Perl standalone http server that could pre-load an entire web application.
The other key factor was to use canonical expressions when writing your Perl code. There was always "more than one way to do it", but there were only a few ways that the compiler would recognize and use optimized functions for. Once you learned to take advantage of that, you could get your code running mostly in hand-written C functions. It wasn't as fast as native C because of the way Perl variables were represented, but it could be pretty close.
The "trick" is to use HTTP caching (build right into the protocol) so that not every request needs to trigger full-page rebuilding. If you have eg. mod_cache (or external reverse caches) enabled, your page composition script could be as slow as it wants, it doesn't really matter.
For static or static-ish content, absolutely. But I was talking about (and writing) web applications and web services, where the responses generally aren't cachable.
What made "Service-orientated-architecture" stand out in the 90's and 2000's was standardized communication protocols. It made integrating across teams, and even across companies easier than it's ever been before.
But there were no best practices around service architecture. Sure, methods were decoupled by responsibility (maybe). But when companies thought about their "Web Service" strategy that meant putting up ONE service exposed to the world, and moving on.
Internally, "SOA" meant 1-5 services for a mid-sized shop.
This worked for a while until it didn't. First, the services became monoliths of conflicting responsibilities, then the mobile revolution and big data led to scalability challenges that the era of SOA never had to consider.
THAT is what microservices promised to solve. By being brutal with decomposing and decoupling your concerns you could scale each intelligently and independently. Containers are just an implementation detail, and weren't around yet when Microservices first made their impact.
Of course, plenty of people don't get this. They think of microservices as just another buzzword. So, it's been ruined like "agile" has, and you hear companies about their "Microservice architecture", when really all they've done is migrated their backend to node.js and moved to the cloud.
Shame. It's a good idea a lot of the time (and sometimes it isn't, as with all good ideas).
You're also making a lot of assumptions. I started working in 2004 and in 2006 I was already doing 'microservices', I was responsible for a few web services (.NET) (with a few endpoints each), other colleagues had others. These web services where part of a bigger system (out of the many that we had). Emphasis here is on few endpoints, the services where split on functionality, some services talked to other services, some directly to the database. One of the reasons for the split was also because we were using Visual Source Safe and multiple people working on the same code was hard. So yes, I think 'Microservices' is a buzzword, it's been driven by the use of containers, deployment automation, and the assumptions that people have done monotliths because they didn't know better is wrong. Every sane engineer who has put too much functionality in a service which at some point caused problems with scaling, agility, etc, would take measures to address it.
Yes, people were doing microservices before the name appeared, but that's true of all design and architectural patterns. A few people/teams implement similar solutions to similar problems, then someone recognizes the similitude and names that general pattern. Then it becomes easier to know in which conditions it might be useful to apply it, especially for people with less experience (assuming they can see through the snake-oil sellers that inevitably pop up).
>> Then it becomes easier to know in which conditions it might be useful to apply it, especially for people with less experience
In my experience it's exactly the opposite that has happened. I see microservices being abused all the time, just because someone read a blog and thought they were cool.
The same thing happened with SOA by the way.
It's funny that the main reason you switched was because of the difficulties of VSS. Your tools forced you to make decisions about how you code!
We just switched to SVN. The migration was an easy < one day job, a few weeks of pain with some developers grumbling that it was too complex and then everyone was (mostly) happy that we no longer had to hunt down the consultant who had locked a particularly critical file and was now out of the office the rest of the week.
No, this was a decade ago at a company I used to work for.
Although a couple of years ago I did have a job where they used it, but I quit after 2 months. For some reason a couple of the Devs would constantly check in blank commits. Not sure if that was a product of VSS or because the Devs were awful (part of the many reasons I left so quickly).
It's not quite as bad any more, but still terrible when compared to Git or SVN.
That sounds pretty terrible. Why did they need a bunch of wrappers?
I’m a big fan of vanilla tooling. Complex build, version control, etc gets no love and ends up ten years later as a big mess that no one understands or wants to support.
Because their entire workflow had been about these RCS wrappers for 5+ years and they weren't ready to change that for existing projects etc. New projects (such as the one I did after that) were thankfully just plain ol' SVN commands from the start.
I've always got the impression that micro-services are a bad idea most of the time, apart from sometimes where they're not. Mainly if you're really big.
There are a lot of advantages to having a monolith, especially in statically typed languages.
Every extra service you add is costly, it involves setup, build steps that can break, extra things to monitor, more complex development environment, less flexibility, harder to redesign, harder to fix flaws in the original architecture, etc.
To me it seems guaranteed that it is more expensive, regardless of if you use [insert trendy tool here].
There's always something that can break. More parts, more breakdowns. The services have to communicate, those communication channels are extra points of failure. What happens when one service inadvertently passes along something another one can't decode properly?
Ooops, wasn't expecting unicode! Ooops, wasn't expecting a long int! Oops, the JSON encoder we used in this service spits out stuff the JSON decoder in this one hates! Ooops, we upgraded a package in this service, but not that one! And on, and on.
And when something does inevitably break in the chain, there's the extra cost in debugging of figuring out which bit actually broke.
Agreed on all counts. The only time I recommend multiple services, rather than just load-balancing across monolithic applications, is when part of it's written on a different stack. As an example: I'm writing a gizmo with a Ruby core and a NodeJS ingest endpoint, and yeah, I'll probably run them on separate machines.
But here's the dirty secret: if you have a load balancer in front of your application, you already have a way to send requests for feature X to a separate pool of machines if that feature is a hot spot for you currently--and unless your application hits some really degenerate cases your base RAM load Simply Should Not be that high for your application in the first place, making the marginal cost and machine sizing floor less of an issue.
There are benefits to microservices in some environments, chiefly to reflect your org chart when your human protocols are breaking down. They tend to be marginal, and whether they outstrip the increased overhead of dealing with more moving parts (whether in your code or not; I quite like k8s but I still wouldn't want somebody less comfortable with it than I am running it in production, and plenty of those folks do!) is in many ways up in the air.
It's important to realize the level of abstraction going on here. This is all philosophical discussion of how much Buddha-nature is in a particular company.
If your application is composed of single-purpose modules, which are well documented, which have clean public interfaces, which are instrumented for monitoring, which can be scaled by using a load-balancing message router of some kind... and it works well, who cares whether you call it micro-services, Web Services, SOA, or FactoryFactory-Producer/Consumer Ecosystem?
You should have exactly the number of services required.
This may be hard to plan out from the beginning, but if all is basically sound, you may be able to reduce that number later. (E.g. when a module is called only by one other module, and there's no requirement for them to scale independently, maybe they should be one module.)
I feel like you're just saying the same thing a different way.
Most development doesn't happen in a "we know exactly where we're going" way.
This all sounds like some perfect, waterfall development cycle, everything's designed on paper in UML and the code is then written to spec without any changes, no requirements change, no interfaces change.
That model of development simply doesn't exist in the real world.
Your profile and blog say you're a sysadmin, do micro-services keep you in a job ;) It's one thing to make a bunch of micro-services as a sysadmin. It's quite another to base an ever growing and mutating production code base around them that actually has no clear path to success and an only vague understanding of future requirements.
Oh, no, my workplace avoids microservices like the plague. I can only think of one: PDF generation is handled by a network call that hands a webpage to a heavily firewalled (in and out) special container that hands back a PDF. There's no state in the container other than what it's processing, so we can discard and regenerate them.
We keep development and operations staff tied closely together as a team, and have done since well before "devops" was coined.
Even if you're not really big, sometimes it's helpful just to have everything broken up and not have to go to a 750,000 LOC solution and find the 10-line file that you need to edit.
I'm on the architecture side so am bearing the brunt of the complexity but the senior and especially mid level developers who have code open 7.8 hours a day say the few things we do that have real microservices are the easiest to debug and work on. The monolithic apps take longer to get less done.
Agreed though that it certainly adds complexity on the admin/arch/ops side.
Having microservices doesn't really solve this problem, though, because you still have all those lines of code, plus all the extra code to make it all talk to each other. If the problem is one of organization, that can probably also be solved in large codebases without breaking into multiple systems.
A monolith is extremely simple to traverse with a debugger, a microservice is not. I can make a dozen jumps over/into/out in less time than it would take me to open Fiddler or man tcpdump.
I have heard that some companies have tooling that makes the microservices debugging situation tolerable or sometimes even better than monolith (e.g. writing tests for you by snooping the network layer), but I have yet to play with any. Do you have recommendations?
It may be worthwhile for really complex and "business vice separated" services to be written as micro services because it gives you clear separation of concerns. But, in reality, to often you see a single logical service getting implemented with a lot of (N/W + serialization + management) costs unnecessarily just because it is cool to say that you did micro services.
My thumb rule is : If you don't foresee different teams of devs working on these different services, then you're better off leaving it as a monolith and scaling it as a whole application instead.
Yes, I've always seen microservices as an organizational win over any type of technical win.
Basically having separate teams work on one code base with one deployment cycle can sometimes be really tough. And sometimes you can get a big win by splitting up this service so multiple teams can each have their own service to focus on.
This whole team of one developer supporting 8 microservices seems ludicrious to me.
I tend to like the idea of the 2 pizzas rule from Jeff Bezos.
Having a small team (let say under 10 people) manage a service end to end (from spec to code to production) can be really efficient.
On large code bases/large products, this can help a lot, offsetting the cost of a more complex architecture.
However in a microservice architecture, the APIs between services is critical. It's a contract between services, and it should be clearly stated, you cannot go berserk every other day and change it. You also need to establish clear rules about versioning, deprecation and end of life of said APIs.
(by the way, it would be nice if we had a standard to negotiate API versions, for example stating that a rest API must always have a /version which lists the available versions, if some versions are deprecated and when they will be EOLed).
That's completely orthogonal to microservices, though. Having your modules communicate to each other via function calls does not make them significantly less modular than having them communicate to each other via network requests, but it removes a lot of complexity in serialization and network protocols.
Technically you're right. Practically, though, putting some code in one process and some in another helps you leverage Conway's Law to make lazy, rushed, or untrained developers respect the intended separations in your designs.
You won't just jam another getter in that Singleton if that Singleton is in another process.
It can work the opposite way just as easily. Code that should go in one microservice ends up in a different one because the developer is rushed and doesn't want to go to the extra work of changing multiple services.
Microservices really shine when your setup requires HA everywhere. If you're running multiple copies of everything and need your overall product to still run while parts fail, then it's a lot easier when the individual parts are small microservices that do one thing with predictable operating conditions.
If you're not running in High Availability, and can tolerate one copy of everything, then monolithic probably makes more sense. Glue it all together and roll out improvements across all the parts at once.
This is not a real distinction, though. You can have N instances of a monolith if it's written smartly (generally speaking, where "smart" equals "is a twelve-factor app").
And frankly, for the 95% and probably the 99% case, running a copy of each monolithic application its own auto-scaling group is probably significantly less work and less hair-pulling than the chasing the microservice dragon.
> I've always got the impression that micro-services are a bad idea most of the time, apart from sometimes where they're not.
I'll argue that if you're building something, you should probably build it as a monolith, with the exception of some well-defined services (like authentication & authorisation). Once you've got version 1 built, then you know enough about the problem that you're ready to consider breaking stuff off into microservices (this is why auth^2 make sense to break off from the get-go: they're pretty well-understood at this point, so we all should know how to handle them).
Premature microservicement is the root of much evil. Trying to break things up into services before you actually understand the problem will only lead to tears & suffering.
"By being brutal with decomposing and decoupling your concerns you could scale each intelligently and independently."
Yep. This is the first concept he misses.
"They think of microservices as just another buzzword. "
And this is the second one.
What he's describing as SOA doesn't match what I've seen. Even your 1-5 services example is rare. It was mostly taking a buzzword, loading up on some tooling, and cranking WSDLs. You could end up with dozens of those little rascals right out of the starting gate. (It should have been 1-5, agreed. I just never saw it work like that in shops of any significant size)
One common theme of the tech industry is that excellent ideas get picked up by bullshit artists with time to waste and ink to spill and are then turned into something completely anathema to their original intent. Anecdotally, this seems to happen with more speed and vigor than in other domains.
It's worth remembering how painful SOAP was, and what a disaster the early implementations were, and how little agreement there was on anything, for many years. Pete Lacey has written a hilarious satire of the whole thing:
Maybe SOAP wasn't the entirety of SOA, but back then it was a big part of it.
I agree the term "microservices" has been ruined by those who came in with a very specific agenda. When I was at Timeout.com, in 2013, I used the phrase "an architecture of small apps":
It was a few months later, when Martin Fowler wrote his essay about it, that I first heard the term "microservices". When I read his essay, I assumed he was talking about the same thing that I was talking about. But I later found that the people promoting microservices defined it entirely in terms of HTTP, so I've gone back to talking about "an architecture of small apps" when I am talking about small apps that talk to each other with something other than HTTP, such as this kind of architecture:
> Maybe SOAP wasn't the entirety of SOA, but back then it was a big part of it.
Well, SOAP and Wiz-Dull.
Many of the robotics platforms in common use (e.g., ROS) consist of independent, single-responsibility services that communicate over a shared message bus using a protocol that's not HTTP.
But for enterprise apps, it's 2018. Other protocols? What other protocols? You should be using HTTP/2. It can MUX multiple streams over the same connection. FINALLY! At long last! We're no longer limited to a single concurrent stream!
Sure, but you could point a generic client at a SOAP endpoint, and have a client for it, since the endpoint told the client what methods were available and what their signatures were.
REST lost that.
I think GraphQL has it back again. Any GraphQL people want to comment?
GraphQL Person. It does. But GraphQL is not a silver bullet. It adds a decent amount of complexity on the front end (assuming web / mobile app) that I don't think is worth it unless you're really making use of GraphQL's feature set.
Thanks. I feel the endpoint redundancy issues in REST APIs for what I'm doing now (and what I've worked on on the past) but there's so many other things that could deliver value I'm holding off on GraphQL till my next big project.
Our shop tried microservices, and so far it's a BIG FLOP and time-drain. Conway's Law overpowered it: if the org structure (both dev's and domain org) don't match the microservices partitioning, bleep happens. You cannot force sharing/reuse where the parties don't want it (both dev and domain).
If you share something, you create a dependency; and that dependency needs to be managed both technically and domain-wise. Group X now has to depend on Group Y's decisions if they both use/share service Z. If X and Y don't buy off on a common dependency on Z, they get frustrated and blame devs and the other service users.
Stored procedures and SQL allow data/service sharing with less code anyhow.
> With microservices, though, you can make changes one service at a time.
That's the main property that sets microservices apart from mere SOAs: Microservices reduce the risk of change.
With properly architected microservices a single deployment is unlikely to impair an application as a whole whereas with traditional monolithic deployments a single change can severely affect the functionality and operations of an application.
This is also why you probably don't need microservices if you don't need that kind of fine-grained control and risk mitigation.
But often you will end up with a critical service, like for example the service responsible for the order database. Breakage in this one will impact the system as a whole.
This in turn might point to the services not being cut alongside the right dimensions, that is they might be cut horizontally rather than vertically.
Microservices lend themselves to being used in conjunction with domain-driven design. A single microservice should only deal with one domain.
If you have a service that covers cross-cutting concerns such as orders that service is not a microservice. In order to improve the design in your case each microservice could access and manipulate order data according to its scope.
That way, if one of the services fails only the small part of the order functionality implemented by that service will fail while the remaining order features will remain intact (to some extent; steps in a linear order process that depend on each other of course will still fail without successful outcomes from previous steps).
And we went from SOAP to REST because REST with JSON was supposedly simpler.
Then they realized that they needed some type of way to validate the JSON so they introduced JSON schema.
Then they realized that they needed some method to make a REST Api discoverable so they added the Swagger protocol (OpenAI). Now we are right back to being as complicated as SOAP.
this one is my favorite. seeing my coworkers add swagger and get all excited over a description language for their API.... but then act like WSDL is some abomination is really funny
WSDL is the ultimate api :) it would make my day to see the wsdl comeback of 2018
One of the service providers we use recently started deprecating their "SOAP" APIs in favor of "REST". Which doesn't bother me because with this provider they're far easier to work with.
But since they provided WSDL before, their "REST" APIs now come with WADL files that offer zero value to anyone. Swagger would be a massive step up from that.
The value-add of SOA over "REST" (as in JSON-RPC-over-HTTP) is standardized request/response formats for error messages, distributed transactions, and authentication/authorization, and standardized interface artifacts (WSDL and XSD schemas, policy files).
I think that's a bit harsh - I've been through a number of these cycles: ONC RPCs, CORBA, XML/SOAP and now JSON/JSON Schema/REST and I much prefer the latter.
At least with current JSON and REST API I stand a good chance of experimenting using just an editor and curl - good luck trying that with SOAP.
I didn't do too much with SOAP back in the day and only got into web services while REST was gaining popularity. But when I go back and have to support a "legacy" system and integrate with SOAP, I'm still amazed at the maturity of some of the tooling around SOAP.
Personally, I think SOAP tooling can be a very leaky abstraction - things are great (e.g. adding a reference to a web service in Visual Studio is trivial) but then when something doesn't work quite as expected (particularly when calling between different tech platforms) debugging could be a nightmare.
My experiences with SOAP were that 99% of the time it was fine - but those other 1% seems to take the other 99% of my time... ;-)
Yes, my experience with SOAP is that even well written and maintained clients only work out-of-the-box with like half of the vendor-supplied services you come across.
In many cases, I'll simply formulate xml requests myself and use a generic http request client because the implementation ended up being simpler & easier to maintain than all the confusing hacks I had to do to make zeep or suds (the most popular SOAP clients implemented in python) function properly.
I don't hate SOAP, and I think it gets a terribly bad rap, but it's pretty easy to spit out a WSDL and not follow it in execution. Ditto OpenAPI, of course, but at least anecdotally it seems like less of a problem. It's not no problem, of course, and one of the reasons I wrote Modern[1], for OpenAPI 3, is to have response validation in development mode, not just request validation mode in dev and prod.
> Then they realized that they needed some method to make a REST Api discoverable
Which is silly, because REST is discoverable by definition, since you should be using hypermedia. And if your API is not discoverable, that's a good sign your service is not actually REST.
The "No True Scotsman" defense seems to be used a lot for REST. Don't get me wrong, I try to stay RESTful as much as possible but hardly anyone writes "truely RESTful APIs".
That's pretty much what I said! Web frameworks don't help you at all in this regard, which is why REST is unncessarily difficult to implement. Most web frameworks give you just enough for HTTP-based RPC.
Fielding's thesis doesn't really specify anything in particular about human discoverability, except that REST systems are typically user agent driven compared to typical distributed systems.
As for the nightmare of REST, the real question is whether this difficult is intrinsic, or merely a product of immature tooling and poor understanding. No web framework I've seen, apart from the Waterken server, actually helps you achieve REST. They're largely all HTTP-based RPC frameworks.
> Then they realized that they needed some type of way to validate the JSON so they introduced JSON schema.
This is not necessary in many languages that centre around domain objects. The validation is done automatically when the JSON gets mapped back to the object. It's far better than JSON schema anyway since you can enforce whatever custom validation you like.
> Then they realized that they needed some method to make a REST Api discoverable
But it doesn't have to be discoverable. And if you do want it then you could write the documentation by hand or expose a simple REST index endpoint.
> Now we are right back to being as complicated as SOAP.
> It's far better than JSON schema anyway since you can enforce whatever custom validation you like.
IMO, the value of JSON Schema isn't that you have no validation that exists outside of JSON Schema, the value is that you can front load a TON of the validation into it, and share that vocabulary between your services. If you publish a JSON Schema about your endpoint, I can read that or feed it into tooling to produce a form to consume it. As someone working on the Elixir JSON Schema lib currently, the places where we COULDN'T express the validation with JSON Schema were often more interesting.
SOAP is an absolute nightmare to work with. A WSDL is not even remotely comparable to Swagger.
For one, Swagger is for humans not machines to consume. At it's core it is a way to standardize documentation that can be easily displayed to humans. WSDL is not only used by humans but also by client and server apps.
REST APIs themselves if done right though hypermedia can be self-documenting and require no download of large spec files to use it.
SOAP also suppers from a number of quirks. One for example, that order often matters. Yes you can write a WSDL that doesn't require order but nothing is worse than having a bug because <price> has to appear after <token> but before <expirationMonth> and any other order throws anon-specific validation error. Yes, I know there are tools that can help with this but I have never felt compelled to download a tool for REST. Curl does just fine for me. And to top it off a lot of the best SOAP tools are Windows only.
What I think you really should be pointing at is RPC (of which SOAP is a terrible no good implementation of). RPC indeed offers some benefits over REST and can even be simpler. At the expense of losing the discover and hypermedia aspects of REST if done right.
How is that any different than a WSDL? A WSDL could also be said to be designed for humans -- it's just XML -- but there are code generators for it too.
I've been developing so-called "REST" services for over ten years (and SOAP services before that), and not even once were REST principles actually applied in the projects I worked on. As in, the client/browser discovers from an URL the interactions with a resource without out-of-band info for minimal coupling of client and service. I agree though getting SOAP/XML right takes typically considerably more effort than JSON.
Re: statefulness ("order matters"): there are intrinsical stateful processes that aren't represented adequately by a REST service. Think of a basic "Purchase Order" resource representing purchased items on an e-commerce website. You generally can't add and delete line items freely once you have placed your order, but you may cancel individual items (unless they're already physically on your way), you may RMA items, etc. Representing this as a flat "Purchase Order" resource with GET, POST, DELETE, PUT primitives is useless as it isn't telling your web shop front-end developers what actions are available at any one time.
> I've been developing so-called "REST" services for over ten years (and SOAP services before that), and not even once were REST principles actually applied in the projects I worked on.
As a consultant who has talked at conferences on APIs and is brought in to explicitly fix these kind of issues I have seen a lot of this too but correct REST APIs do exist. I know because my team builds them.
You're experience is anecdotal and as is mine. But just because a lot of people assume REST is something it isn't doesn't mean REST is inferior to SOAP.
> Re: statefulness ("order matters")
My order matters comment had nothing to do with statefulness. I mean in the XML. In SOP this might work:
Why the heck does the order of keys in my object make the API fail!?
Granted that is not a limitation of SOAP but more XML schemas in general. Json is more likely to be order of values agnostic.
But to play devil's advocate I can see the argument where if you are using SOAP to make multiple RPC calls in the same envelope than order is important.
Ah ok. But to assemble XML/SOAP for these kind of payloads you'd typically be using a lib such as JAXB, Jackson/gson, Spring (for Java at least) or SCA anyway.
Moreover, you could design your XML Schema such that both variants are accepted (using XSD "all-groups"), though typically record-like data structures are indeed described using XSD sequences which would cause the issue you mentioned. Again I agree XML/XSD isn't a great payload/schema mechanism for data structures such as maps; after all SGML/XML was invented for encoding text. It's being used because it was the first mechanism on the scene, and it continues to be used because it's still much more capable and mature/established in terms of data validation than anything JSON, especially for protocols used in communication with external partners/banks/authorities.
And we went from SOAP to REST because REST with JSON was supposedly simpler.
Then they realized that they needed some type of way to validate the JSON so they introduced JSON schema.
Then they realized that they needed some method to make a REST Api discoverable so they added the Swagger protocol (OpenAI). Now we are right back to being as complicated as SOAP.
Except you can ignore the schema and swagger and have it just be simple, if that's all you need.
The biggest problem with SOAP is that it is too easy for the service provider to just dump their internal data structures without any thought to how it should be serialized for people using the data. XML is Particularly nasty in this regard, because it doesn't map well to the data primitives of most dynamic languages.
That said, I usually just treat SOAP APIs as if they were REST. By writing a class, and some wrapper methods for the parts I need. Instead of trying to consume the WSDL and use it the way they envisioned I would. Then again, if Python had better SOAP support I probably wouldn't even do that.
I'm not an expert, but what is the difference between a plain old ESB and what we now call an "API Gateway" or "API Manager" (such as Kong or WSO2) in the world of microservices?
I know of some architectures where the API Gateway has a lot of logic put into it... And you have to go through it to reach any of the microservices in the network.
Isn't this a single point of failure and similar to an ESB?
The biggest difference is that services don't use the gateway to talk to each other. And hopefully, the gateway isn't transforming responses. If services do use the gateway to talk to each other, then yes, it's pretty much an esb
There is no microservices protocol description or RFC so its hard to say anything definitive, but yes you are right. We typically like to have a DB table (or Mongo collection) owned by 1 microservice. This allows changes to be made to the underlying datastore without affecting multiple applications. The other apps that need data, just get it from the microservice without caring about how the microservice handles it.
I am in a team now that is just started using to use microservices. And I am noticing that we are stuck on a lot of DB changes we want to make because there are various applications accessing the same tables. So now we are getting into the habit of constraining back-end data stores (tables and collections) to just one microservice because we feel the pain of not doing that now.
Microservice is a monolith. So what you're describing is essentially: I've got a few front-end microservices tied to 1 huge microservice I call the database.
There's no "data layer" that is managed outside the microservice. It _is_ a microservice.
That's one of the reasons Google uses a monorepo: any microservice can easily import a database instance and start using it as part of it's structure.
Probably non related but I have a question - how do microservices scale in terms of communication to other microservices or to a monolith?
We have broken off two of our monolith’s components into a microservice and for now it’s working great - but there are just some things I don’t think make sense to decouple and isolate. I’m afraid I’d get in an integration hell. Has anyone had such an issue?
When that will happen in your case is highly dependent on what you're doing, but it will happen. Eventually you will run into situations where implementing a single small feature requires changes to 3 or more systems, plus deploying all of those systems to all of your environments, code reviews for each service, etc. The overhead explodes as you break more things apart.
I started coding when SOA was in full swing and microservices was just something people were doing before it was called microservices. Basically the SOA contractors (and newly minted certificate holders) were pitching these complicated protocol interactions. There was just way too much overhead. So, many developers that just needed to get stuff working whipped up REST services for the front-end to consume. Ultimately these REST services made their way to production and they worked. And people coded them faster. This seemed to have been going on in multiple places and people were discussing it online and the term microservice was coined somewhere along the way.
In the end SOA just had too much overhead, rules, and protocols. I remember seeing what these guys were doing with SOA and I would just ignore it and create REST services with JSON. We were able to get stuff in production. Eventually I found out that what we were doing was part of a movement called microservices and then started following and learning what other folks were doing.
I've been working for telecoms for ages and it's been like this since forever. Bunch of independent hardware devices from dozens of vendors + software services from independent vendors and you gotta integrate them into one. HLR knows about your users, billing knows about other part of those users, then you have devices, everything independent and glued together via SOAP. In my previous job company had multiple SMSCs (short message service center) and you had to target the proper one, depending on the geography of the customer and you'd go to proper HLR to find the customer and their geo location (you should read country here, not precise location).
And then people running around screaming SOA, MICROSERVICES, JSON and I like "You don't say? o_O?"
I hate the graybeard need to say $NEW_THING is actually $OLD_THING but with a silly name because these silly kids don't know anything new, while both glossing over the bleak hellscape of $OLD_THING (WCF service development, anyone?) while grossly misunderstanding $NEW_THING.
As other have pointed out he's ignoring key failings of SOA and critical differences between SOA and microservices.
Overall I like the article but I think he's misstating a few key things.
And maybe a nitpick but:
> It lends itself to the Continuous Integration/Continuous Deployment (CI/CD) model of developing large, complex applications.
Really? Because I thought CICD was the standard way to deploy any application more complex than a spike or proof of concept (which probably doesn't need to be deployed anyway).
The thing you're missing though is that as $NEW_THINGs get more popular and more people try to implement them, they do it badly, spread their bad implementation as gospel, and that actually changes the popular definition into something barely recognizable that becomes $OLD_THING.
A few smart non-graybeards eventually play around with $OLD_THING enough that they actually stumble upon the original definition of it, but don't realize that they are really just rediscovering the original definition. So of course they (re)name their discovery as the next $NEW_THING. Then cue the graybeards.
SOA is really just object oriented programming.
Microservices is really just object oriented programming.
Object oriented programming is really just dependency management.
Sorry but trying to boil everything down to "dependency management" shows a basic lack of understanding of software architecture and is objectively wrong.
SOA & Micro Services are not the same. Services in 90s & 2000 are SOA in a monolith. They were not isolated all the way till databases. Micro Services by definition need to be isolated all the way.
> ...microservices need to be isolated all the way
They may be, but don't have to be. Like many other technology terms that go through a hype cycle, microservices also vary in definition by a large measure. This talk from 2014 by Martin Fowler [1] is about the breadth of definitions as well as contrasts with monoliths.
Microservices by theory are a return to the early theory of SOA, which may frequently have been diverged from in practice, but that probably happens with microservices, too.
> Services in 90s & 2000 are SOA in a monolith.
The theory of SOA is loose coupling through independent services interacting through a shared communication medium. That may not always have been observed strictly in practice, but that's true of microservices, too.
"...So what does this mean? The long and short of it is that microservices are an SOA variation for container and cloud computing..."
I do not agree with this at all. I think one of the biggest problems microservices face is the over-application of pre-existing paradigms. The failures I see are big object models, large frameworks, dependency on shared resources, complexity in the service, and so forth -- all signs of 90s and 00s thinking moving over to microservices.
I agree that everything old is new again. I'd suggest you need to go back further. Good microservices, in my opinion, are not only loosely coupled, any sort of coupling at all is provided by the OS, nothing else. So you'd need to go back to JCL days to find a good analogy for microservices. Trying to stick these other concepts into the microservices bucket looks like a non-starter to me.
I would say much of today’s micro services are mainly s function of current front end fashion of fat clients (mobile and SPAs) consuming JSON-formatted data. Less driven by the SOA qualities of a microservices architecture.
Almost none of that is strictly true. SOA and "microservices" are so close to being synonyms that we might as well say that they are. At worst, they're different "flavors" of the same thing. Like Vanilla ice cream versus Strawberry ice cream.
I read the whole topic looking for enlightenment and insights to understand micro-services. I've been only in small companies (less that 10 years old with a dozen of engineers each) and AFAICT services grew organically without much architecture supervision or even if there was architecture supervision, it was a way to escape the technical debt of previous services. Don't even add to that the technical difficulty to manage multiple services in development or production or in between.
On all the code bases I saw, I never told myself (micro) services are helpful, I will redo that. And if I had to rewrite any of them, I would use a single 'service' that can do everything. IMHO it's more sane to optimize for the developer instead of whatever micro-services try to optimize.
What’s interesting is seeing SOA evolve: first it was service calls everywhere, now it’s increasingly asynchronous publish/subscribe architectures (think AWS SNS and SQS) and service orchestration frameworks (SWF, Step Functions). In my department we realized that 700+ services is untenable and we are now exploring home grown frameworks and SDKs to help manage those levels of complexity.
142 comments
[ 2.7 ms ] story [ 209 ms ] threadAlthough frankly I have to admit when I first heard about the "newest" trend in "static websites" I got a bit flabbergasted.
Most changes over time are evolutionary vs. revolutionary. This often means at an abstract level, new things look very similar to old things because they are. But, there is change (sometimes minor) that makes the new thing new. That minor change can also be the difference between something being used or going away into history.
http://rvirding.blogspot.com/2008/01/virdings-first-rule-of-...
Long an short of it is the ESB is an anti-pattern, always has been and SOA and microservices are the same thing.
It just seemed weird to me to leave that out.
ESB simply add more complexity in the middle that needs yet another team to manage.
Enterprise Service Bus was coined by Progress/Sonic Software for their middleware product around 2002
Service Oriented Architecture as a term to describe an enterprise architecture around web services was only used after SOAP became a thing in 1999
SOA was not that big of a hit because the time wasn't right. Just like MySpace before Facebook, the market wasn't ready. And to say they are the same is to ignore the advances each tech made.
You could push this argument to the extremes and say "well, Microservices are just Processes on different computers talking through a network IPC and this $unknownProject has done that back when Ethernet first started".
Nothing in tech is original and everything is.
Huh? It dominated the 90s, especially in enterprise, in a way that micro-services today don't.
But I do think microservices will become bigger, we'll have to wait and see how k8s pans out.
Containers have nothing to do with it.
- you can keep a service running long after the project that created it has disbanded and/or the runtime and build environment is deprecated on a given O/S
- you don't necessarily need to deep-dive into the business logic of legacy services
- you can replace a service implementation with another one running on another stack without having to adapt service clients
- you can change auth mechanism outside of the service implementations
- you can scale out (CPU-bound) services onto multiple hosts
Mind you, these benefits are not achieved by using microservices/SOA in itself; the contribution of microservices/SOA, however, is that these design goals aren't impossible to achieve because of architected reasons.
"Those who don't understand Unix are condemned to reinvent it, poorly." -- Henry Spencer
Both of these are wrong.
A SOA service of yore would be much lighter (in CPU, if not memory) than a microservice of today. Heck, it had to be -- in the 90s they couldn't afford to run a JS runtime to power their service. It was either C++ or Java.
The protocols were also lighter, then, not now. Except if the author means simpler and easier to work with. Else, CORBA or DCOC are lighter than serializing to JSON and back.
By the way, Server Side JS was a thing in the 90s, though little used. On the other hand, Perl was quite common on websites, despite being much slower than C/C++.
The early days of running a cgi script for every request had awful performance, caused by the overhead of loading and compiling the Perl code before executing it. But there were ways to avoid that by pre-loading the code. One way was to use Mod_Perl on Apache, for example. I wrote a number of services that way, and also wrote my own pure-Perl standalone http server that could pre-load an entire web application.
The other key factor was to use canonical expressions when writing your Perl code. There was always "more than one way to do it", but there were only a few ways that the compiler would recognize and use optimized functions for. Once you learned to take advantage of that, you could get your code running mostly in hand-written C functions. It wasn't as fast as native C because of the way Perl variables were represented, but it could be pretty close.
What made "Service-orientated-architecture" stand out in the 90's and 2000's was standardized communication protocols. It made integrating across teams, and even across companies easier than it's ever been before.
But there were no best practices around service architecture. Sure, methods were decoupled by responsibility (maybe). But when companies thought about their "Web Service" strategy that meant putting up ONE service exposed to the world, and moving on.
Internally, "SOA" meant 1-5 services for a mid-sized shop.
This worked for a while until it didn't. First, the services became monoliths of conflicting responsibilities, then the mobile revolution and big data led to scalability challenges that the era of SOA never had to consider.
THAT is what microservices promised to solve. By being brutal with decomposing and decoupling your concerns you could scale each intelligently and independently. Containers are just an implementation detail, and weren't around yet when Microservices first made their impact.
Of course, plenty of people don't get this. They think of microservices as just another buzzword. So, it's been ruined like "agile" has, and you hear companies about their "Microservice architecture", when really all they've done is migrated their backend to node.js and moved to the cloud.
Shame. It's a good idea a lot of the time (and sometimes it isn't, as with all good ideas).
In my experience it's exactly the opposite that has happened. I see microservices being abused all the time, just because someone read a blog and thought they were cool. The same thing happened with SOA by the way.
We just switched to SVN. The migration was an easy < one day job, a few weeks of pain with some developers grumbling that it was too complex and then everyone was (mostly) happy that we no longer had to hunt down the consultant who had locked a particularly critical file and was now out of the office the rest of the week.
Also, why SVN? That seems an antiquated choice at this point.
Although a couple of years ago I did have a job where they used it, but I quit after 2 months. For some reason a couple of the Devs would constantly check in blank commits. Not sure if that was a product of VSS or because the Devs were awful (part of the many reasons I left so quickly).
It's not quite as bad any more, but still terrible when compared to Git or SVN.
I’m a big fan of vanilla tooling. Complex build, version control, etc gets no love and ends up ten years later as a big mess that no one understands or wants to support.
(I wasn't around in the SOA era)
In reality the definition of both was always fairly fuzzy.
There are a lot of advantages to having a monolith, especially in statically typed languages.
Every extra service you add is costly, it involves setup, build steps that can break, extra things to monitor, more complex development environment, less flexibility, harder to redesign, harder to fix flaws in the original architecture, etc.
Deploying an extra service to a kubernetes cluster, for example, is neither costly nor prone to breaking nor harder to monitor, etc.
I think the best practice nowadays is for a small startup to first write a monolith and then later split it out into microservices when they're ready.
There's always something that can break. More parts, more breakdowns. The services have to communicate, those communication channels are extra points of failure. What happens when one service inadvertently passes along something another one can't decode properly?
Ooops, wasn't expecting unicode! Ooops, wasn't expecting a long int! Oops, the JSON encoder we used in this service spits out stuff the JSON decoder in this one hates! Ooops, we upgraded a package in this service, but not that one! And on, and on.
And when something does inevitably break in the chain, there's the extra cost in debugging of figuring out which bit actually broke.
But here's the dirty secret: if you have a load balancer in front of your application, you already have a way to send requests for feature X to a separate pool of machines if that feature is a hot spot for you currently--and unless your application hits some really degenerate cases your base RAM load Simply Should Not be that high for your application in the first place, making the marginal cost and machine sizing floor less of an issue.
There are benefits to microservices in some environments, chiefly to reflect your org chart when your human protocols are breaking down. They tend to be marginal, and whether they outstrip the increased overhead of dealing with more moving parts (whether in your code or not; I quite like k8s but I still wouldn't want somebody less comfortable with it than I am running it in production, and plenty of those folks do!) is in many ways up in the air.
If your application is composed of single-purpose modules, which are well documented, which have clean public interfaces, which are instrumented for monitoring, which can be scaled by using a load-balancing message router of some kind... and it works well, who cares whether you call it micro-services, Web Services, SOA, or FactoryFactory-Producer/Consumer Ecosystem?
This may be hard to plan out from the beginning, but if all is basically sound, you may be able to reduce that number later. (E.g. when a module is called only by one other module, and there's no requirement for them to scale independently, maybe they should be one module.)
Most development doesn't happen in a "we know exactly where we're going" way.
This all sounds like some perfect, waterfall development cycle, everything's designed on paper in UML and the code is then written to spec without any changes, no requirements change, no interfaces change.
That model of development simply doesn't exist in the real world.
Your profile and blog say you're a sysadmin, do micro-services keep you in a job ;) It's one thing to make a bunch of micro-services as a sysadmin. It's quite another to base an ever growing and mutating production code base around them that actually has no clear path to success and an only vague understanding of future requirements.
We keep development and operations staff tied closely together as a team, and have done since well before "devops" was coined.
I'm on the architecture side so am bearing the brunt of the complexity but the senior and especially mid level developers who have code open 7.8 hours a day say the few things we do that have real microservices are the easiest to debug and work on. The monolithic apps take longer to get less done.
Agreed though that it certainly adds complexity on the admin/arch/ops side.
Microservice: a->async->serialization->network->serialization->scheduling->b
A monolith is extremely simple to traverse with a debugger, a microservice is not. I can make a dozen jumps over/into/out in less time than it would take me to open Fiddler or man tcpdump.
I have heard that some companies have tooling that makes the microservices debugging situation tolerable or sometimes even better than monolith (e.g. writing tests for you by snooping the network layer), but I have yet to play with any. Do you have recommendations?
It may be worthwhile for really complex and "business vice separated" services to be written as micro services because it gives you clear separation of concerns. But, in reality, to often you see a single logical service getting implemented with a lot of (N/W + serialization + management) costs unnecessarily just because it is cool to say that you did micro services.
My thumb rule is : If you don't foresee different teams of devs working on these different services, then you're better off leaving it as a monolith and scaling it as a whole application instead.
Basically having separate teams work on one code base with one deployment cycle can sometimes be really tough. And sometimes you can get a big win by splitting up this service so multiple teams can each have their own service to focus on.
This whole team of one developer supporting 8 microservices seems ludicrious to me.
I tend to like the idea of the 2 pizzas rule from Jeff Bezos.
Having a small team (let say under 10 people) manage a service end to end (from spec to code to production) can be really efficient.
On large code bases/large products, this can help a lot, offsetting the cost of a more complex architecture.
However in a microservice architecture, the APIs between services is critical. It's a contract between services, and it should be clearly stated, you cannot go berserk every other day and change it. You also need to establish clear rules about versioning, deprecation and end of life of said APIs.
(by the way, it would be nice if we had a standard to negotiate API versions, for example stating that a rest API must always have a /version which lists the available versions, if some versions are deprecated and when they will be EOLed).
You won't just jam another getter in that Singleton if that Singleton is in another process.
If you're not running in High Availability, and can tolerate one copy of everything, then monolithic probably makes more sense. Glue it all together and roll out improvements across all the parts at once.
And frankly, for the 95% and probably the 99% case, running a copy of each monolithic application its own auto-scaling group is probably significantly less work and less hair-pulling than the chasing the microservice dragon.
I'll argue that if you're building something, you should probably build it as a monolith, with the exception of some well-defined services (like authentication & authorisation). Once you've got version 1 built, then you know enough about the problem that you're ready to consider breaking stuff off into microservices (this is why auth^2 make sense to break off from the get-go: they're pretty well-understood at this point, so we all should know how to handle them).
Premature microservicement is the root of much evil. Trying to break things up into services before you actually understand the problem will only lead to tears & suffering.
Really depends on how they were written and how they were maintained.
>then the mobile revolution and big data led to scalability challenges that the era of SOA never had to consider.
Can you describe this in more detail?
Yep. This is the first concept he misses.
"They think of microservices as just another buzzword. "
And this is the second one.
What he's describing as SOA doesn't match what I've seen. Even your 1-5 services example is rare. It was mostly taking a buzzword, loading up on some tooling, and cranking WSDLs. You could end up with dozens of those little rascals right out of the starting gate. (It should have been 1-5, agreed. I just never saw it work like that in shops of any significant size)
http://harmful.cat-v.org/software/xml/soap/simple
Maybe SOAP wasn't the entirety of SOA, but back then it was a big part of it.
I agree the term "microservices" has been ruined by those who came in with a very specific agenda. When I was at Timeout.com, in 2013, I used the phrase "an architecture of small apps":
http://www.smashcompany.com/technology/an-architecture-of-sm...
It was a few months later, when Martin Fowler wrote his essay about it, that I first heard the term "microservices". When I read his essay, I assumed he was talking about the same thing that I was talking about. But I later found that the people promoting microservices defined it entirely in terms of HTTP, so I've gone back to talking about "an architecture of small apps" when I am talking about small apps that talk to each other with something other than HTTP, such as this kind of architecture:
http://matthiasnehlsen.com/blog/2014/11/07/Building-Systems-...
In that example, Mattias Nehlsen builds a system where the communication is via Redis.
Well, SOAP and Wiz-Dull.
Many of the robotics platforms in common use (e.g., ROS) consist of independent, single-responsibility services that communicate over a shared message bus using a protocol that's not HTTP.
But for enterprise apps, it's 2018. Other protocols? What other protocols? You should be using HTTP/2. It can MUX multiple streams over the same connection. FINALLY! At long last! We're no longer limited to a single concurrent stream!
REST lost that.
I think GraphQL has it back again. Any GraphQL people want to comment?
If you share something, you create a dependency; and that dependency needs to be managed both technically and domain-wise. Group X now has to depend on Group Y's decisions if they both use/share service Z. If X and Y don't buy off on a common dependency on Z, they get frustrated and blame devs and the other service users.
Stored procedures and SQL allow data/service sharing with less code anyhow.
That's the main property that sets microservices apart from mere SOAs: Microservices reduce the risk of change.
With properly architected microservices a single deployment is unlikely to impair an application as a whole whereas with traditional monolithic deployments a single change can severely affect the functionality and operations of an application.
This is also why you probably don't need microservices if you don't need that kind of fine-grained control and risk mitigation.
Microservices lend themselves to being used in conjunction with domain-driven design. A single microservice should only deal with one domain.
If you have a service that covers cross-cutting concerns such as orders that service is not a microservice. In order to improve the design in your case each microservice could access and manipulate order data according to its scope.
That way, if one of the services fails only the small part of the order functionality implemented by that service will fail while the remaining order features will remain intact (to some extent; steps in a linear order process that depend on each other of course will still fail without successful outcomes from previous steps).
Then they realized that they needed some type of way to validate the JSON so they introduced JSON schema.
Then they realized that they needed some method to make a REST Api discoverable so they added the Swagger protocol (OpenAI). Now we are right back to being as complicated as SOAP.
WSDL is the ultimate api :) it would make my day to see the wsdl comeback of 2018
But since they provided WSDL before, their "REST" APIs now come with WADL files that offer zero value to anyone. Swagger would be a massive step up from that.
At least with current JSON and REST API I stand a good chance of experimenting using just an editor and curl - good luck trying that with SOAP.
REST is getting there.
My experiences with SOAP were that 99% of the time it was fine - but those other 1% seems to take the other 99% of my time... ;-)
In many cases, I'll simply formulate xml requests myself and use a generic http request client because the implementation ended up being simpler & easier to maintain than all the confusing hacks I had to do to make zeep or suds (the most popular SOAP clients implemented in python) function properly.
[1] - https://github.com/modern-project/modern-ruby
Which is silly, because REST is discoverable by definition, since you should be using hypermedia. And if your API is not discoverable, that's a good sign your service is not actually REST.
That's pretty much what I said! Web frameworks don't help you at all in this regard, which is why REST is unncessarily difficult to implement. Most web frameworks give you just enough for HTTP-based RPC.
As for the nightmare of REST, the real question is whether this difficult is intrinsic, or merely a product of immature tooling and poor understanding. No web framework I've seen, apart from the Waterken server, actually helps you achieve REST. They're largely all HTTP-based RPC frameworks.
This is not necessary in many languages that centre around domain objects. The validation is done automatically when the JSON gets mapped back to the object. It's far better than JSON schema anyway since you can enforce whatever custom validation you like.
> Then they realized that they needed some method to make a REST Api discoverable
But it doesn't have to be discoverable. And if you do want it then you could write the documentation by hand or expose a simple REST index endpoint.
> Now we are right back to being as complicated as SOAP.
You chose to make it complicated though.
Like all documentation written by hand, it rapidly gets out of date with the actual implementation if you're not careful.
Also tools can automatically create proxy classes from WSDL's and now Swagger endpoints.
IMO, the value of JSON Schema isn't that you have no validation that exists outside of JSON Schema, the value is that you can front load a TON of the validation into it, and share that vocabulary between your services. If you publish a JSON Schema about your endpoint, I can read that or feed it into tooling to produce a form to consume it. As someone working on the Elixir JSON Schema lib currently, the places where we COULDN'T express the validation with JSON Schema were often more interesting.
For one, Swagger is for humans not machines to consume. At it's core it is a way to standardize documentation that can be easily displayed to humans. WSDL is not only used by humans but also by client and server apps.
REST APIs themselves if done right though hypermedia can be self-documenting and require no download of large spec files to use it.
SOAP also suppers from a number of quirks. One for example, that order often matters. Yes you can write a WSDL that doesn't require order but nothing is worse than having a bug because <price> has to appear after <token> but before <expirationMonth> and any other order throws anon-specific validation error. Yes, I know there are tools that can help with this but I have never felt compelled to download a tool for REST. Curl does just fine for me. And to top it off a lot of the best SOAP tools are Windows only.
What I think you really should be pointing at is RPC (of which SOAP is a terrible no good implementation of). RPC indeed offers some benefits over REST and can even be simpler. At the expense of losing the discover and hypermedia aspects of REST if done right.
For one, Swagger is for humans not machines to consume. At it's core it is a way to standardize documentation that can be easily displayed to humans.
c# - https://github.com/Azure/autorest
Java - https://github.com/swagger-api/swagger-parser/blob/master/RE...
And basically everything else.
https://github.com/swagger-api/swagger-codegen/blob/master/R...
I am not aware of any code generators that read the swagger file every single API call and use that as if it were the code.
Re: statefulness ("order matters"): there are intrinsical stateful processes that aren't represented adequately by a REST service. Think of a basic "Purchase Order" resource representing purchased items on an e-commerce website. You generally can't add and delete line items freely once you have placed your order, but you may cancel individual items (unless they're already physically on your way), you may RMA items, etc. Representing this as a flat "Purchase Order" resource with GET, POST, DELETE, PUT primitives is useless as it isn't telling your web shop front-end developers what actions are available at any one time.
As a consultant who has talked at conferences on APIs and is brought in to explicitly fix these kind of issues I have seen a lot of this too but correct REST APIs do exist. I know because my team builds them.
You're experience is anecdotal and as is mine. But just because a lot of people assume REST is something it isn't doesn't mean REST is inferior to SOAP.
> Re: statefulness ("order matters")
My order matters comment had nothing to do with statefulness. I mean in the XML. In SOP this might work:
But this will fail: Why the heck does the order of keys in my object make the API fail!?Granted that is not a limitation of SOAP but more XML schemas in general. Json is more likely to be order of values agnostic.
But to play devil's advocate I can see the argument where if you are using SOAP to make multiple RPC calls in the same envelope than order is important.
Moreover, you could design your XML Schema such that both variants are accepted (using XSD "all-groups"), though typically record-like data structures are indeed described using XSD sequences which would cause the issue you mentioned. Again I agree XML/XSD isn't a great payload/schema mechanism for data structures such as maps; after all SGML/XML was invented for encoding text. It's being used because it was the first mechanism on the scene, and it continues to be used because it's still much more capable and mature/established in terms of data validation than anything JSON, especially for protocols used in communication with external partners/banks/authorities.
Except you can ignore the schema and swagger and have it just be simple, if that's all you need.
The biggest problem with SOAP is that it is too easy for the service provider to just dump their internal data structures without any thought to how it should be serialized for people using the data. XML is Particularly nasty in this regard, because it doesn't map well to the data primitives of most dynamic languages.
That said, I usually just treat SOAP APIs as if they were REST. By writing a class, and some wrapper methods for the parts I need. Instead of trying to consume the WSDL and use it the way they envisioned I would. Then again, if Python had better SOAP support I probably wouldn't even do that.
I know of some architectures where the API Gateway has a lot of logic put into it... And you have to go through it to reach any of the microservices in the network.
Isn't this a single point of failure and similar to an ESB?
(I am basing myself on just a superficial interest for MS so I might be wrong - here is what I am talking about: http://blog.christianposta.com/microservices/the-hardest-par...)
If this is correct, than no, Microservices are really not "just like SOA".
(If I am wrong, please correct me - I am here to learn).
I am in a team now that is just started using to use microservices. And I am noticing that we are stuck on a lot of DB changes we want to make because there are various applications accessing the same tables. So now we are getting into the habit of constraining back-end data stores (tables and collections) to just one microservice because we feel the pain of not doing that now.
There's no "data layer" that is managed outside the microservice. It _is_ a microservice.
That's one of the reasons Google uses a monorepo: any microservice can easily import a database instance and start using it as part of it's structure.
That said, there is nothing "micro" about most microservices I've come across over the years. They're "miniservices" at best.
We have broken off two of our monolith’s components into a microservice and for now it’s working great - but there are just some things I don’t think make sense to decouple and isolate. I’m afraid I’d get in an integration hell. Has anyone had such an issue?
When that will happen in your case is highly dependent on what you're doing, but it will happen. Eventually you will run into situations where implementing a single small feature requires changes to 3 or more systems, plus deploying all of those systems to all of your environments, code reviews for each service, etc. The overhead explodes as you break more things apart.
In the end SOA just had too much overhead, rules, and protocols. I remember seeing what these guys were doing with SOA and I would just ignore it and create REST services with JSON. We were able to get stuff in production. Eventually I found out that what we were doing was part of a movement called microservices and then started following and learning what other folks were doing.
And then people running around screaming SOA, MICROSERVICES, JSON and I like "You don't say? o_O?"
EDIT: spelling
As other have pointed out he's ignoring key failings of SOA and critical differences between SOA and microservices.
Overall I like the article but I think he's misstating a few key things.
And maybe a nitpick but:
> It lends itself to the Continuous Integration/Continuous Deployment (CI/CD) model of developing large, complex applications.
Really? Because I thought CICD was the standard way to deploy any application more complex than a spike or proof of concept (which probably doesn't need to be deployed anyway).
Can you describe the fundamental differences?
A few smart non-graybeards eventually play around with $OLD_THING enough that they actually stumble upon the original definition of it, but don't realize that they are really just rediscovering the original definition. So of course they (re)name their discovery as the next $NEW_THING. Then cue the graybeards.
SOA is really just object oriented programming.
Microservices is really just object oriented programming.
Object oriented programming is really just dependency management.
They may be, but don't have to be. Like many other technology terms that go through a hype cycle, microservices also vary in definition by a large measure. This talk from 2014 by Martin Fowler [1] is about the breadth of definitions as well as contrasts with monoliths.
[1]: https://www.youtube.com/watch?v=wgdBVIX9ifA
Microservices by theory are a return to the early theory of SOA, which may frequently have been diverged from in practice, but that probably happens with microservices, too.
> Services in 90s & 2000 are SOA in a monolith.
The theory of SOA is loose coupling through independent services interacting through a shared communication medium. That may not always have been observed strictly in practice, but that's true of microservices, too.
I do not agree with this at all. I think one of the biggest problems microservices face is the over-application of pre-existing paradigms. The failures I see are big object models, large frameworks, dependency on shared resources, complexity in the service, and so forth -- all signs of 90s and 00s thinking moving over to microservices.
I agree that everything old is new again. I'd suggest you need to go back further. Good microservices, in my opinion, are not only loosely coupled, any sort of coupling at all is provided by the OS, nothing else. So you'd need to go back to JCL days to find a good analogy for microservices. Trying to stick these other concepts into the microservices bucket looks like a non-starter to me.
On all the code bases I saw, I never told myself (micro) services are helpful, I will redo that. And if I had to rewrite any of them, I would use a single 'service' that can do everything. IMHO it's more sane to optimize for the developer instead of whatever micro-services try to optimize.