This seems like the idea of modularizing code, and using specific function sighatures for data exchange as an API is being re-invented by people using AI. Aren't we already mostly doing things this way, albeit via submodules in a monolith, due to the cognitive ctrain it puts on humans to understand the whole thing at any given time?
What matters for LLMs is what matters for humans, which usually means DX. Most Microservice setups are extremely hard to debug across service boundaries, so I think in the future, we'll see more architectural decisions that make sense for LLMs to work with. Which will probably mean modular monoliths or something like that.
That's an argument for components with well-defined contracts on their interfaces, but making them microservices just complicates debugging for the model.
It's also unclear whether tight coupling is actually a problem when you can refactor this fast.
A typical rant (composed from memory) goes something like this:
> "These AI types are all delusional. My job is secure. Sure your model can one-shot a small program in green field in 5 minutes with zero debugging. But make it a little larger and it starts to forget features, introduces more bugs than you can fix, and forget letting it loose on large legacy codebases"
What if that's not a diagnosis? What if we see that as an opportunity? O:-)
I'm not saying it needs to be microservices, but say you can constrain the blast radius of an AI going oops (compaction is a famous oops-surface, for instance); and say you can split the work up into self-contained blocks where you can test your i/o and side effects thoroughly...
... well, that's going to be interesting, isn't it?
Programming has always supposed to be about that: Structured programming, functions (preferably side-effect-less for this argument), classes&objects, other forms of modularization including -ok sure- microservices. I'm not sold on exactly the latter because it feels a bit too heavy for me. But ... something like?
I don't think LLMs push us to use microservices as much as Borgers says they do. They don't avoid the problems microservices have always faced, and encapsulation is mostly independent from whether a boundary is a service-to-service boundary:
I think this is a promise, probably also for spec driven development. You write the spec, the whole thing can be reimplemented in rust tomorrow. Make small modules or libraries.
One colleague describes monolith vs microservices as "the grass is greener of the other side".
In the end, having microservices is that that the release process becomes much harder. Every feature spans 3 services at least, with possible incompatibility between some of their versions. Precisely the work you cannot easily automate with LLMs.
Like almost all of these articles, there's really nothing AI- or LLM-specific here at all. Modularization, microservices, monorepos etc have all been used in the past to help scale up software development for huge teams and complex systems.
The only new thing is that small teams using these new tools will run into problems that previously only affected much larger teams. The cadence is faster, sometimes a lot faster, but the architectural problems and solutions are the same.
It seems to me that existing good practices continue to work well. I haven't seen any radically new approaches to software design and development that only work with LLMs and wouldn't work without them. Are there any?
I've seen a few suggestions of using LLMs directly as the app logic, rather than using LLMs to write the code, but that doesn't seem scalable, at least not at current LLM prices, so I'd say it's unproven at best. And it's not really a new idea either; it's always been a classic startup trick to do some stuff manually until you have both the time and the necessity to automate it.
Maybe just a refinement of "LLM as app logic" but "LLM as emergent workflow" seems powerful.
For example, a customer service playbook may have certain ways to handle different user problems, but that breaks down as soon as there are complications or compound issues. But an LLM with the ability to address individual concerns may be able to synthesize solution given fundamental constraints. It's kind of lile building mathematics from axioms.
It seems entirely logical that if an llm allows each ic to do the work of a 2-3 person team (debatable, but assume it’s true for the sake of argument) then you’ve effectively just added a layer to the org chart, meaning any tool that was effective for the next scale up of org becomes a requirement for managing a smaller team.
What should give anyone pause about this notion is that historically, by far the most effective teams have been small teams of experts focusing on their key competencies and points of comparative advantage. Large organizations tend to be slower, more bureaucratic and less effective at executing because of the added weight of communication and disconnect between execution and intent.
If you want to be effective with llms, it seems like there are a lot of lessons to learn about what makes human teams effective before we turn ourselves into an industry filled with clueless middle managers.
One thing I find interesting is how GraphQL has evolved from an API technology for API consumers with "different needs" to an API technology for agents. What helped organizations scale GraphQL across multiple teams is Federation, a way to split one supergraph into multiple subgraphs. So, what works well to scale teams actually works equally well for agents. The core value you can get from Federation is a "coordination" layer that is deterministic. Now, what's interesting is that you can scale agentic software development pretty well when you have a deterministic layer where everyone involved can agree. I wrote more about this on our blog if anyone is interested: https://wundergraph.com/blog/graphql-api-layer-for-ai-agents
I think of "Federation" as aggregating content from disparate in-situ sources into a common index under a consistent schema. I agree that schema is good, am always tempted by big schema, and seem to always settle for task-specific schema with mappings between schemas strictly as necessary. LLM's seem nice for schema to schema mapping and supernice for binding entities from unstructured sources into schema. But overall I feel it is pointless to talk about structure without talking about hydration - about binding and mapping. Do you see that as "solved"?
> It seems to me that existing good practices continue to work well. I haven't seen any radically new approaches to software design and development that only work with LLMs and wouldn't work without them.
I've been thinking about it lately and I think you are right. LLMs haven't changed what is 'good software'. But they changed some proxies I used to have for what is 'good software'.
In the past I've always loved projects that had good documentation, and many times I've used this metric to select a project/library to use. But LLMs transformed something that was (IMHO) a good indicator for "care"/"software quality" into something that is becoming irrelevant (see Goodhart's law).
If this is an issue you must be starting with a blank slate and not giving any direction. If you collaborate on the design you're not going to find it accidentally used microservices.
Makes me wonder if the LLM, out of the box would go one way or another. I am big fan of services, without the micro part and always guide my model in that direction.
I had the same thought as I'm working with LLMs. Then I reached the same conclusion as I did without LLMs: you can get most of the benefits without many of the drawbacks using well-bounded 'modules' within a monolith. The article doesn't distinguish these:
> When coding in a monolith, you have to worry about implicit coupling. The order in which you do things, or the name of a cache key, might be implicitly relied-upon by another part of the monolith. It’s a lot easier to cross boundaries and subtly entangle parts of the application. Of course, you might not do such unmaintainable things, but your coworkers might not be so pious.
What it's saying could also apply to a monorepo with distinctly deployed artifacts. The reason many don't think about clear boundaries between modules is that popular interpreted languages don't support them. Using the Java ecosystem as an example, each module can be a separate .jar containing one or more package namespaces. These must have an explicit X uses Y declaration.
The problem I see isn't so much that misuse it's easy (though that's a part of it), it's that there's to clear indication that boundaries are being crossed since calling from one package to another is normal, and the fact that some packages belong to other modules isn't always obvious.
I don't see a direct causal link here - architecture choices are still mostly about suitability and the specific needs of your app. That said, if you do opt for microservices, modern Edge runtimes like Cloudflare Workers and lightweight frameworks like Hono have changed the game. They’ve made deploying and maintaining microservices significantly easier and more cost-effective, while providing incredible performance.
I have built both monoliths and microservices over thirty-plus years, and the honest answer is that LLMs make it easier to generate more code, not better architecture. The hard part of microservices was never writing the services. It was debugging the network calls between them at 3am when one service silently started returning empty responses. LLMs do not help with that. They just help you create more services to debug.
28 comments
[ 1.9 ms ] story [ 48.2 ms ] threadIn fact, my argument is that there will be more monolith applications due to AI coding assistants, not less.
It's also unclear whether tight coupling is actually a problem when you can refactor this fast.
> "These AI types are all delusional. My job is secure. Sure your model can one-shot a small program in green field in 5 minutes with zero debugging. But make it a little larger and it starts to forget features, introduces more bugs than you can fix, and forget letting it loose on large legacy codebases"
What if that's not a diagnosis? What if we see that as an opportunity? O:-)
I'm not saying it needs to be microservices, but say you can constrain the blast radius of an AI going oops (compaction is a famous oops-surface, for instance); and say you can split the work up into self-contained blocks where you can test your i/o and side effects thoroughly...
... well, that's going to be interesting, isn't it?
Programming has always supposed to be about that: Structured programming, functions (preferably side-effect-less for this argument), classes&objects, other forms of modularization including -ok sure- microservices. I'm not sold on exactly the latter because it feels a bit too heavy for me. But ... something like?
https://www.natemeyvis.com/agentic-coding-and-microservices/
One colleague describes monolith vs microservices as "the grass is greener of the other side".
In the end, having microservices is that that the release process becomes much harder. Every feature spans 3 services at least, with possible incompatibility between some of their versions. Precisely the work you cannot easily automate with LLMs.
The only new thing is that small teams using these new tools will run into problems that previously only affected much larger teams. The cadence is faster, sometimes a lot faster, but the architectural problems and solutions are the same.
It seems to me that existing good practices continue to work well. I haven't seen any radically new approaches to software design and development that only work with LLMs and wouldn't work without them. Are there any?
I've seen a few suggestions of using LLMs directly as the app logic, rather than using LLMs to write the code, but that doesn't seem scalable, at least not at current LLM prices, so I'd say it's unproven at best. And it's not really a new idea either; it's always been a classic startup trick to do some stuff manually until you have both the time and the necessity to automate it.
For example, a customer service playbook may have certain ways to handle different user problems, but that breaks down as soon as there are complications or compound issues. But an LLM with the ability to address individual concerns may be able to synthesize solution given fundamental constraints. It's kind of lile building mathematics from axioms.
What should give anyone pause about this notion is that historically, by far the most effective teams have been small teams of experts focusing on their key competencies and points of comparative advantage. Large organizations tend to be slower, more bureaucratic and less effective at executing because of the added weight of communication and disconnect between execution and intent.
If you want to be effective with llms, it seems like there are a lot of lessons to learn about what makes human teams effective before we turn ourselves into an industry filled with clueless middle managers.
I've been thinking about it lately and I think you are right. LLMs haven't changed what is 'good software'. But they changed some proxies I used to have for what is 'good software'.
In the past I've always loved projects that had good documentation, and many times I've used this metric to select a project/library to use. But LLMs transformed something that was (IMHO) a good indicator for "care"/"software quality" into something that is becoming irrelevant (see Goodhart's law).
Only if you’re a sufficiently bad programmer to not tell it the architecture it must comply to that hopefully you have the skills to define.
> When coding in a monolith, you have to worry about implicit coupling. The order in which you do things, or the name of a cache key, might be implicitly relied-upon by another part of the monolith. It’s a lot easier to cross boundaries and subtly entangle parts of the application. Of course, you might not do such unmaintainable things, but your coworkers might not be so pious.
What it's saying could also apply to a monorepo with distinctly deployed artifacts. The reason many don't think about clear boundaries between modules is that popular interpreted languages don't support them. Using the Java ecosystem as an example, each module can be a separate .jar containing one or more package namespaces. These must have an explicit X uses Y declaration.
The problem I see isn't so much that misuse it's easy (though that's a part of it), it's that there's to clear indication that boundaries are being crossed since calling from one package to another is normal, and the fact that some packages belong to other modules isn't always obvious.
They may be useful for you, on the other hand atomising things brings overhead and maintenance burden and is often not worth it