This starts out strong and then trails off without a real conclusion or opinion on glue systems. A bad build system or lack of maintenance can happen to any project.
Not sure if this is exactly what the article refers to, but at one of my customers I helped create an app literally called Glue. It has things like a task scheduler for scheduling http posts to developers apps, or monitoring email boxes and posting to web hooks when email comes in. It _does_ need maintenance but I think it beats putting that logic in each app.
There seem to be a lot of components like this that need to be developed.
For example, there are plenty of tools to create ML model endpoints (send data, receive prediction) but many times, those models are not customer/user facing. They will be hit by some other program/system (often on a regularly timed schedule).
One solution is to embed the model in the application that needs the predictions, but this can get hairy when it comes to updates/maintanence/etc.
Over the next few years, we will see some generalized "model invocation" components that integrate with common data sources for these use cases.
I'd definitely say so. Combined with Docker, its job is to "tame" / coordinate / connect extremely heterogeneous software, much of which wasn't designed to run on clusters.
I think of shell as the language for glue (and we need a better one). and I explored this relationship in a followup to a popular post [1]:
Seems like people think so, but I would say no, it's just a platform that runs VMs. It doesn't really care about the data running through services other than making sure it makes it to the destination. That being said, it's quickly becoming easier to plug and play traditional message busses or queuing systems by deploying a container to it. So it helps simplify the addition of some kind of glue system.
The article makes the interesting claim that Clojure, Elixir and similar languages are good for "glue" but doesn't make any effort to develop the argument.
For a long-time we've heard the opinion that complex systems are best written in a combination of a "systems" language (say C++ or Java) and a "scripting" language (say Tcl, Lua or Clojure.) That's how video games are written, for instance.
Clojure and Elixir have the particular angle of being suitable for concurrency, which more conventional scripting languages (e.g. Python, Ruby, PHP, ...) fall down at thanks to the various global interpreter locks.
Does this really matter for a glue language in a distributed system? Async for non-blocking IO and then just run multiple processes. Allows for things like killing a process and letting the OS cleanup. For example, in Java you're not supposed to just kill a thread.
I've written a lot of "glue" in Python using asyncio, websockets, AQMP and similar things for my IoT system at home. The coding is fun and the performance seems excellent, but that's because the workload is low. If I was at the point where I needed more than one core worth of performance I wouldn't be so happy.
One case where I am a little frustrated with Python now is a "batch job" that processes and resizes images. There are numerous half-baked ways to farm out the work to separate processes. These would be "good enough" but they are still half baked, and the frameworks that give a little structure to the process have one thing in common: they don't run on Windows.
In a pure "glue" case there is also an Amdahl's law kind of situation: it's not crazy to have a 32-core or more system today and a single-threaded "glue" system that dispatches tasks will have a hard time keeping those cores busy if it is spending even 2% of the CPU effort that it takes to do the real work.
I've tried repeatedly over the last 25 years, in Python, with a lot of success for large amounts of computation. Architecture and algorithms matter so much more than limitations like the GIL, slow function calls, etc. And in my experience, Python gives me the time and mental bandwidth to focus on architecture and algorithms. It's hard work, but in my experience it's not as hard as getting the same level of success without a very high level language gluing things together.
One memorable situation was sitting in a sales meeting in ~2003, showing the speed of one such system. We let the potential client customize some input data, then ran a computation (the point was to show that something that used to be a batch process could now be interactive). It took ~30 seconds on our laptop running on battery power. On the spot, their CTO accused us of lying because it took days of dedicated mainframe time for them to run the same computation. Our sales person later asked if we could introduce configurable delays that we'd only turn on during sales presentations so that it would be easier to believe and less threatening. Our original implementation was 1000s of lines of Python and would take weeks to run. Over time we optimized that down to about half a day of run time. Then we figured out that the thing taking all the time was equivalent to finding shortest paths in graphs over and over again. So we found an open source C extension with a nicely optimized implementation of Dijkstra's algorithm and our Python code reduced to 100s of lines and was literally unbelievably fast.
Today I'm working on a simulation system for use by people who are experts in game theory. The thing being simulated is an algorithm in an online system (that almost everyone reading this in the US has used) that does server side optimization of the selection and placement of content during web / mobile app page loads. We want to share code between online and simulation to avoid semantic differences. The online system is in Python. Python is what our math folks know. The algorithm currently runs tens of millions of times per day online with billions of inputs, all on a small cluster of machines. Online we need low latency, O(1 ms) per run. Offline we need as much computational bandwidth as we can get to keep math folks going - improvements to this algorithm can mean big revenue wins. We haven't shipped yet, but the current version replays hundreds of millions of runs (10s of billions of inputs) per minute in a large Spark cluster (developers only ever see Python, not the PySpark API). The online backend runs the exact same Python code in O(100 µs) with a Python backend (only one non-builtin extension is used online - Pyrsistent, which gives us efficient immutable data structures - the online backend uses FP concepts to avoid complexity).
In my experience, running up against things like the GIL for doing large amounts of computation means I'm stuck in a local optimum for speed. Backing up and thinking about my architecture / algorithms gives me a real chance of making improvements that show that the GIL wasn't my core issue. Eliminating the GIL might get me an order of magnitude speed improvement (often at the expense of more complicated code). Finding better architecture / algorithms often gets me several orders of magnitude of speed improvement and simplifies my code at the same time. A very high level language that is good at gluing things together gives me more time / options for doing that.
I've written high performance code in Java and often been able to quickly recruit many cores just by adding an Executor. (as opposed to "try to recruit many cores and recruit a few cores" with parallelStream(), Scala's actors and the many snake oil parallelism libraries -- there are more of those than there are cores on a big machine.)
Your example doesn't makes sense to me. If there was no GIL then you'd still need a dispatch thread. And if that thread took 2% of the same compute then you're out of luck. The lack of a GIL doesn't magically distribute compute across many CPUs. You could off course make a minimal dispatch thread and then offload the 2% to other threads but you can do that with a minimal dispatch process as well.
Don't know about Elixir, but Clojure has a strong focus on dealing with external system the following way:
- convert the data output/need as input in core data structures
- Don't use classes or custom types to encapsulate data and make it accessible using a custom interface
- instead manipulate the data structures instead using the large standard library.
- convert the data back into the required format for storage/output/etc.
Very ETL-like, that is needed as different systems that need to be glued together may be written in different languages and most likely will export serialised data or write in a database. Certainly binary formats are rare nowadays.
Abstractions like the one for sequences allow reuse of the same function for different data structures (you don't have to convert things into lists for processing). It's possible to have other, non-core data structures implement the same abstractions, so that the existing library can be used.
ORMs are not desirable with that approach, for example. The query result tables are usually converted into things like Clojure hash-maps instead being created into unique objects with different ways of accessing the encapsulated data.
Then you have functional programming, which while nice by itself also allows using your cores efficiently. Less of a need for additional "infrastructure" glue that way. Asynchrony is also provided using libraries like core.async or agents.
Been thinking about glue systems a bunch - the platform I'm working on (Inngest [0]) solves this kind of problem.
I think "glue systems" can be generalized by saying: systems that consolidate events, and react to events in real-time. Most everything that happens in your system is an event, and things need to react to this. For example:
- A glue system will almost certainly plumb stripe events info a billing system, plus other systems for things like failed payments.
This system shouldn't go down (HA). It should be able to react to changes easily - as biz ops change frequently. It should be auditable, easy to debug, and easy to deploy.
The first pass at this in startups is usually... just doing stuff in the API request, or building webhook endpoints that manage this for you within the API. Then, you might add Sidekiq, or SQS/Lambda. Then, you might build out event-driven architecture using SQS/Kafka. It's kind of a pain, and still it's not easy to grok, debug, or architect.
Inngest handles all of this for you, though. It consolidates events from every system (internal and external, via APIs, webhooks, oauth, etc.) and then allows you to run DAG-based workflows in real-time whenever events are received. With full logging, debugging (step-over debugging), retries, user-auditing, changelogs, version control, etc.
I think that event-driven systems and glue-based systems overall haven't had much love in the developer UX, but I'm hopeful we can change that. If you're interested in using us or working with us on building the platform, ping me :)
So many words to just say ESB. I know it's not fancy and falling out of fashion, but what you describe here is what different ESB/MIddleware systems have been doing for ages.
Inngest looks very cool - is there any chance of this being an "open-source SaaS" / open-core type thing like, for example, TimescaleDB or Temporal? Would love to contribute to something like this, but not as paid employment.
Yeah, soon! I need to redo some of the DAG-based workflow logic to rip out our internal scheduler (we have two: internal and nomad via firecracker/gvisor).
Once that's done and I've refactored some stuff I'm planning on opening up the core runner. Our FoundationDB persistence we're working on may likely remain closed - we will provide an interface which allows you to persist stuff and a psql adapter :)
I don't understand what he means with the build systems comment? What does that have to do with Glue systems? I could speculate that he wishes the build systems accounted for all the supposedly siloed dependent systems in some way but it's not clear what exactly his complaint here is.
1) "Complex systems should be designed like video games"
2) "Complex systems should be designed around a huge event bus SaaS"
Video games/engines are architected around responsiveness (low latency) & content creation.
Sending every little event to a remote SaaS for processing is sure to add tremendous latency.
To be fair, many video games due make heavy use of event processing/message passing. But the architecture of a video game event system & SaaS event bus are quite different for performance, security, reliability reasons.
Complex distributed systems are, well, complex, and can't be reduced to a simple formula or architecture. Better to spend time accurately understanding the problem that is being solved.
31 comments
[ 3.1 ms ] story [ 77.6 ms ] threadSounds like modern web development.
For example, there are plenty of tools to create ML model endpoints (send data, receive prediction) but many times, those models are not customer/user facing. They will be hit by some other program/system (often on a regularly timed schedule).
One solution is to embed the model in the application that needs the predictions, but this can get hairy when it comes to updates/maintanence/etc.
Over the next few years, we will see some generalized "model invocation" components that integrate with common data sources for these use cases.
You can see an inkling of this with the seldon-core kafka integration: https://docs.seldon.io/projects/seldon-core/en/stable/stream...
I think a glue system is meant to connect disparate pieces that otherwise could not interact with each other.
I think of shell as the language for glue (and we need a better one). and I explored this relationship in a followup to a popular post [1]:
http://www.oilshell.org/blog/2021/07/cloud-review.html
[1] https://news.ycombinator.com/item?id=27903720
For a long-time we've heard the opinion that complex systems are best written in a combination of a "systems" language (say C++ or Java) and a "scripting" language (say Tcl, Lua or Clojure.) That's how video games are written, for instance.
Clojure and Elixir have the particular angle of being suitable for concurrency, which more conventional scripting languages (e.g. Python, Ruby, PHP, ...) fall down at thanks to the various global interpreter locks.
Does this really matter for a glue language in a distributed system? Async for non-blocking IO and then just run multiple processes. Allows for things like killing a process and letting the OS cleanup. For example, in Java you're not supposed to just kill a thread.
I've written a lot of "glue" in Python using asyncio, websockets, AQMP and similar things for my IoT system at home. The coding is fun and the performance seems excellent, but that's because the workload is low. If I was at the point where I needed more than one core worth of performance I wouldn't be so happy.
One case where I am a little frustrated with Python now is a "batch job" that processes and resizes images. There are numerous half-baked ways to farm out the work to separate processes. These would be "good enough" but they are still half baked, and the frameworks that give a little structure to the process have one thing in common: they don't run on Windows.
In a pure "glue" case there is also an Amdahl's law kind of situation: it's not crazy to have a 32-core or more system today and a single-threaded "glue" system that dispatches tasks will have a hard time keeping those cores busy if it is spending even 2% of the CPU effort that it takes to do the real work.
One memorable situation was sitting in a sales meeting in ~2003, showing the speed of one such system. We let the potential client customize some input data, then ran a computation (the point was to show that something that used to be a batch process could now be interactive). It took ~30 seconds on our laptop running on battery power. On the spot, their CTO accused us of lying because it took days of dedicated mainframe time for them to run the same computation. Our sales person later asked if we could introduce configurable delays that we'd only turn on during sales presentations so that it would be easier to believe and less threatening. Our original implementation was 1000s of lines of Python and would take weeks to run. Over time we optimized that down to about half a day of run time. Then we figured out that the thing taking all the time was equivalent to finding shortest paths in graphs over and over again. So we found an open source C extension with a nicely optimized implementation of Dijkstra's algorithm and our Python code reduced to 100s of lines and was literally unbelievably fast.
Today I'm working on a simulation system for use by people who are experts in game theory. The thing being simulated is an algorithm in an online system (that almost everyone reading this in the US has used) that does server side optimization of the selection and placement of content during web / mobile app page loads. We want to share code between online and simulation to avoid semantic differences. The online system is in Python. Python is what our math folks know. The algorithm currently runs tens of millions of times per day online with billions of inputs, all on a small cluster of machines. Online we need low latency, O(1 ms) per run. Offline we need as much computational bandwidth as we can get to keep math folks going - improvements to this algorithm can mean big revenue wins. We haven't shipped yet, but the current version replays hundreds of millions of runs (10s of billions of inputs) per minute in a large Spark cluster (developers only ever see Python, not the PySpark API). The online backend runs the exact same Python code in O(100 µs) with a Python backend (only one non-builtin extension is used online - Pyrsistent, which gives us efficient immutable data structures - the online backend uses FP concepts to avoid complexity).
In my experience, running up against things like the GIL for doing large amounts of computation means I'm stuck in a local optimum for speed. Backing up and thinking about my architecture / algorithms gives me a real chance of making improvements that show that the GIL wasn't my core issue. Eliminating the GIL might get me an order of magnitude speed improvement (often at the expense of more complicated code). Finding better architecture / algorithms often gets me several orders of magnitude of speed improvement and simplifies my code at the same time. A very high level language that is good at gluing things together gives me more time / options for doing that.
- convert the data output/need as input in core data structures - Don't use classes or custom types to encapsulate data and make it accessible using a custom interface - instead manipulate the data structures instead using the large standard library. - convert the data back into the required format for storage/output/etc.
Very ETL-like, that is needed as different systems that need to be glued together may be written in different languages and most likely will export serialised data or write in a database. Certainly binary formats are rare nowadays.
Abstractions like the one for sequences allow reuse of the same function for different data structures (you don't have to convert things into lists for processing). It's possible to have other, non-core data structures implement the same abstractions, so that the existing library can be used.
ORMs are not desirable with that approach, for example. The query result tables are usually converted into things like Clojure hash-maps instead being created into unique objects with different ways of accessing the encapsulated data.
Then you have functional programming, which while nice by itself also allows using your cores efficiently. Less of a need for additional "infrastructure" glue that way. Asynchrony is also provided using libraries like core.async or agents.
For what it's worth, each instance of the new "Ractor" concurrency primitive in Ruby 3 has its own interpreter lock: https://docs.ruby-lang.org/en/master/doc/ractor_md.html
https://pythondev.readthedocs.io/subinterpreters.html
I think "glue systems" can be generalized by saying: systems that consolidate events, and react to events in real-time. Most everything that happens in your system is an event, and things need to react to this. For example:
- A glue system will almost certainly plumb stripe events info a billing system, plus other systems for things like failed payments.
This system shouldn't go down (HA). It should be able to react to changes easily - as biz ops change frequently. It should be auditable, easy to debug, and easy to deploy.
The first pass at this in startups is usually... just doing stuff in the API request, or building webhook endpoints that manage this for you within the API. Then, you might add Sidekiq, or SQS/Lambda. Then, you might build out event-driven architecture using SQS/Kafka. It's kind of a pain, and still it's not easy to grok, debug, or architect.
Inngest handles all of this for you, though. It consolidates events from every system (internal and external, via APIs, webhooks, oauth, etc.) and then allows you to run DAG-based workflows in real-time whenever events are received. With full logging, debugging (step-over debugging), retries, user-auditing, changelogs, version control, etc.
I think that event-driven systems and glue-based systems overall haven't had much love in the developer UX, but I'm hopeful we can change that. If you're interested in using us or working with us on building the platform, ping me :)
[0] https://www.inngest.com
Once that's done and I've refactored some stuff I'm planning on opening up the core runner. Our FoundationDB persistence we're working on may likely remain closed - we will provide an interface which allows you to persist stuff and a psql adapter :)
That'll likely happen early next year.
1) "Complex systems should be designed like video games"
2) "Complex systems should be designed around a huge event bus SaaS"
Video games/engines are architected around responsiveness (low latency) & content creation.
Sending every little event to a remote SaaS for processing is sure to add tremendous latency.
To be fair, many video games due make heavy use of event processing/message passing. But the architecture of a video game event system & SaaS event bus are quite different for performance, security, reliability reasons.
Complex distributed systems are, well, complex, and can't be reduced to a simple formula or architecture. Better to spend time accurately understanding the problem that is being solved.