I guess I don't really get dagger... It reminds me of the early days gulp.js sort of? The article also confused me for a hot minute, it enthusiastically goes into a demonstration that really does make it seem like things are more complex than they should be?
Looking at the dagger docs (https://docs.dagger.io) I still don't really get the feeling there is a solid use case. The who is it for section does a poor job of winning me over.
> A developer wishing your CI pipelines were code instead of YAML.
Are there developers who wish this? Specifically developers who will not use it as an excuse to make an overly complex difficult to maintain solution instead? Not to mention that YAML pipelines can be maintained by a wider array of people some of who don't necesairly have the code skills to do the the same in code.
Furthermore, the "you can't use code in pipelines" also isn't true. If you really can't do the things you want to do with standardized YAML pipeline blocks all of them support the option to actually run any type of code as part of the pipeline. This has the benefit of still offering the readability of clearly defined steps and stages.
> Your team's "designated devops person", hoping to replace a pile of artisanal scripts with something more powerful.
By replacing it by a pile of different artisanal scripts?
> A platform engineer writing custom tooling, with the goal of unifying continuous delivery across organizational silos.
That's something most pipeline solutions can already do if you want? Pipeline templates are a thing and can be enforced across an organization.
> A cloud-native developer advocate or solutions engineer, looking to demonstrate a complex integration on short notice.
Again, something you can already do with any modern pipeline.
And maybe there are good dagger use cases, but my initial impressions based on this article and the documentation don't present them to me. Instead, I can't help but shake the feeling that this is one of those technologies that is looking for a problem to solve rather than something build to solve a problem.
The benefit of Dagger is that your builders are fully containerized, you can run CI locally, and you get to do all of that using your favorite language.
You could achieve those goals independently using things like Make + Docker + keeping CI dumb, but that is a lot of plumbing and still has edge cases. For example, Makefiles often use utilities like sed, curl, etc., which behave differently on macOS versus Linux. It also requires a lot of discipline to keep CI from growing, and you need to learn the Make syntax, which, while useful, can be a bit of an investment, especially as you get to the more advanced features/syntax.
This shines when you can write your pipelines in the same language as your app. Often, teams might have only 1-2 people who _really_ understand Bash/Make, but the rest of the team understands the language they use daily (Go, TypeScript, etc.)
Dagger lets you leverage your knowledge of more common languages to write your CI pipeline, and additionally, these builds are containerized and can all be run locally.
I have used Dagger a couple of times and liked it. One competitor in the space is Earthly [0], which has a Makefile/Dockerfile syntax. I've had good experiences with Earthly, but Dagger is a better pick for medium/large projects.
Being able to run your pipelines locally is a pretty neat use case.
That's something that often is difficult or basically impossible. Except for maybe GitHub actions through Act (https://github.com/nektos/act). I'd still lean to something in the yaml sphere if it eventually would be used in deployment pipelines and such. For example a solution incorporating ansible.
It also seems to me that the argument you make is mostly focused on the building step? Earthly certainly seems focused on that aspect.
Act is also quite imperfect around its emulation and has a lot of edge cases. Not by any virtue of being a bad project or lack of effort, it’s just that the way GHA works is slightly insane
Act works pretty well in my experience, but it's tied to GHA.
Once nice thing about Earthly + Dagger is that your build + pipeline code are bundled together, and you can run them on any compute.
As for YAML, it is ubiquitous and more accessible to those who are less comfortable writing code. One thing that I don't like about YAML si that for larger GHA workflows become hard to understand quite quickly. It's nice to work in "real" languages like Go/TypeScript where you can create and compose functions, especially if the language you're writing your pipeline in is the same language as your app -- I think that's a very powerful use case where all devs can feel comfortable/empowered to tweak CI/build steps.
> It also seems to me that the argument you make is mostly focused on the building step? Earthly certainly seems focused on that aspect.
Earthly and Dagger solve both builds and CI pipelines. They're essentially identical aside from the syntax that you use.
Earthly and Dagger fit into CI in the same way, too. You'll have a GHA that installs Earthly/Dagger, and then calls your pipeline/build step. For Dagger, this is invoking your pipeline code. For Earthly, you'd want to define a `ci` target, and invoke it, e.g. `earthly +ci` (this means that locally you can run the entire pipeline with the same command). In both cases, you'd want your GHA (or whatever runner you're using) to be as dumb as possible, e.g. it should literally clone the repo, install the CLI, and run one shell command. Maybe it can also upload test reports/annotate PRs/etc, but the more you add to your GHA/test runner code outside of these tools, the less these tools can help you.
One thing that generally is not immediately obvious is all the side benefits that using a programming language actually unlocks.
Need type checking at build time? done.
Need a package manager to version and publish your CI code? done.
Need linting / formatting? done.
Need to import functions from other places? done
Need a test framework? done.
Need a better collaborative IDE? done.
All these things above bring massive improvements to both Dev and Devops engineers that need to interact with build & deploy pipelines daily. Generally it'll be quite difficult for any other configuration or custom language to provide these kind of tooling.
I completely agree, which is why Dagger is perfect for medium/large projects.
I think Earthly is better for small projects with a simple build system since it's so much simpler to write and understand. I think Dagger is a worthwhile investment once your Earthfile is >200-400 lines.
> I completely agree, which is why Dagger is perfect for medium/large projects.
Initially that was the case, but with the upcoming work on modules (aka project Zenith), hopefully this changes. After all, IMO the ultimate aspects that will make it super easy to adopt is the ecosystem and learning how to use the tool. The later is mostly tackled by the "pick your favorite language" story; the former is where we hope modules can help users the most.
I'm sorry, I can't make sense of what you wrote, those "side benefits that using a programming language [in CI] actually unlocks". Could you please give a concrete example?
> Need type checking at build time? done.
It's not unlocked by Dagger since every CI can already do this. And it's almost always about launching a (shell) command. How is that line easier in Go/any-language than in yaml?
> Need to import functions from other places? done
You mean "import functions into the CI code"? Why would I need that? It's usually possible through plugin systems, but I've never felt the need to put complex code into CI actions.
> Need a test framework? done.
Is that a test framework to test the CI code from within the CI? I hope I'll never want one! I guess I completely misunderstood...
On the other side of those "benefits", I can smell problems. For instance, when Docker images and Compose systems are built from the Dagger API. If the need for some advanced features comes in, this extra layer could prove costly.
The parent comment was saying that those benefits could be applied to your build/CI scripts.
Consider a complex (1000+ line) YAML GitHub action that involves building, testing, linting, deploying, releasing, etc.
They are pretty common and are extremely hard to both read and write.
Dagger fixes those issues for complex pipelines since you get to use a "real" language.
> You mean "import functions into the CI code"? Why would I need that? It's usually possible through plugin systems, but I've never felt the need to put complex code into CI actions.
This might just not be a use case for you, which is okay. This is just a general property of modern programming languages, and the result of Dagger being plain Go/TypeScript/etc.
> Is that a test framework to test the CI code from within the CI? I hope I'll never want one!
At AWS we could only deploy on certain days, e.g. we wanted to ensure that our pipelines didn't deploy to too many regions at once, that we didn't deploy during holidays, or during peak shopping times of the year.
We had unit tests for our pipelines that ensured that the pipeline had the proper rules to enforce the above. I would imagine there are many other use cases for testing your pipeline.
> Consider a complex (1000+ line) YAML GitHub action that involves building, testing, linting, deploying, releasing, etc.
If one has a 1000+ yaml pipeline - this is not because they did not hear about dagger, but because they haven’t done proper decomposition and refactoring where all the complexity is abstracted away from pipelines dsl into proper general purposes language libraries
That’s how Dagger works. You use Dagger to make those building blocks cleaner, faster, and more portable. Then you can run them directly, or from your favorite CI configuration.
Yes,debugging long running /complex / multistage pipelines is a tedious process, however imho it does not make a huge difference if one runs the same pipelines locally (they won’t turn into something more lightweight after that local run), however instead we would rather have simple steps ( scripts) which we can debug individually
There's also Dagger's "Project Zenith" [1]. Details are scarce, but I asked in the Discord once and got this description of it:
Zenith is still experimental and evolving, but we think that calling functions from the CLI may become the dominant way to run Dagger because of the many benefits:
- reduced host env requirements/setup: you only need the `dagger` cli and the ability to run containers (no local dependencies like `golang`, `python`, `node`, etc
- cross-language modules ecosystem: Any module written in any SDK is usable in your module written in your SDK of choice (e.g. use a Python mod from Go)
- expressive verbs: `dagger exec`, `dagger save`, `daggger print`, `dagger call`, etc (names in flux a bit)
- easier/shorter/quicker to write for most folks
Apparently they're also trying to make Zenith Dagger consumable from non-Zenith Dagger [2] (though the terminology is confusing).
> Often, teams might have only 1-2 people who _really_ understand Bash/Make, but the rest of the team understands the language they use daily (Go, TypeScript, etc.)
This is not what matches my experience, make and Bash are relatively easy to start working with, the problem starts when one wants to use them for _everything_ instead of having a _reasonable_ mixture of bash/make AND general purpose high level languages ( golang among them ), this is why I’ve created SparrowCI where neither or tools are enforced.
YAML is the worst of both worlds. It pretends to be just simple configuration, but it by necessity cannot be that, because it gets interpreted by the CI system. The result is an unholy language selling itself as simple JSON with comments and less quote escaping. (At least nobody got the bright idea to put jinja in there.)
The CDK people saw this and went the much saner strongly typed typescript JSON synthesis. It’s still shit but at least you get your config statically analyzed before pushing it 17 times until it finally works (because why would you want to test locally, nobody does that).
> Are there developers who wish this? Specifically developers who will not use it as an excuse to make an overly complex difficult to maintain solution instead? Not to mention that YAML pipelines can be maintained by a wider array of people some of who don't necesairly have the code skills to do the the same in code.
As someone who has been building automatic deployment systems for 5+ years, I do.
I'd like to create libraries and workflows that expose infrastructure and deployment best practices to developers so they can spend their time solving business issues.
Dockerfiles + Docker Compose + YAML template files are good enough for simple cases, but they will quickly break once you need to support different, possible configurable use cases.
I've been using AWS CDK in Python to expose infrastructure libraries for developers and the difference is night and day vs using plain YAML. People just build stuff and deploy apps without talking to our infra team at all, object / functions serve as neat, statically checked, documented interfaces for our devs.
I'm not sure Dagger is THE solution I'm looking for CI/CD, but having something that exposes rich declarative interfaces for automatic deployments is most definitely not a baseless idea.
Disclaimer: I build and maintain'ish the rust sdk of dagger.
If you've ever worked with a sufficiently large pipeline, or set of pipelines, you face the fact that stuff breaks and you have to spam commits to try to get the weird yaml dsl to do the thing you want. Not that dagger is that much different in that context. But you can actually test your pipeline locally, shortening the feedback loop from 10 minutes to 10 seconds.
We've even gone as far as integrating it into our developer tools, with caching we still have nice to use clis and apps. We just don't have to think too much about deployment as we're heavily invested in docker already.
From a platform engineering perspective. We'd like actual guarantees about our pipelines when we roll them out to thousands of repositories. With dagger, we can do traditional unit tests on parts of our pipeline, and full integration test (or whatever you'd like to call them) on whole example pipelines.
This is something that would suck to do, with github actions, bash and docker by itself. And as such isn't done often, and when done brittle. Been there done that.
Another example is doing programmatic builds in a more comfortable language than Docker or bash. Lets say I'd like to solve rusts build/caching issues. With dagger, or other programmatic build tools. I can now mix docker commands, with computation to solve my problem. I can for example rely on golangs hashing libraries etc. rather than installing a bunch of packages, I don't want to end up in my final images anyway.
Dagger shouldn't be used by developers directly. IMO the same as traditional pipelines, if the developers have to touch these files, then something is wrong. Instead it should be built and maintained by a platform team, with contributions by developers whom need specific features (which dagger allows, because you can just use a regular package manager).
This may sound a little fanboyish, but we've just felt the need for less friction in traditional CI systems on actually having modern software engineering tools and methodologies available. Something which Dagger enables for us.
To add to what's already been said: If you think about it, CI pipelines are typically a complete description of how your system is built, tested, and deployed.
Which is pretty fantastic except for how walled off they are. You can't really re-use these descriptions for e.g. development, they're not vendor agnostic, and they only way to run them is by pushing your code.
Maybe it's a silly analogy but it's almost like being a web dev that doesn't have a browser and needs to send their code to a friend who can tell them if that font size looks good.
I think we're way over due for freeing these "blueprints" of our system from the confines of CI and making them portable and flexible. And containers are the technology that's enabling that.
Full disclaimer (as always): I work at Garden[0] where we're also solving that problem but taking a slightly different approach to Dagger (it's still a DAG). Garden config is declarative and the jobs (we call them actions) have a semantic meaning. You can e.g. have a Build action of type container or a Deploy action of type Helm and Garden will figure out what to do with it.
We've also put a lot of emphasis on the inner loop development story with hot reloading functionality, log streaming and more.
We did indeed have a reputation for being a bit more complicated then some of the other tools out there (and in turn more flexible). But we've put enormous effort into simplifying Garden itself and also improving documentation.
If you already have your stack containerised you should be good to go but experience with Kubernetes is definitely a plus. Most of our users use K8s in production and use Garden for developing and testing in production-like environments in "inner loop" dev and CI.
Dagger fits very nice with small and large dev and devops teams. Similarly to other tools, the more you grow, the better you'll have to organize your projects to make the best use of it.
We are big fans of https://earthly.dev/! Although we haven't personally used Dagger, Earthly has solved our multi-service integration testing problem with elegance. Simple builds + caching baked in.
So . . . it's a complicated way to do test fakes with a dependency injection library? And not really anything to do with CD or much to do with CI except that it runs the same tests as on dev boxen?
The article seems like it's trying to use click-baity terms and is frankly not the easiest to follow. LLM?
Test fakes are still a kind of white-box testing. They are built right into the code of the application and re-use the types that you define within. The kind of integration tests defined in the blog post are, in my opinion, are the most black-box kind of approach you can have. And as such, they are a great way of testing APIs that have many external clients since you are validating it as if you were one. You only look at HTTP, you don't look at how it is implemented.
> frankly not the easiest to follow
I would love to know more about this. As you can see in my blog, I'm not a very frequent writer, if you have concrete feedback I can use to write better and make blog posts that are easier to follow that would help me tremendously.
> LLM?
No. I wrote every word myself, even the mistakes. I based it on something I built for the company I'm currently working at.
Dagger is really just a streamlined implementation of two very common patterns:
1) Keeping your CI configuration as light and "dumb" as possible, by moving as much logic as possible into portable scripts. This minimizes "push and pray", where any pipeline change requires committing, pushing, and waiting for the proprietary CI black box to give you a green or red light. Ideally those scripts use containers for maximum reproduceability.
2) Replacing artisanal shell scripts, Makefiles and arcane DSLs with regular code, backed by a modern API. This allows using modern development techniques, a broader ecosystem of tools, and growing the pool of people on the team who can understand and improve the pipelines they use.
To many devops teams, these patterns are just common sense, an everyday reality. For others, they are aspirational: maybe one day, if we find the time... Implementations remain very custom. Our goal at Dagger is to help democratize this way of creating pipelines, and making it a standard, so that an actual software ecosystem can appear, where devops engineers can actually reuse each other's code, the same way application developers can.
Dagger seems to attempt to reinvent Tekton which is based on https://cloud.google.com/build which is a good way to do cicd, so credit to the Dagger guys for knowing what DevOps (at least if they are real DevOps) folks know.
34 comments
[ 4.8 ms ] story [ 82.1 ms ] threadLooking at the dagger docs (https://docs.dagger.io) I still don't really get the feeling there is a solid use case. The who is it for section does a poor job of winning me over.
> A developer wishing your CI pipelines were code instead of YAML.
Are there developers who wish this? Specifically developers who will not use it as an excuse to make an overly complex difficult to maintain solution instead? Not to mention that YAML pipelines can be maintained by a wider array of people some of who don't necesairly have the code skills to do the the same in code.
Furthermore, the "you can't use code in pipelines" also isn't true. If you really can't do the things you want to do with standardized YAML pipeline blocks all of them support the option to actually run any type of code as part of the pipeline. This has the benefit of still offering the readability of clearly defined steps and stages.
> Your team's "designated devops person", hoping to replace a pile of artisanal scripts with something more powerful.
By replacing it by a pile of different artisanal scripts?
> A platform engineer writing custom tooling, with the goal of unifying continuous delivery across organizational silos.
That's something most pipeline solutions can already do if you want? Pipeline templates are a thing and can be enforced across an organization.
> A cloud-native developer advocate or solutions engineer, looking to demonstrate a complex integration on short notice.
Again, something you can already do with any modern pipeline.
And maybe there are good dagger use cases, but my initial impressions based on this article and the documentation don't present them to me. Instead, I can't help but shake the feeling that this is one of those technologies that is looking for a problem to solve rather than something build to solve a problem.
You could achieve those goals independently using things like Make + Docker + keeping CI dumb, but that is a lot of plumbing and still has edge cases. For example, Makefiles often use utilities like sed, curl, etc., which behave differently on macOS versus Linux. It also requires a lot of discipline to keep CI from growing, and you need to learn the Make syntax, which, while useful, can be a bit of an investment, especially as you get to the more advanced features/syntax.
This shines when you can write your pipelines in the same language as your app. Often, teams might have only 1-2 people who _really_ understand Bash/Make, but the rest of the team understands the language they use daily (Go, TypeScript, etc.)
Dagger lets you leverage your knowledge of more common languages to write your CI pipeline, and additionally, these builds are containerized and can all be run locally.
I have used Dagger a couple of times and liked it. One competitor in the space is Earthly [0], which has a Makefile/Dockerfile syntax. I've had good experiences with Earthly, but Dagger is a better pick for medium/large projects.
[0]: https://earthly.dev/
It also seems to me that the argument you make is mostly focused on the building step? Earthly certainly seems focused on that aspect.
Once nice thing about Earthly + Dagger is that your build + pipeline code are bundled together, and you can run them on any compute.
As for YAML, it is ubiquitous and more accessible to those who are less comfortable writing code. One thing that I don't like about YAML si that for larger GHA workflows become hard to understand quite quickly. It's nice to work in "real" languages like Go/TypeScript where you can create and compose functions, especially if the language you're writing your pipeline in is the same language as your app -- I think that's a very powerful use case where all devs can feel comfortable/empowered to tweak CI/build steps.
> It also seems to me that the argument you make is mostly focused on the building step? Earthly certainly seems focused on that aspect.
Earthly and Dagger solve both builds and CI pipelines. They're essentially identical aside from the syntax that you use.
Earthly and Dagger fit into CI in the same way, too. You'll have a GHA that installs Earthly/Dagger, and then calls your pipeline/build step. For Dagger, this is invoking your pipeline code. For Earthly, you'd want to define a `ci` target, and invoke it, e.g. `earthly +ci` (this means that locally you can run the entire pipeline with the same command). In both cases, you'd want your GHA (or whatever runner you're using) to be as dumb as possible, e.g. it should literally clone the repo, install the CLI, and run one shell command. Maybe it can also upload test reports/annotate PRs/etc, but the more you add to your GHA/test runner code outside of these tools, the less these tools can help you.
Here's an example using Earthly for CI:
https://github.com/shepherdjerred/glitter-boys/blob/main/.gi...
A similar example using Dagger:
https://github.com/shepherdjerred/shepherdjerred.com/blob/ma...
If you like Earthly and feel limited by syntax, then try out Dagger.
One thing that generally is not immediately obvious is all the side benefits that using a programming language actually unlocks.
Need type checking at build time? done.
Need a package manager to version and publish your CI code? done.
Need linting / formatting? done.
Need to import functions from other places? done
Need a test framework? done.
Need a better collaborative IDE? done.
All these things above bring massive improvements to both Dev and Devops engineers that need to interact with build & deploy pipelines daily. Generally it'll be quite difficult for any other configuration or custom language to provide these kind of tooling.
I think Earthly is better for small projects with a simple build system since it's so much simpler to write and understand. I think Dagger is a worthwhile investment once your Earthfile is >200-400 lines.
Initially that was the case, but with the upcoming work on modules (aka project Zenith), hopefully this changes. After all, IMO the ultimate aspects that will make it super easy to adopt is the ecosystem and learning how to use the tool. The later is mostly tackled by the "pick your favorite language" story; the former is where we hope modules can help users the most.
> Need type checking at build time? done.
It's not unlocked by Dagger since every CI can already do this. And it's almost always about launching a (shell) command. How is that line easier in Go/any-language than in yaml?
> Need to import functions from other places? done
You mean "import functions into the CI code"? Why would I need that? It's usually possible through plugin systems, but I've never felt the need to put complex code into CI actions.
> Need a test framework? done.
Is that a test framework to test the CI code from within the CI? I hope I'll never want one! I guess I completely misunderstood...
On the other side of those "benefits", I can smell problems. For instance, when Docker images and Compose systems are built from the Dagger API. If the need for some advanced features comes in, this extra layer could prove costly.
It sounds like you're not in need for most of the use cases but each of these points presents decent value at certain scales.
Consider a complex (1000+ line) YAML GitHub action that involves building, testing, linting, deploying, releasing, etc.
They are pretty common and are extremely hard to both read and write.
Dagger fixes those issues for complex pipelines since you get to use a "real" language.
> You mean "import functions into the CI code"? Why would I need that? It's usually possible through plugin systems, but I've never felt the need to put complex code into CI actions.
This might just not be a use case for you, which is okay. This is just a general property of modern programming languages, and the result of Dagger being plain Go/TypeScript/etc.
> Is that a test framework to test the CI code from within the CI? I hope I'll never want one!
At AWS we could only deploy on certain days, e.g. we wanted to ensure that our pipelines didn't deploy to too many regions at once, that we didn't deploy during holidays, or during peak shopping times of the year.
We had unit tests for our pipelines that ensured that the pipeline had the proper rules to enforce the above. I would imagine there are many other use cases for testing your pipeline.
If one has a 1000+ yaml pipeline - this is not because they did not hear about dagger, but because they haven’t done proper decomposition and refactoring where all the complexity is abstracted away from pipelines dsl into proper general purposes language libraries
[1] https://docs.dagger.io/zenith/
[2] https://github.com/dagger/dagger/issues/5993
This is not what matches my experience, make and Bash are relatively easy to start working with, the problem starts when one wants to use them for _everything_ instead of having a _reasonable_ mixture of bash/make AND general purpose high level languages ( golang among them ), this is why I’ve created SparrowCI where neither or tools are enforced.
The CDK people saw this and went the much saner strongly typed typescript JSON synthesis. It’s still shit but at least you get your config statically analyzed before pushing it 17 times until it finally works (because why would you want to test locally, nobody does that).
As someone who has been building automatic deployment systems for 5+ years, I do.
I'd like to create libraries and workflows that expose infrastructure and deployment best practices to developers so they can spend their time solving business issues.
Dockerfiles + Docker Compose + YAML template files are good enough for simple cases, but they will quickly break once you need to support different, possible configurable use cases.
I've been using AWS CDK in Python to expose infrastructure libraries for developers and the difference is night and day vs using plain YAML. People just build stuff and deploy apps without talking to our infra team at all, object / functions serve as neat, statically checked, documented interfaces for our devs.
I'm not sure Dagger is THE solution I'm looking for CI/CD, but having something that exposes rich declarative interfaces for automatic deployments is most definitely not a baseless idea.
If you've ever worked with a sufficiently large pipeline, or set of pipelines, you face the fact that stuff breaks and you have to spam commits to try to get the weird yaml dsl to do the thing you want. Not that dagger is that much different in that context. But you can actually test your pipeline locally, shortening the feedback loop from 10 minutes to 10 seconds.
We've even gone as far as integrating it into our developer tools, with caching we still have nice to use clis and apps. We just don't have to think too much about deployment as we're heavily invested in docker already.
From a platform engineering perspective. We'd like actual guarantees about our pipelines when we roll them out to thousands of repositories. With dagger, we can do traditional unit tests on parts of our pipeline, and full integration test (or whatever you'd like to call them) on whole example pipelines.
This is something that would suck to do, with github actions, bash and docker by itself. And as such isn't done often, and when done brittle. Been there done that.
Another example is doing programmatic builds in a more comfortable language than Docker or bash. Lets say I'd like to solve rusts build/caching issues. With dagger, or other programmatic build tools. I can now mix docker commands, with computation to solve my problem. I can for example rely on golangs hashing libraries etc. rather than installing a bunch of packages, I don't want to end up in my final images anyway.
Dagger shouldn't be used by developers directly. IMO the same as traditional pipelines, if the developers have to touch these files, then something is wrong. Instead it should be built and maintained by a platform team, with contributions by developers whom need specific features (which dagger allows, because you can just use a regular package manager).
This may sound a little fanboyish, but we've just felt the need for less friction in traditional CI systems on actually having modern software engineering tools and methodologies available. Something which Dagger enables for us.
Which is pretty fantastic except for how walled off they are. You can't really re-use these descriptions for e.g. development, they're not vendor agnostic, and they only way to run them is by pushing your code.
Maybe it's a silly analogy but it's almost like being a web dev that doesn't have a browser and needs to send their code to a friend who can tell them if that font size looks good.
I think we're way over due for freeing these "blueprints" of our system from the confines of CI and making them portable and flexible. And containers are the technology that's enabling that.
Full disclaimer (as always): I work at Garden[0] where we're also solving that problem but taking a slightly different approach to Dagger (it's still a DAG). Garden config is declarative and the jobs (we call them actions) have a semantic meaning. You can e.g. have a Build action of type container or a Deploy action of type Helm and Garden will figure out what to do with it.
We've also put a lot of emphasis on the inner loop development story with hot reloading functionality, log streaming and more.
[0] https://github.com/garden-io/garden
We did indeed have a reputation for being a bit more complicated then some of the other tools out there (and in turn more flexible). But we've put enormous effort into simplifying Garden itself and also improving documentation.
If you already have your stack containerised you should be good to go but experience with Kubernetes is definitely a plus. Most of our users use K8s in production and use Garden for developing and testing in production-like environments in "inner loop" dev and CI.
You can check out our quickstart guide, it can get you up and running in a few minutes: https://docs.garden.io/getting-started/quickstart
We're also always happy to help on Discord if you have questions: https://discord.gg/FrmhuUjFs6
Getting your CI/CD code out of some CI-only runnable format and into something you can run on your local or wherever is game-changing.
You should do it :)
The article seems like it's trying to use click-baity terms and is frankly not the easiest to follow. LLM?
https://blog.pragmatists.com/test-doubles-fakes-mocks-and-st...
> frankly not the easiest to follow I would love to know more about this. As you can see in my blog, I'm not a very frequent writer, if you have concrete feedback I can use to write better and make blog posts that are easier to follow that would help me tremendously.
> LLM? No. I wrote every word myself, even the mistakes. I based it on something I built for the company I'm currently working at.
Dagger is really just a streamlined implementation of two very common patterns:
1) Keeping your CI configuration as light and "dumb" as possible, by moving as much logic as possible into portable scripts. This minimizes "push and pray", where any pipeline change requires committing, pushing, and waiting for the proprietary CI black box to give you a green or red light. Ideally those scripts use containers for maximum reproduceability.
2) Replacing artisanal shell scripts, Makefiles and arcane DSLs with regular code, backed by a modern API. This allows using modern development techniques, a broader ecosystem of tools, and growing the pool of people on the team who can understand and improve the pipelines they use.
To many devops teams, these patterns are just common sense, an everyday reality. For others, they are aspirational: maybe one day, if we find the time... Implementations remain very custom. Our goal at Dagger is to help democratize this way of creating pipelines, and making it a standard, so that an actual software ecosystem can appear, where devops engineers can actually reuse each other's code, the same way application developers can.