I was interested in the beginning for CI/CD, but then they tried to take a kind of "AI-oriented" view in order to ride the AI wave, and the value prop of their tool was completely muddied up...
This looks interesting but I’m trying to understand it in more layman’s terms. Is it more about providing abstractions for llms to work within to do things?
Dagger was something I looked into two or so years ago before they got consumed by the LLM and AI agent hype, and while the promise of being able to run the exact CI workflows locally seemed excellent, it seemed that there's basically no way be a Dagger user without buying into their Dagger Cloud product.
I ended up opting for CUE and GitHub Actions, and I'm glad I did as it made everything much, much simpler.
I might be getting old but the videos are too fast for me to understand. Why can't they just put the full command text and the output of it instead of a video.
A lot of the comments here feel like they're disappointed that this is a "Docker with unnecessary LLM crap thrown in" when I think what they're really going for is more "LLM workflows with a higher degree of observability and sanity".
I think a more interesting point of comparison is the Claude Code Github Action, Co-Pilot code reviews, etc.
I think it started as some kind of CI/CD tools, then they jumped on the AI hype and they started to use it to make it possible to run agents in containers easily... perhaps to do automated actions on CI/CD pipelines that use agents (eg try to solve some minor bugs automatically when you push on a branch, etc)
Although I'm not sure if that's so much a value-added? It's not so hard to just create a container and launch an agent in it.
The whole interesting thing was to use actual programming languages for Docker build, which I think was what they initially tried to do, but now it's a bit incomprehensible... I guess conceptually Dagger relates to Dockerfile a bit like Pulumi related to Terraform?
I thought Dagger had/has a lot of potential to be "AWS-CDK for CI pipelines".
I.e. declaratively setup a web of CI / deployment tasks, based on docker, with a code-first DSL, instead of the morass of copy-pasted (and yes orbs) CircleCI yaml files we have strewn about our internals repos.
But their DSL for defining your pipelines is ... golang? Like who would pick golang as "a friendly language for setting up configs".
The underlying tech is technically language-agnostic, just as aws-cdk's is (you can share cdk constructs across TypeScript/Python), but it's rooted in golang as the originating/first-class language, so imo will never hit aws-cdk levels of ergonomics.
That technical nit aside, I love the idea; ran a few examples of it a year or so ago and was really impressed with the speed; just couldn't wrap my around "how can I make this look like cdk".
If I understand correctly, this is essentially a more composable way to write Dockerfiles? That alone is a very welcome improvement. They would do themselves a big favor if they were more clear on that in their marketing, instead of boasting around the bush with all kinds of other terminology and claims of redefining foundations.
If I already have a Dockerfile that doesn’t need composition, how does this help me vs being a small cosmetic improvement over ”docker build” command line?
I loved the original promise of Dagger and it’s still 90% great.
But one flaw (IMO) that it can’t export artifacts and import into other steps without breaking the cache.
Eg if you provide monorepo as input, and then on some step narrow your build to one specific dir, then even when files change outside of that dir then caching still is invalidated.
Which makes it extremely verbose and maintenance nightmare to keep multiple narrow inputs and keep all those paths up to date.
As someone that has used Dagger a lot (a previous daggernaut / ambassador dropped off after LLMs was announced, and was changing jobs at the time. implemented it at a previous company across 95% of services, built the rust sdk) the approach was and is amazing for building complex build chains.
It serves a place where a dockerfile is not enough, and CI workflows are too difficult to debug or reason about.
I do have some current problems with it though:
1. I don't care at all about the LLM agent workflows, I get that it is possible, but the same people that chose dagger for what it was, is not the same audience that runs agents like that. I can't choose dagger currently, because I don't know if they align with my interests as an engineer solving a specific problems for where I work (delivering software, not running agents).
2. I advocated for modules before it was a thing, but I never implemented it. It is too much magic, I want to write code, not a DSL that looks like code, dagger is already special in that regard, to modules takes it a step too far. You can't find the code in their docs anymore, but dagger can be written with just a .go, .py or .rs file. Simply take in dagger as a dependency and build your workflow.
3. Too complex to operate, dagger doesn't have runners currently, and it is difficult to run a setup in production for CI yourself, without running it in the actions themselves, which can be disastrous for build times, as dagger often leads you into using quite a few images, so having a cache is a must.
Dagger needs to choose and execute; not having runners, even when we we're willing to throw money at them was a mistake IMO. Love the tool, the team, the vision but it is too distracted, magical and impatient to pick up at the moment.
if you think dagger is a CI/CD enabler, it is, but its more – its module and function orchestration is a much more basic and first principle it embraces. The team has iterated from the CI/CD narrative to something much more powerful.
20 comments
[ 0.18 ms ] story [ 43.0 ms ] threadThen... it wasn't. The more I read the less I ever want to see this again. The LLM train has got to end at some point.
Without the LLM bits, this is basically like Bazel or buck2, right?
I ended up opting for CUE and GitHub Actions, and I'm glad I did as it made everything much, much simpler.
What else could be used to abstract away your CICD from the launcher (Jenkins, Argo Workflows, GitHub Actions, etc.)?
They don't seem to have jumped for AI hype (yet?)...
https://www.windmill.dev/
I think a more interesting point of comparison is the Claude Code Github Action, Co-Pilot code reviews, etc.
Although I'm not sure if that's so much a value-added? It's not so hard to just create a container and launch an agent in it.
The whole interesting thing was to use actual programming languages for Docker build, which I think was what they initially tried to do, but now it's a bit incomprehensible... I guess conceptually Dagger relates to Dockerfile a bit like Pulumi related to Terraform?
I.e. declaratively setup a web of CI / deployment tasks, based on docker, with a code-first DSL, instead of the morass of copy-pasted (and yes orbs) CircleCI yaml files we have strewn about our internals repos.
But their DSL for defining your pipelines is ... golang? Like who would pick golang as "a friendly language for setting up configs".
The underlying tech is technically language-agnostic, just as aws-cdk's is (you can share cdk constructs across TypeScript/Python), but it's rooted in golang as the originating/first-class language, so imo will never hit aws-cdk levels of ergonomics.
That technical nit aside, I love the idea; ran a few examples of it a year or so ago and was really impressed with the speed; just couldn't wrap my around "how can I make this look like cdk".
If I already have a Dockerfile that doesn’t need composition, how does this help me vs being a small cosmetic improvement over ”docker build” command line?
But one flaw (IMO) that it can’t export artifacts and import into other steps without breaking the cache.
Eg if you provide monorepo as input, and then on some step narrow your build to one specific dir, then even when files change outside of that dir then caching still is invalidated.
Which makes it extremely verbose and maintenance nightmare to keep multiple narrow inputs and keep all those paths up to date.
It serves a place where a dockerfile is not enough, and CI workflows are too difficult to debug or reason about.
I do have some current problems with it though:
1. I don't care at all about the LLM agent workflows, I get that it is possible, but the same people that chose dagger for what it was, is not the same audience that runs agents like that. I can't choose dagger currently, because I don't know if they align with my interests as an engineer solving a specific problems for where I work (delivering software, not running agents).
2. I advocated for modules before it was a thing, but I never implemented it. It is too much magic, I want to write code, not a DSL that looks like code, dagger is already special in that regard, to modules takes it a step too far. You can't find the code in their docs anymore, but dagger can be written with just a .go, .py or .rs file. Simply take in dagger as a dependency and build your workflow.
3. Too complex to operate, dagger doesn't have runners currently, and it is difficult to run a setup in production for CI yourself, without running it in the actions themselves, which can be disastrous for build times, as dagger often leads you into using quite a few images, so having a cache is a must.
Dagger needs to choose and execute; not having runners, even when we we're willing to throw money at them was a mistake IMO. Love the tool, the team, the vision but it is too distracted, magical and impatient to pick up at the moment.