Claude, find a key underpinning of most development workflows, then make a product to disrupt (and eventually SaaSify/enshitify) it. Finally, write a convincing ad disguised blog post, and purchase a fancy domain. Here's my credit card.
Over at varlock (https://varlock.dev -- also free, open source), we agree that .env as we know it is full of problems. But instead of abandoning it, we evolved it. We replace your .env.example with a .env.schema - using decorator style comments to add schema info, and functions to load and compose values.
A big difference between our tool and many other similar tools is that we combine the schema and value setting into one surface, with a way of merging many definitions together, much like cuelang - but in a way that feels more intuitive. It's extremely flexible, and can even do credential brokering for untrusted workloads.
I've been enjoying the secretspec content lately, and watching it evolve :)
That's what we are using as well in our small team. `mise` for our non-sensitive values/ environment variables and then `fnox` (from the same author) to encrypt and manage the secrets.
`mise` has a plugin for `fnox` that allows to automatically load the secrets in the context when running a mise task/command.
This way, our secrets are never exposed, even during local development.
Setting this up was effortless. Pretty happy about these tools.
I'm sure there are many better alternatives to .env, but its ubiquity and support across various tools makes it super convenient.
I'm using 1Password's .env integration[1] and although the UX is a bit clunky, I really like it. My API keys are secure, tools that ordinarily support
.env just work and there's a team-sharing feature too (although I'm yet to use it). It's pretty neat.
Hm, I haven't seen these issues personally. We only have one `.env` file and it's just for local secrets. Configuration emphatically does not go in `.env` and ideally is in docker compose and defined in code (we use Pydantic Settings).
I hate that it's hidden too. There's no reason for it to be hidden and arguably some reason for it not to be hidden.
I did discover one reason for it not to be just "env" though, which surprised me, which is that "source env" will yield "bash: source: /usr/bin/env: cannot execute binary file". I did not expect the source command to use the $PATH to resolve the filename. Probably some minor security issues that can result from that out there in the world. Arguably anyone loading it should use "source ./.env" or equivalent, with full path.
It's documented in the bash manual, of course. But it is rather complicated as to when it will and will not use $PATH.
> If filename does not contain a slash, . searches for it. If -p is supplied, . treats path as a colon-separated list of directories in which to find filename; otherwise, . uses the directories in PATH to find filename. filename does not need to be executable. When Bash is not in POSIX mode, it searches the current directory if filename is not found in $PATH, but does not search the current directory if -p is supplied. If the sourcepath option (see The Shopt Builtin) is turned off, . does not search PATH.
From "The Open Group Base Specifications Issue 8 - IEEE Std 1003.1-2024" [2]:
. file
> If file does not contain a <slash>, the shell shall use the search path specified by PATH to find the directory containing file. Unlike normal command search, however, the file searched for by the dot utility need not be executable.
So it seems that it's not something Bash specific.
~/code
» cd foo
~/code/foo
» git init
± foo:main:/
» touch .env
± foo:main:/
» git add .
± foo:main:/
» g s
On branch main
No commits yet
Changes to be committed:
(use "git rm --cached <file>..." to unstage)
new file: .env
± foo:main:/
»
File is added, kaboom. "It should be in .gitignore" yes, true, but that has nothing to do with it not getting committed because it is hidden.
And even with it in .gitignore, I've dealt with multiple security incidents where someone has managed to commit it anyways. (And yes, I'm aware there are commands to do this, but what I haven't figured out is why someone would work around the safety and not think "what's the point of this?" prior to the bullet ending up in the foot.)
This is why I love direnv (the tool) and just a simple .envrc (not .env). It has none of the problems posited in the OP's article. It assumes nothing, only that direnv manges loading up the .envrc into your shell.
This article reads like more of an ad than anything else.
> Environment variables only deliver values
Yes, the problem .env files try to solve is having "environment variables" be injectable from a file so that different applications can have different environment variables by default.
> A string is not a schema
Yes, input validation is an application concern. The application should know what these values represent / how to parse them and error if they're invalid.
I could go on, but the article is all about trying to use a hammer as a screwdriver and complaining that the hammer doesnt work.
I see many .env files with unquoted values, like the first example in OP
REDIS_URL=redis://localhost:6379
and it triggers me big time. Always quoting shell variable values is so deeply ingrained from very painful experiences decades ago that it still causes a somatic flashback whenever I see it.
This is what went wrong with .env
Yes, I know
docker --env-file
doesn't work with quotes but that's just docker being broken and why they fixed it in docker compose.
.env files are fine, as long as you don't import a random dependency to open and read a file. If you think about it for ten seconds, it's unjustifiable, and it's only done because others do it.
Type "dotprot" to copy your .env file into 1Password using the 1P CLI, it verifies it's there, then deletes the local copy of .env. When you're ready to work again, type "dotprot" again to restore the .env file from 1P. There is a .prot file in the directory that you can declare other files to store in 1P as well.
Varlock (https://varlock.dev ) is also good for this and more mature. I use it on some work projects, but for local dev, I wanted something to allow me to be even lazier than that.
I can't understand for the love of me why our industry settled on env as The Way to store configuration. It's poorly discoverable, it's a shared namespace polluted by everyone and their `libdog`, it's stringly typed, etc. All of that just flies in the face of otherwise accepted dev wisdom.
I love Django's approach to configs: your config is just a plain old Python file with some constants. As simple as possible, and works great.
I'm working with .env and secrets and teamwide configs for the first time, trying to understand solutions from first principles.
Are these valid observations so far?
- .env is the simplest and oldest / most boring solution. Secrets are passed to teammates by DM?
- .env.example seems like a nice home for documentation about secrets
- but I'd rather avoid writing sensitive credentials at all -- instead, teams can use a cloud-based secrets manager and the project fetches secrets at runtime. Is this ever a hassle / any downsides? How standard is this practice these days? I hear this also helps updating secrets so you don't have to e.g. tell every teammate when you rotate a secret. Generally not a fan of introducing a network call and a saas dependency, tho
- non-sensitive environment variables also need a home. I don't like the idea of cluttering the project with a .env.local or worse like .env.development.local
- I hear lots of secrets and config management happens in container orchestration, but what if I barely have a container in the first place? I guess docker compose is one of the simplest tools at this layer?
I guess I'm just looking for a safe, simple solution for a small team, and maybe the problem is that every team does things differently, and that many tools are marketed towards huge enterprise teams.
> - .env is the simplest and oldest / most boring solution. Secrets are passed to teammates by DM?
I don't see a reason to ever pass them. Sensitive production values should never be accessible to everyone, and for development the team should be able to create them themselves.
I usually keep very stupid values in .env.example, with the instructions to `cp .env.example .env`. In development it doesn't matter if the password is "password". In production the privileged admin sets up actual values.
> - non-sensitive environment variables also need a home. I don't like the idea of cluttering the project with a .env.local or worse like .env.development.local
`cp .env.example .env` and the application uses .env just like in production, only with dummy values.
I wish there was better support for something like `SECRET_CMD="..."` (ie: "run this command to get/refresh secrets"), but that's just baking arbitrary command execution into your development pipeline.
Personally, I tend to have a `source ./source-me-auth` which does stuff like `DB_PASSWORD="$( pass show blah.com | head -1 )"`, but that still means I end up with secrets dangling around in my environment. The "source-me" file is safe to share, but my runtime environment is tainted (for good and bad...).
I saw something w.r.t. the way open-claw handles things where you basically say `export DB_PASSWORD="REDACTED@SECRET_001"` and then it basically outbound filters network requests to hydrate that with `s/REDACTED@SECRET_001/$REAL_SECRET/g`, and I _really_ like that mechanism b/c it defers secret usage to runtime and keeps it physically "out" of the application itself.
Basically, having `with $SECRETS -- some_app.sh --some params ...` wouldn't be terrible (conceptually).
40 comments
[ 2.3 ms ] story [ 21.5 ms ] threadA big difference between our tool and many other similar tools is that we combine the schema and value setting into one surface, with a way of merging many definitions together, much like cuelang - but in a way that feels more intuitive. It's extremely flexible, and can even do credential brokering for untrusted workloads.
I've been enjoying the secretspec content lately, and watching it evolve :)
(Am I getting thrown out the window now? :)
I'm using 1Password's .env integration[1] and although the UX is a bit clunky, I really like it. My API keys are secure, tools that ordinarily support .env just work and there's a team-sharing feature too (although I'm yet to use it). It's pretty neat.
[1] https://www.1password.dev/environments
https://x.com/0xPrajwal_/status/2084163731534311627
https://x.com/nilbuild/status/2084339074639200488
The issue is that agents need structure and context.
start with making it a dot file. Why would you want to hide the fact your app was using loaded environment variables?
I did discover one reason for it not to be just "env" though, which surprised me, which is that "source env" will yield "bash: source: /usr/bin/env: cannot execute binary file". I did not expect the source command to use the $PATH to resolve the filename. Probably some minor security issues that can result from that out there in the world. Arguably anyone loading it should use "source ./.env" or equivalent, with full path.
It's documented in the bash manual, of course. But it is rather complicated as to when it will and will not use $PATH.
From "The Open Group Base Specifications Issue 8 - IEEE Std 1003.1-2024" [2]:
> If file does not contain a <slash>, the shell shall use the search path specified by PATH to find the directory containing file. Unlike normal command search, however, the file searched for by the dot utility need not be executable.So it seems that it's not something Bash specific.
[1]: https://www.gnu.org/software/bash/manual/html_node/Bourne-Sh... )
[2]: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V...
And even with it in .gitignore, I've dealt with multiple security incidents where someone has managed to commit it anyways. (And yes, I'm aware there are commands to do this, but what I haven't figured out is why someone would work around the safety and not think "what's the point of this?" prior to the bullet ending up in the foot.)
It may be time for Hackernews to integrate a Pangram detector into the UI, similar to what substack is doing :)
> Environment variables only deliver values
Yes, the problem .env files try to solve is having "environment variables" be injectable from a file so that different applications can have different environment variables by default.
> A string is not a schema
Yes, input validation is an application concern. The application should know what these values represent / how to parse them and error if they're invalid.
I could go on, but the article is all about trying to use a hammer as a screwdriver and complaining that the hammer doesnt work.
This is what went wrong with .env
Yes, I know
doesn't work with quotes but that's just docker being broken and why they fixed it in docker compose.Also see jt2290's comment:
https://news.ycombinator.com/item?id=49171684
Type "dotprot" to copy your .env file into 1Password using the 1P CLI, it verifies it's there, then deletes the local copy of .env. When you're ready to work again, type "dotprot" again to restore the .env file from 1P. There is a .prot file in the directory that you can declare other files to store in 1P as well.
Varlock (https://varlock.dev ) is also good for this and more mature. I use it on some work projects, but for local dev, I wanted something to allow me to be even lazier than that.
I love Django's approach to configs: your config is just a plain old Python file with some constants. As simple as possible, and works great.
Are these valid observations so far?
- .env is the simplest and oldest / most boring solution. Secrets are passed to teammates by DM?
- .env.example seems like a nice home for documentation about secrets
- but I'd rather avoid writing sensitive credentials at all -- instead, teams can use a cloud-based secrets manager and the project fetches secrets at runtime. Is this ever a hassle / any downsides? How standard is this practice these days? I hear this also helps updating secrets so you don't have to e.g. tell every teammate when you rotate a secret. Generally not a fan of introducing a network call and a saas dependency, tho
- non-sensitive environment variables also need a home. I don't like the idea of cluttering the project with a .env.local or worse like .env.development.local
- I hear lots of secrets and config management happens in container orchestration, but what if I barely have a container in the first place? I guess docker compose is one of the simplest tools at this layer?
I guess I'm just looking for a safe, simple solution for a small team, and maybe the problem is that every team does things differently, and that many tools are marketed towards huge enterprise teams.
I don't see a reason to ever pass them. Sensitive production values should never be accessible to everyone, and for development the team should be able to create them themselves.
I usually keep very stupid values in .env.example, with the instructions to `cp .env.example .env`. In development it doesn't matter if the password is "password". In production the privileged admin sets up actual values.
> - non-sensitive environment variables also need a home. I don't like the idea of cluttering the project with a .env.local or worse like .env.development.local
`cp .env.example .env` and the application uses .env just like in production, only with dummy values.
https://12factor.net/
This was really popular, and lots of people bought the factor III of only configuring via environment variables.
The whole point of it was: keep config simple. Env vars are austere and will keep you from complexity.
But people found a way to complicate them, so they could have both the 12 factor literal pass and the complexity they wanted.
That resulted in the mess we are today.
Personally, I tend to have a `source ./source-me-auth` which does stuff like `DB_PASSWORD="$( pass show blah.com | head -1 )"`, but that still means I end up with secrets dangling around in my environment. The "source-me" file is safe to share, but my runtime environment is tainted (for good and bad...).
I saw something w.r.t. the way open-claw handles things where you basically say `export DB_PASSWORD="REDACTED@SECRET_001"` and then it basically outbound filters network requests to hydrate that with `s/REDACTED@SECRET_001/$REAL_SECRET/g`, and I _really_ like that mechanism b/c it defers secret usage to runtime and keeps it physically "out" of the application itself.
Basically, having `with $SECRETS -- some_app.sh --some params ...` wouldn't be terrible (conceptually).