But I keep putting more and more env vars into my programs. Enough command line switches already, but for debugging I always fall back to undocumented getenv's, rather than adding and removing silly -Dhexmasks.
Maybe I live under a rock, but I feel like command line utilities taking environment variable arguments just isn’t that common of a practice? The only examples I can think of are when an argument is more of a “global variable” that many different commands may find useful such as JAVA_HOME, GITHUB_TOKEN, etc. For web applications and docker containers, environment variables fit easily into the deployment process and for all intents and purposes, are pretty much static.
>Maybe I live under a rock, but I feel like command line utilities taking environment variable arguments just isn’t that common of a practice?
Nope, it's quite common. Many standard UNIX cli utilities take some environment variables (including things like LOCALE). Also common in scripts, setting up programming languages paths, etc.
It's also not a bad practice, contrary to what the author says.
These are good examples but I think these fit under my “global argument” umbrella. Imagine if every Linux command had a different argument name for the pager?
I disagree with this post so strongly - having spent most of my career installing, configuring, and managing other people’s software.
> The answer is that you, the end user, can not now. Every program is free to do its own thing and most do. If you have ever spent ages wondering why the exact same commands work when run from one terminal but not the other, this is probably why.
If the same program is behaving differently between two systems - it can -only- be the environment that’s different.
> Instead of coming up with a syntax that is as good as possible for the given problem, instead the goal is to produce syntax that is easy to use
Oh to be a developer. The “best possible syntax” is a universe of possibilities - environment variables are, thankfully, limited to strings. If all programs had to be configured with a Turing complete config language - that would just be a programming language! Limitations can set you free.
Sorry for the harsh tone. Please, and I believe I speak for most sysadmins, please continue to use environment variables.
I came here for this comment. I'm a developer and I have been using env vars for quite some time after I got burned numerous times by other options. I just can't see why I would use anything else.
Mostly configuration stored in the database (who configures the database) or 3rd party configuration services without an SLA and config files that are either present or not. Env vars are really simple since they are completely decoupled from the app and you can have default values for all of them. You just need a single class that loads all config on startup and you can go from there (or fail if you don't have the mandatory vars).
Environment variables are also more portable and cheaper/simpler than a DBMS, a LDAP service, an application server's proprietary configuration repository, etc. without being weaker at specifying simple configuration values.
In practical terms, specifying environment variables in cleanly isolated and composable layers (defaults by user, a specific terminal session, a script that call another script or the useful program) is a major advantage over more enterprisey and monolithic mechanisms.
I forgot the question mark: (who configures the database?). So it is about the problem that if you have your configs in a database then you will carry the additional burden to separately set up databases for each environment....it is like trying to put the hash of an image on the image.
The author's title is unfortunate: "Never use environment variables for configuration"
Not everyone is writing CLI scripts. Some are writing multi-environment software for the web. Some people care about git and distributed teams.
Let's say you backup and migrate a db which has config in the db. Well your other environment is using the wrong config! Now you're possibly using prod SMTP credentials and sending notifications to the wrong people because all you wanted to do was have live content and do some testing or show debug messages because you're on a test environment.
Web frameworks which have the HTTP host in the database drive me nuts. Why do you need that? Your webserver is responding on a domain name. Why do some store the absolute url in the db. It makes no sense. For the 3 people that want to serve up domainA.com but have all links readily be domainB.com maybe that makes sense.
Don't get burned people. For most things, just ignore this article, keep your secrets out of version control and keep your code ready to deploy to multiple environments. Decouple that. Your app should be able to work with various configurable services (SMTP, push notifications, database, etc) and you don't want that config in version control.
So either a file that's outside of version control with variables you set per environment or actual server environment variables.
Environment variables in many web languages are just namespaced globals. Still better than global variables. They serve a purpose.
While it helps the author's agenda, that's not a legitimate example.
A real example would be a service provider which a developer would understand to have the sole purpose of pulling from environment or config values to initialize.
You wanted a Twilio client? Well, we know it needs some keys. Use environment variables. Boom. Everywhere you ask for this Twilio client you get the same instance with the config that that environment needs.
This is not a problem. It works well. Better than any alternative.
Author also says this "Environment variables is exactly this: mutable global state."
For some CLI applications maybe. But for web languages this is not accurate. Some languages, yes, you can mutate the env variables at runtime. But your code shouldn't rely on any mutable env vars. You can load those env vars into a config. Even cache it. Not allow mutations. Many modern frameworks support this or you could implement it yourself.
And if you have a CLI that really needs to be explicit about the environment vars to run? And not be different the next time you call it with no env vars? Well maybe take those as arguments?
Plenty of CLI apps that store config in json and have a wizard to set those credentials. AWS CLI as an example.
Now you got JSON as the author wanted. That can work across platforms. But guess what? You work on multiple clients and hosts and next time you call it? Well you might not be expecting that it had config for another client or app.
Same problem. The state was mutated. You called it again. You potentially got burned. Now you have to do the CLI wizard to reconfigure. Or you pass in explicit params if allowed.
Definitely a consideration if you are writing such a program. Do you make it explicit with arguments and options? Load from ENV vars? Have some CLI wizard and save to JSON?
The author's suggestion of JSON is no reason to toss out ENV vars. Just solves slight differences between Windows and Unix. Which is why you can program a CLI wizard if you care about that problem.
We don't need to say no to env vars because we want CLI app users across different platforms to have the exact same API. Setting up a JSON file is really lame to use a program. Which is why you see cli wizards when you run them.
One of the projects I currently work on, the configuration system/model/table is a monster.. I wish it was just strings. It basically contains boolean flags, strings, numbers etc but the most insidious one is, it can contain groups of related configs - meaning people started dumping stuff into that should be a normal table (ex, ShippingType:, Road, Rail, Air), so we get no foreign key constraints for reference data. This caused them to implement soft-deletes for it, so now you have some config values that float around forever because they were referenced somewhere (aka pseudo-foreignkey). It's so utterly dumb I want delete the whole thing but everyone thinks it works great (non-tech people).
I'm a developer, not a system-admin, but this is too much. Doesn't help that 5 different people has added 'features' to it over the years. The same can be accomplished with something waaaay simpler/cleaner. We also keep having config-related issues where people blame the system/servers/devops etc... every single time it is due to misconfig the project, so the system admins cannot do anything about it anyways (meaning we keep wasting their time, thinking the problem was with the servers). Let me pause here, need to take a blood pressure pill.
100% this. The developer behind Prometheus was a huge dick to people about env vars a while back, in similar fashion. Just the other day, they held another closed-doors vote after a year or so and finally decided they were OK.
Doesn't surprise me the creator of Meson of all people made the same dogmatic assertion.
> The “best possible syntax” is a universe of possibilities - environment variables are, thankfully, limited to strings.
The same can be said for command line arguments. I usually prefer those instead of environment variables, because they must be explicitly specified instead of being implicitly passed by the parent process. I think of env vars as being more useful when repetitively calling commands interactively, to save some typing, or when you really do want processes to inherit config from parents (like PATH and its variants).
> If the same program is behaving differently between two systems - it can -only- be the environment that’s different.
Yes, but the entire problem is that "the environment" is massive as it includes all hardware and software running on the device in question (and quite possible other devices as the network can easily be considered part of "the environment") which makes it difficult to track down differing behaviour.
"The environment" is not just environment variables. I've run into a spreadsheet bug where I got wrong results because of a CPU bug. Just because some global mutable state exists, that doesn't mean it's a good software design to have program behaviour depend on it.
> Oh to be a developer. The “best possible syntax” is a universe of possibilities - environment variables are, thankfully, limited to strings. If all programs had to be configured with a Turing complete config language - that would just be a programming language! Limitations can set you free.
I was thinking about this while reading the post. What's the best syntax language or syntax for configuration, if environment variables are too simplistic and full-fledged programming languages are too powerful? That's why I'm interested in Dhall, which is a configuration language that's limited to compile-time only–i.e. it can't do anything at runtime.
>If the same program is behaving differently between two systems - it can -only- be the environment that’s different.
As a developer, I've generally used configuration files for changing the operation of my software and environment variables for information my code needs to know about WHERE the code is running. For example, the same code doing the exact same thing on 10 machines would have the same configuration file (or command line parameters in simpler cases) but the environment variables may change from machine to machine.
Author doesn't know about https://12factor.net and as another commenter mentioned, probably hasn't deployed something to a 'production' environment (or rather, doesn't know about separation of such environments in the first place).
Well, sure, you shouldn't be putting secrets or other sensitive data in environment variables. But garden-variety configuration is fine to put in env vars. Seems like whoever assigned you this task didn't really know what they were doing.
Not in any multi-user multi-process OS. You set environment variables in a process (ie. shell/CMD.EXE) and spawn child process (the program) from that parent. The environment variables will only be visible to those two processes.
On Solaris/SunOS, you could use `pargs -e $PID`. And so on.
Having separate UIDs to run your processes A and B under shields either one from peeking at the other's environment, though. UNIX DAC is simple and powerful enough for MOST security concerns, I would argue.
Avoiding environment variables reduces the risk but doesn't eliminate it. The secrets still live in memory in some form, correct? However, it does help to eliminate generic attempts to exfiltrate environment variables.
Tight control of egress network traffic is better but more difficult to implement.
To be fair, "configuration" means a somewhat different thing when we're talking about a user application vs a server. It's easy to forget on HN that some software engineers don't write web servers at all.
Having used the profiling tool TAU (subtle dig), I instantly understood what the author is driving at, and I somewhat agree for many use cases. I shouldn't have to fill my dotfiles with 10 new variables just to use a utility.
It looks like the author is talking about command line tools that use env vars for things that should be arguments. In the comments on the page he admits that for example key credentials are valid usages for env vars.
Meh. Most configuration libraries I've used make all of these issues into non-issues. It's very useful to be able to set configuration with environment variables, even if you're loading from a file.
I could agree with "don't only use environment variables for configuration", but at the same time, by all means do use environment variables to allow for easy overriding of configuration.
I cannot but think of the large number of times that being able to quickly override some parameter with an env var has helped me achieve something which was not exactly intended by the original author.
Also, env vars are the most common way to override configuration of software that has been dockerized (prepared to run in a Docker container). Plus, containers remember their initial environment, so there is no issue of running afterwards without the env var.
Changing a variable of an existing container without recreating or re-running the docker run command is tricky, though. With a configuration file mapped to an external folder it's "just" modifying the file and docker restart, with environment variables it's somewhat more convoluted. I don't think it is as clear-cut as the title suggest, each kind of configuration has its place.
That's not what I'm saying, what you mention can be done indeed.
What I say is that if you are in a machine with an already running docker container, one you don't have the corresponding `run` command of (i.e. you _can't docker run_ for whatever reason), changing the environment variables of that container is tricky.
You could always run the `env` command inside of the container to get all of the set environment variables. Then you can define, reconstruct and run your container with whatever args you want.
But yeah the env_file approach is the way to go here. I've been using Docker in production since 2015 and never ran into your use case. I always had an .env file ready to go that was loaded in with env_file and always had control over being able to run the container with whatever command I see fit.
And in cases where I have no control over how things are run (like Heroku), environment variables still work because a ton of hosting platforms expect you to set and read them for various configuration.
And you can also commit an example env file to git with no secrets so developers can `cp .env.example .env` to get going in 1 second.
The environment variable pattern is incredibly standard in the world of web dev.
Yes, that was my point actually, although I might have expressed it poorly:
The article says that one problem of using env vars is that you might set it for one run, but then forget to set it for the next one. This could be problematic if the program is stateful. So I wanted to make the passing comment, while talking about containers, that a container will "remember" the initially set env vars, so if you stop and start the process, the variables would persist across executions.
Indeed, that's actually an awesome feature because guarantees the container will work as it was running upon a restart.
Problem is when you have an environment variable you want to change for some reason without triggering a redeployment/release/whatever, with a configuration file in a volume you can change it and restart (of course this breaks the premise of "restart keeps full state", but I can live with file modifications, with docker at least you can keep them at least "triggered externally"). With an environment variable the only option I have found is stopping docker, digging into the container configuration, changing said variable and starting docker again (lovingly called "Indy swap" when we've had to do it, luckily it's less than once a year).
The article is not against env vars per se as the title suggests. Instead it makes a case against undocumented features and improper use of env vars. It also complains about env vars being plain strings but fails to recognize config files are just plain strings without the appropriate parser. What stops anyone from putting JSON arrays in env vars?
The best argument, perhaps only valid argument, is lack of an array type. Easy to work around. The rest seems misguided or ridiculous.
>There is no way to know which one of these is the correct form
What? Of course there is.
The author is also calling it mutable global state, and seems to reference an application becoming confused when ENV isn't set. This reads to me, perhaps incorrectly, that the author doesn't understand that envvars aren't and don't behave like shared global variables. That is, changing one won't affect running applications.
Easy fix (if it's the shared, mutable state which bugs you):
* Create one class responsible for ingesting env vars at startup.
* Call it from main, and abort early with nice messages if it fails to read something.
* Now you have a nice (preferably immutable) class which guarantees the config is in a 'good state', and is self-documenting because it lists all the keys it uses to lookup env vars with.
This is essentially what I do in Rails apps. The only reference to an env var is in an initializer that sets an option in the global rails config structure.
In node.js there are quite a few packages that do exactly this - it's a great pattern and forces you to define which env values you will rely on in one place, rather than dripping them all over your codebase.
The mutable state can be helpful. It is sometimes helpful to be able to change an app’s config without having to restart it. Ingesting the envs on startup into a class removes this ability.
It's ok for interactive applications, but if you are writing a CLI command (what is different from an interactive CLI application), a system library or a deamon, don't ever let the same application that uses a configuration also change it.
When your non-interactive programs do that and anything at all goes wrong, it's basically impossible to determine the source of the problem. Also, it is common that bugs that one could just avoid triggering by configuration now become unavoidable.
(But if you mean reload the config after getting a SIGHUP or something like that, yeah, this is ok, and the best way to do that is by restarting everything on your program, even if you keep the same process, so your read-once class won't be a problem.)
If The same pattern works well in python at the module level, if your application is setup as a package. A module config.py sets a bunch of python variables like
import os
ENV_VAR=os.environ.get('ENV_VAR', default_value)
then the rest of the application can grab configuration with
from .config import ENV_VAR
Since the assignment code executes on import, all config is read in when any piece of it is first used, consistency checks and logging can be written into the config.py module as normal python statements, config values can be cast to appropriate types (raising exceptions if they fail), etc.
After years of struggling with config file in heterogenous production environments, I'll argue the opposite: environment variables are the BEST option to manage your configuration.
Mixed feelings about this. I strongly agree with some, but also feel it's missing the point in a lot of places.
Environment vars should of course not be used to configure specific programs. Having an environment variable to specify the args with which to call a program is needless complication. Just pass it as an argument. Use a configuration file to configure the program.
Environment variables should (only) be used to describe the environment. That should primarily be variables that transcend individual programs. Things like proxy settings are perfect for environment variables. (Well, almost; see below.) Perhaps the location of the configuration file, if that can vary per system (which it probably should be able to; hard-coded locations can also be a problem).
But even then, environment variables can fail. I noticed that some Azure/Kubernetes-related commands on my work Macbook need to run with the proxy on, and others with the proxy off, so I created aliases to enable/disable this environment variable, which completely defeats the purpose of the environment variable. Maybe I should be able to configure this proxy per application after all. Or at least configure whether to use or ignore the proxy settings. And then there are applications that ignore the proxy env var for whatever reason, and require me to configure it specifically for that one program, again defeating the purpose of environment variables (I think npm does this).
But when we deploy our application to different environments, our deployment configuration does set specific env vars so the application knows how to behave in that environment. It's what environment variables are for. But they're not a great fit. For example, we're currently in the process of migrating from AWS to Azure, and some things need to be enabled or disabled there. So we set some environment variable to 'false', except that environment variables are always strings, and in javascript, 'false' evaluates to true. Json configuration might actually make more sense here.
So I don't think we can or should do without environment variables, but I think I agree they're overused, and often used badly.
Yeah, we have a service that needs to use a proxy for most calls, but one component mustn't use the proxy, in a way that iirc is not properly captured by a `NO_PROXY` entry. Now the abstraction breaks down and the service has some ugly special-case code. :(
I wish there was wider adoption of configuration services. Something like Enovy xDS protocol, but for general configurations.
Puppet & Co did it to some extent, but for the whole server. On boot it asks centralized service for a configuration, providing just server identity and some basic attributes (datacenter, rack, stage) and receives full server configuration. Implementing this pattern for a service itself, would make it easier to configure swarms of services.
I experimented with using Open Policy Agent, where it dispatches exact configuration based on client identity and quite liked the result. Only downside is that it requires polling.
> Environment variables is exactly this: mutable global state. Envvars have some legitimate usages (such as enabling debug logging) but they should never, ever be used for configuring core functionality of programs
Doesn’t matter where configuration lives, the name itself suggests that it is by definition global mutable state, I mean that’s the whole point of it.
And it doesn’t matter if it’s used to configure core functionality (e.g feature toggles) or secondary functionality. The whole purpose is to do exactly that :)
What I mean is that when an advice is expressed in absolute it tends to be a radical view on the problem.
Radicals tend to dismiss things that don't agree with their world view and I am always wary of this. And you should to. I don't mean radicals can't be right. What I mean is you should be cautious about it.
It indeed seems like there are a lot of fundamental misconceptions in this article:
> For comparison using JSON configuration files this entire class os problems would not exist.
This is the trade-of between structure and embedding. Writing '{"ARG": "-Dfoo=\"bar bar\" -Dbaz"}' would satisfy the authors need for JSON but not really do any difference.
The initial example is also wrong. There certainly are example where one would use variables defined in out scope.
IMO, the author is writing from the perspective of Meson, a command line build tool used by individuals that takes arguments and caches them in a per-project file, whereas most of the negative replies are commenting from the perspective of sysadmins deploying software into homogeneous servers or Docker containers. Would make be a better program if MAKEFLAGS was not an environment variable? (IDK.) Would Git be a better program if the project directory was passed as an argument rather than as inherited state (the cwd)? (IDK.) Would less or nano be a better program if file paths were passed in as environment variables rather than arguments? (no.)
Wait, make has a global state? I was under the impression most make targets are stored somewhere in the project itself (like a build/ directory). I think this is a fine approach, BTW. I think global state should be avoided unless necessary.
Make does what you tell it to do. While some projects have the setup that you've described, many others do not. Make will only check the timestamps of the generated assets and decide to build (or not) based on that.
CMake does that and is a truly demented implementation, mixing user-specified initial state (supplied the first time you run CMake and carried on to future argument-less invocations), derived state, and cached compiler locations and versions in a single CMakeCache.txt file. Edit CMakeLists.txt? Time to delete CMakeCache.txt! Upgrade your compiler? Time to delete CMakeCache.txt!
I haven't used Meson all that much, but I recall it's a bit better than CMake but I still ran into a similar issue at one point.
Oh this guy again. This guy created Meson and has a pattern of being 1) Quite toxic and 2) Entirely dogmatic when it comes to software design. He shows no interest in discussing design problems with Meson and asserts his viewpoints as truth and fact, resorting to snide comebacks instead of having thoughtful conversation.
Doesn't surprise me he wrote an article like this. Completely misguided and isn't rooted in reality.
Okay, so when you write that this blog author, who made a post arguing how environmental variables are global mutable state, is quite toxic and resorting to snide comebacks instead of having thoughtful conversation, then that is just you engaging in thoughtful conversation about the issue (which is envvars), and not you making a toxic ad-hominem attack at all, right?
It surprises me that people put more attention to the author than to the content. I have no idea who the author is, but his post doesn't seem to me toxic at all nor dogmatic.
On the other hand, your comment sounds a bit toxic, to be honest: "because the author is X it must be that all of his articles are X as well".
295 comments
[ 4.5 ms ] story [ 283 ms ] threadI have no issue with env vars for read only configuration. Programs that set env vars... thats where it gets very ugly.
Nope, it's quite common. Many standard UNIX cli utilities take some environment variables (including things like LOCALE). Also common in scripts, setting up programming languages paths, etc.
It's also not a bad practice, contrary to what the author says.
LC_ALL=C foo
PAGER=cat man ls
TZ=UTC date
I don't think picking apart the (assumed) experience of the author in this way is conducive to a high quality discussion.
> The answer is that you, the end user, can not now. Every program is free to do its own thing and most do. If you have ever spent ages wondering why the exact same commands work when run from one terminal but not the other, this is probably why.
If the same program is behaving differently between two systems - it can -only- be the environment that’s different.
> Instead of coming up with a syntax that is as good as possible for the given problem, instead the goal is to produce syntax that is easy to use
Oh to be a developer. The “best possible syntax” is a universe of possibilities - environment variables are, thankfully, limited to strings. If all programs had to be configured with a Turing complete config language - that would just be a programming language! Limitations can set you free.
Sorry for the harsh tone. Please, and I believe I speak for most sysadmins, please continue to use environment variables.
What kind of ways did you get burned by other things than env vars in a way in which env vars would not?
In practical terms, specifying environment variables in cleanly isolated and composable layers (defaults by user, a specific terminal session, a script that call another script or the useful program) is a major advantage over more enterprisey and monolithic mechanisms.
or, what configures the database connection
The author's title is unfortunate: "Never use environment variables for configuration"
Not everyone is writing CLI scripts. Some are writing multi-environment software for the web. Some people care about git and distributed teams.
Let's say you backup and migrate a db which has config in the db. Well your other environment is using the wrong config! Now you're possibly using prod SMTP credentials and sending notifications to the wrong people because all you wanted to do was have live content and do some testing or show debug messages because you're on a test environment.
Web frameworks which have the HTTP host in the database drive me nuts. Why do you need that? Your webserver is responding on a domain name. Why do some store the absolute url in the db. It makes no sense. For the 3 people that want to serve up domainA.com but have all links readily be domainB.com maybe that makes sense.
Don't get burned people. For most things, just ignore this article, keep your secrets out of version control and keep your code ready to deploy to multiple environments. Decouple that. Your app should be able to work with various configurable services (SMTP, push notifications, database, etc) and you don't want that config in version control.
So either a file that's outside of version control with variables you set per environment or actual server environment variables.
Environment variables in many web languages are just namespaced globals. Still better than global variables. They serve a purpose.
The author has this:
int first_argument;
int second_argument;
void add_numbers(void) { return first_argument + second_argument; }
While it helps the author's agenda, that's not a legitimate example.
A real example would be a service provider which a developer would understand to have the sole purpose of pulling from environment or config values to initialize.
You wanted a Twilio client? Well, we know it needs some keys. Use environment variables. Boom. Everywhere you ask for this Twilio client you get the same instance with the config that that environment needs.
This is not a problem. It works well. Better than any alternative.
Author also says this "Environment variables is exactly this: mutable global state."
For some CLI applications maybe. But for web languages this is not accurate. Some languages, yes, you can mutate the env variables at runtime. But your code shouldn't rely on any mutable env vars. You can load those env vars into a config. Even cache it. Not allow mutations. Many modern frameworks support this or you could implement it yourself.
And if you have a CLI that really needs to be explicit about the environment vars to run? And not be different the next time you call it with no env vars? Well maybe take those as arguments?
Plenty of CLI apps that store config in json and have a wizard to set those credentials. AWS CLI as an example.
Now you got JSON as the author wanted. That can work across platforms. But guess what? You work on multiple clients and hosts and next time you call it? Well you might not be expecting that it had config for another client or app.
Same problem. The state was mutated. You called it again. You potentially got burned. Now you have to do the CLI wizard to reconfigure. Or you pass in explicit params if allowed.
Definitely a consideration if you are writing such a program. Do you make it explicit with arguments and options? Load from ENV vars? Have some CLI wizard and save to JSON?
The author's suggestion of JSON is no reason to toss out ENV vars. Just solves slight differences between Windows and Unix. Which is why you can program a CLI wizard if you care about that problem.
We don't need to say no to env vars because we want CLI app users across different platforms to have the exact same API. Setting up a JSON file is really lame to use a program. Which is why you see cli wizards when you run them.
Or things like "aws configure". And you still ...
One of the projects I currently work on, the configuration system/model/table is a monster.. I wish it was just strings. It basically contains boolean flags, strings, numbers etc but the most insidious one is, it can contain groups of related configs - meaning people started dumping stuff into that should be a normal table (ex, ShippingType:, Road, Rail, Air), so we get no foreign key constraints for reference data. This caused them to implement soft-deletes for it, so now you have some config values that float around forever because they were referenced somewhere (aka pseudo-foreignkey). It's so utterly dumb I want delete the whole thing but everyone thinks it works great (non-tech people). I'm a developer, not a system-admin, but this is too much. Doesn't help that 5 different people has added 'features' to it over the years. The same can be accomplished with something waaaay simpler/cleaner. We also keep having config-related issues where people blame the system/servers/devops etc... every single time it is due to misconfig the project, so the system admins cannot do anything about it anyways (meaning we keep wasting their time, thinking the problem was with the servers). Let me pause here, need to take a blood pressure pill.
The article's point stands if we're being lame and treating internal code details as external.
Doesn't surprise me the creator of Meson of all people made the same dogmatic assertion.
What a circus this industry has become.
I just learned about envsubst https://www.gnu.org/software/gettext/manual/html_node/envsub...
The same can be said for command line arguments. I usually prefer those instead of environment variables, because they must be explicitly specified instead of being implicitly passed by the parent process. I think of env vars as being more useful when repetitively calling commands interactively, to save some typing, or when you really do want processes to inherit config from parents (like PATH and its variants).
But overall I still agree with your sentiment.
Yes, but the entire problem is that "the environment" is massive as it includes all hardware and software running on the device in question (and quite possible other devices as the network can easily be considered part of "the environment") which makes it difficult to track down differing behaviour.
"The environment" is not just environment variables. I've run into a spreadsheet bug where I got wrong results because of a CPU bug. Just because some global mutable state exists, that doesn't mean it's a good software design to have program behaviour depend on it.
I was thinking about this while reading the post. What's the best syntax language or syntax for configuration, if environment variables are too simplistic and full-fledged programming languages are too powerful? That's why I'm interested in Dhall, which is a configuration language that's limited to compile-time only–i.e. it can't do anything at runtime.
As a developer, I've generally used configuration files for changing the operation of my software and environment variables for information my code needs to know about WHERE the code is running. For example, the same code doing the exact same thing on 10 machines would have the same configuration file (or command line parameters in simpler cases) but the environment variables may change from machine to machine.
But I was assigned a task last year to remove configuration from environment variables (for security reasons). I deployed my work to 'production'.
Found this:
https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-...
https://security.stackexchange.com/questions/197784/is-it-un...
On Solaris/SunOS, you could use `pargs -e $PID`. And so on.
Having separate UIDs to run your processes A and B under shields either one from peeking at the other's environment, though. UNIX DAC is simple and powerful enough for MOST security concerns, I would argue.
This is completely false in any modern OS. You can only see environment variables of your own processes.
(I wish, it's April 1 after all!)
Tight control of egress network traffic is better but more difficult to implement.
Configuration files are resistant to this as you note, but command-line arguments are not (--password=1234 will show up in ps for everyone).
Having used the profiling tool TAU (subtle dig), I instantly understood what the author is driving at, and I somewhat agree for many use cases. I shouldn't have to fill my dotfiles with 10 new variables just to use a utility.
I think you're wrong about that. The blog post author is also author of Meson, the build system.
I cannot but think of the large number of times that being able to quickly override some parameter with an env var has helped me achieve something which was not exactly intended by the original author.
Also, env vars are the most common way to override configuration of software that has been dockerized (prepared to run in a Docker container). Plus, containers remember their initial environment, so there is no issue of running afterwards without the env var.
Also if you specifically want to modify a file and re-run the container look at --env-file arg to docker run.
What I say is that if you are in a machine with an already running docker container, one you don't have the corresponding `run` command of (i.e. you _can't docker run_ for whatever reason), changing the environment variables of that container is tricky.
But yeah the env_file approach is the way to go here. I've been using Docker in production since 2015 and never ran into your use case. I always had an .env file ready to go that was loaded in with env_file and always had control over being able to run the container with whatever command I see fit.
And in cases where I have no control over how things are run (like Heroku), environment variables still work because a ton of hosting platforms expect you to set and read them for various configuration.
And you can also commit an example env file to git with no secrets so developers can `cp .env.example .env` to get going in 1 second.
The environment variable pattern is incredibly standard in the world of web dev.
The article says that one problem of using env vars is that you might set it for one run, but then forget to set it for the next one. This could be problematic if the program is stateful. So I wanted to make the passing comment, while talking about containers, that a container will "remember" the initially set env vars, so if you stop and start the process, the variables would persist across executions.
Problem is when you have an environment variable you want to change for some reason without triggering a redeployment/release/whatever, with a configuration file in a volume you can change it and restart (of course this breaks the premise of "restart keeps full state", but I can live with file modifications, with docker at least you can keep them at least "triggered externally"). With an environment variable the only option I have found is stopping docker, digging into the container configuration, changing said variable and starting docker again (lovingly called "Indy swap" when we've had to do it, luckily it's less than once a year).
The best argument, perhaps only valid argument, is lack of an array type. Easy to work around. The rest seems misguided or ridiculous.
>There is no way to know which one of these is the correct form
What? Of course there is.
The author is also calling it mutable global state, and seems to reference an application becoming confused when ENV isn't set. This reads to me, perhaps incorrectly, that the author doesn't understand that envvars aren't and don't behave like shared global variables. That is, changing one won't affect running applications.
* Create one class responsible for ingesting env vars at startup.
* Call it from main, and abort early with nice messages if it fails to read something.
* Now you have a nice (preferably immutable) class which guarantees the config is in a 'good state', and is self-documenting because it lists all the keys it uses to lookup env vars with.
Immutable config via a config class that can exit early (prefereably startup) if there is a misconfiguration
It's ok for interactive applications, but if you are writing a CLI command (what is different from an interactive CLI application), a system library or a deamon, don't ever let the same application that uses a configuration also change it.
When your non-interactive programs do that and anything at all goes wrong, it's basically impossible to determine the source of the problem. Also, it is common that bugs that one could just avoid triggering by configuration now become unavoidable.
(But if you mean reload the config after getting a SIGHUP or something like that, yeah, this is ok, and the best way to do that is by restarting everything on your program, even if you keep the same process, so your read-once class won't be a problem.)
Environment vars should of course not be used to configure specific programs. Having an environment variable to specify the args with which to call a program is needless complication. Just pass it as an argument. Use a configuration file to configure the program.
Environment variables should (only) be used to describe the environment. That should primarily be variables that transcend individual programs. Things like proxy settings are perfect for environment variables. (Well, almost; see below.) Perhaps the location of the configuration file, if that can vary per system (which it probably should be able to; hard-coded locations can also be a problem).
But even then, environment variables can fail. I noticed that some Azure/Kubernetes-related commands on my work Macbook need to run with the proxy on, and others with the proxy off, so I created aliases to enable/disable this environment variable, which completely defeats the purpose of the environment variable. Maybe I should be able to configure this proxy per application after all. Or at least configure whether to use or ignore the proxy settings. And then there are applications that ignore the proxy env var for whatever reason, and require me to configure it specifically for that one program, again defeating the purpose of environment variables (I think npm does this).
But when we deploy our application to different environments, our deployment configuration does set specific env vars so the application knows how to behave in that environment. It's what environment variables are for. But they're not a great fit. For example, we're currently in the process of migrating from AWS to Azure, and some things need to be enabled or disabled there. So we set some environment variable to 'false', except that environment variables are always strings, and in javascript, 'false' evaluates to true. Json configuration might actually make more sense here.
So I don't think we can or should do without environment variables, but I think I agree they're overused, and often used badly.
Puppet & Co did it to some extent, but for the whole server. On boot it asks centralized service for a configuration, providing just server identity and some basic attributes (datacenter, rack, stage) and receives full server configuration. Implementing this pattern for a service itself, would make it easier to configure swarms of services.
I experimented with using Open Policy Agent, where it dispatches exact configuration based on client identity and quite liked the result. Only downside is that it requires polling.
The same is true of command-lines and files. Computers can only use bytes.
Now in the outmost layer where you start your program you get the environmental variables and pass them.
I want to use env vars because:
* I want to reuse builds and my systems have different behavior on different environments.
* I want to override the default config sometimes
* I don't want to expose sensitive configurations, and yet have them explicitly configure in my project configuration
The post it's very opinionated and lacking any valuable point IMO.
Doesn’t matter where configuration lives, the name itself suggests that it is by definition global mutable state, I mean that’s the whole point of it.
And it doesn’t matter if it’s used to configure core functionality (e.g feature toggles) or secondary functionality. The whole purpose is to do exactly that :)
Radicals tend to dismiss things that don't agree with their world view and I am always wary of this. And you should to. I don't mean radicals can't be right. What I mean is you should be cautious about it.
> For comparison using JSON configuration files this entire class os problems would not exist.
This is the trade-of between structure and embedding. Writing '{"ARG": "-Dfoo=\"bar bar\" -Dbaz"}' would satisfy the authors need for JSON but not really do any difference.
The initial example is also wrong. There certainly are example where one would use variables defined in out scope.
That is literally one of the worst ideas I’ve ever heard of. And I’ve heard many bad ideas recently.
I haven't used Meson all that much, but I recall it's a bit better than CMake but I still ran into a similar issue at one point.
But you do not automatically add them to the configuration file unless the user explicitly tells you to do so.
Well, it can do both :) `--git-dir=<path>` will let you run git for another project directory
Doesn't surprise me he wrote an article like this. Completely misguided and isn't rooted in reality.
On the other hand, your comment sounds a bit toxic, to be honest: "because the author is X it must be that all of his articles are X as well".