Show HN: Dotenv, if it is a Unix utility (github.com)
I like the idea of using dotenv files, but I dislike having to use different language-specific libraries to read them.
To solve this, I created a small utility that lets you prefix any command with "dotenv" to load the ".env" file.
This is how I imagine dotenv would work if it had started as a UNIX utility rather than a Node.js library.
107 comments
[ 4.7 ms ] story [ 210 ms ] threadhttps://direnv.net/
From your description, direnv is implicit and noisy, whereas dotenv seems to be (unless you embed it in a script) explicit and quiet.
My assumption overall in this is that most people have just one .env per project (or perhaps in sub-folders per environment, e.g prod, staging, local), but these don't nest. With nested .env files, the mental overhead they bring remove (IMO) most of the benefits, if not more.
As an example of the difference, dotenv is useful for running programs inside Docker containers — which do not inherit your interactive shell's environment variables — whereas direnv isn't particularly useful there. Ditto for programs run via init systems like systemd or even classic SysV init. On the other hand, direnv is convenient for end-user env var config, since it's aware of your shell's working directory and updates the env vars based on it without needing to run extra commands.
https://github.com/ko1nksm/shdotenv
sh -c '. .env; echo $MY_VAR'
do the same thing? (I am not in front of a shell at the moment.)
I have this on my .bashrc:
source: [1]--
1: https://stackoverflow.com/a/60406814/855105
... except I'm thinking this may `set +a` if the environment already had `set -a`, which maybe could cause problems? I wonder if it would make sense to record the existing status of "-a" (allexports) an set it / unset it as necessary.
The code provided has a few potential issues, including security vulnerabilities:
Buffer Overflow and Memory Allocation Errors: The malloc function in read_file does not check if the memory allocation fails (it checks if buffer is NULL instead of buffer). This can lead to a null pointer dereference if malloc fails and returns NULL. There's a possibility of buffer overflow or improper handling if the file size read by ftell is exactly MAX_FILE_SIZE, because an additional byte is added (buffer = malloc(file_size + 1);). Improper Input Validation: The skip_until_charset and skip_charset functions do not check if the end of the buffer is reached within their loops. If the charset does not appear in the buffer, this could lead to reading beyond the buffer's limit, causing undefined behavior. Use of Uninitialized Variables: In the main function, the value_end pointer is set conditionally inside the if (cur == '"' || cur == '\'') block but used unconditionally outside it. This can lead to the use of an uninitialized pointer if the if block is not executed. Command Line Argument Handling: The check if (argc < 1) is unnecessary because argc will always be at least 1 in any C program, as it includes the name of the program itself. Insufficient validation of command arguments, specifically not handling scenarios where argv[1] might not be provided correctly or safely. Potential Command Injection: The execvp call uses user-supplied input (argv[1] and argv[2...]) without sanitization. If the environment or the .env file can be manipulated, it could lead to command injection vulnerabilities. Error Handling: The program's error messages (e.g., "Missing equal after key") are clear but the handling of such errors (e.g., continuing to parse or halting operations) may need to be revisited based on desired robustness.
Claude:
Here are a few potential security vulnerabilities and bugs I noticed in the code:
The read_file function does not properly handle the case where malloc fails and returns NULL. It should check if buffer is NULL after the malloc call and handle the error appropriately, likely by returning an error code. Currently it would dereference a NULL pointer. The code trusts the contents of the .env file. If an attacker can modify the .env file, they could potentially inject malicious values into environment variables that might be used in an unsafe way by the child process. There are no checks on the size of keys or values. A malicious .env file could contain extremely long keys or values that could potentially cause issues. The code uses strchr to look for meta characters like '=', '#', etc. If the .env file contains non-ASCII characters, this could potentially lead to unexpected behavior. The code uses execvp to execute the provided command with the parsed environment. If the command is provided by an untrusted source, this is a command injection vulnerability. The code should validate the command is on an allowlist before executing. The parsing allows for escape sequences in values, but doesn't have bounds checks. A maliciously crafted escape sequence could read or write out of bounds. The code will modify the process environment even if executing the final command fails. It may be better to parse into a separate data structure and only update the environment if the final execvp call will succeed. It's a good practice to set a proper process umask before writing files or creating directories accessible to other users on the system.
While the code looks generally well written, for a utility like this that parses untrusted input and executes commands, it requires very careful security review and hardening. I'd recommend a thorough security audit before using it on untrusted .env files or with untrusted commands. Adding additional input validation, using allowlists, and limiting the scope of what env vars and commands are allowed would help mitigate some risks.
These LLMs produce syntacticly valid language, no more. And information contained within is a by-product and not necessarily factual nor correct.
It’s not necessarily factual or correct, no. But it’s incredibly helpful and useful. As always, the real answer is somewhere in the middle.
Just don’t ask it about politics. It’s so blatantly obviously biased.
I don't it hard to believe that they truly believe they're doing us a favor. Surely they're doing it just to feel smart or included?
* MAX_FILE_SIZE: I don't think this is true.
* skip_until_charset, skip_charset bound check bug: I don't think this is true
* Uninitialized value_end: I don't think this is true - and if true should be caught by -Wall -Werror flags.
* argc < 1 check not being necessary: This is not true, you can make argc == 0 by using the exec family of libc functions.
* Error Handling: Currently all parsing errors should cause the program to exit, which I think is the desired behavior.
* Unsanitized input for .env: Intended behavior.
* Unsanitized input for execvp: Intended behavior.
It's a shame that running modern software requires carefully packaging a virtual environment and then injecting a bunch of ugly global env vars.
I still think Docker shouldn't exist. Programs should simply bundle their dependencies. Running a program should be as simple as download, unzip, run. No complex hierarchical container management needed.
Alas I am not King.
awww. i don't think it's OK in any way to download libc6/msvcrt as many times as I download __any__ software. even more, is there a strong difference between dependency and runtime environment? if sensible people does not bundle the whole python distribution to a "stuff.py" then why bundle libopenssl.so to a webserver application?
IMO, a saner approach would be just not to confuse dependencies: appX depends on libY 1.9; appZ depends on libY 2.0; people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames).
Programs should rely on the global runtime environment as little as possible
> if sensible people does not bundle the whole python distribution to a "stuff.py"
Unfortunately Python deployment is such a such an unmitigated disaster that it's a leading cause of Docker images.
Deploying a portable copy of Python is about 9 megabytes compressed. This is significantly preferable to multi-gigabyte Docker images.
> people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames).
You make a strong and compelling argument as to why programs should bundle their dependencies and not rely on the system environment.
Users should not have to perform any witchcraft to launch a program. Download and run. No further steps should be necessary.
The language runtime dotenv projects are banned in my engineering org.
Would love to hear more about why dotenv is banned at your org though.
I believe in convention over configuration. Most of our apps have hard-coded config, with a concise/short and finite number of things that can be overridden (like 3-4 parameters, tops). Secrets get injected.
I do subscribe to the idea of the 12 factor app, but there is a line that needs to be drawn between env config which is more dynamic and more persistent config that should be baked in to the release.
ETA: the main difference between `env` and `dotenv` seems to be that `env` gets its arguments from the command line, whereas `dotenv` gets its arguments from a file. I think that's a fair difference, but I might also think that perhaps `env` should expand its offering to include some kind of `-f filename` option so that it can focus on the notion of "a configurable sub-environment for a command" and we can avoid subtle distinctions.
Another advantage of env is that you can type `man env` and learn something useful; sourcing and subshells via syntax is a little bit harder.
Finally, I think the major point of this branch of the discussion is to explicitly decorate a command with a special environment. Starting up a subshell isn't the same thing. It might have the same effect, but you can see that you're creating a subshell, running a builtin in the subshell, and then running a command in the subshell. It is something of a difference between declarative (dotenv/env) and imperative (sourcing in a subshell) approaches, and inherits all the pros and cons of the imperative approach.
If it works for you, I make no recommendation against it.
The last one unsets the environment variables that were set by the first command, ensuring they are not persisted beyond the current shell session.
If you are worried about forgetting to execute it, there are a couple of ways to work around it, depending on your case.
`env -S "$(cat .env)" <cmd>`
Believe it or not that’s all you need.
> S, --split-string=S process and split S into separate arguments; used to pass multiple arguments on shebang lines
edit: forgot the quotes around shell substitution
The nice thing about utilities like env and dotenv is that they can be easily exec-ed:
-S is a fairly recently added option to the GNU Coreutils env (possibly inspired by BSD?). I have a window to an Ubuntu 18 VM where it's not available.You want $(cat .env) quoted, as in "$(cat .env)" so that the content of the file is reliably passed as one argument.
-S will split on whitespace; but it respects quoting, so spaces can be protected. Basically .env has to be prepared with the features of -S in mind. Of which that thing has quite a few: escape sequences like \n, commenting, environment variable substitution.
The .env being evaluated as a shell script means that it's in a widely used language, with a widely known syntax. You can look at it and know what it's going to do.
The .env being a data format to some uncommon utility; that's anyone's guess.
For instance, suppose we want a newline character in an environment variable. Does the given "env file" format support that? How?
There is one de-facto standard format: the /proc/<pid>/environ kernel output format on Linux. The variables are null-terminated strings, so it is effectively a binary format. It represents any variable value without requiring quoting mechanisms.
So your suggestion is to munge these for local development. And you're okay with that barrier? That's terrible dx, and it adds surface area for bugs.
The xargs idea made me think of using bash as the parser :
This test .bash file contains multiple source-s of other .bash files, which contain a mix of comments, functions, set and env vars - just the env vars are exported by env. This seems useful e.g. for collating & summarising an environment for docker run -e.This outputs the env vars to stdout; for the OP's purpose, the output could be sourced :
# For Bourne shell, use env -i in place of exec -c :sh -c "env -i sh -c '. $CONFIG/main.sh; env'" > $envFile
envup() {
}You can also specify `envup development` to load .env.development files should you want. Obviously this will pollute the current shell but for me it is fine.
[0] https://github.com/belthesar/sops-run
[1] https://github.com/getsops/sops
https://lists.gnu.org/archive/html/coreutils/2021-10/msg0000...
It came up in the mailing also this March. I saw the posting in my inbox and proposed that a null-terminated format be handled, which is exactly like /proc/<pid>/env:
https://lists.gnu.org/archive/html/coreutils/2024-03/msg0014...
If that feature were available, this becomes