This looks interesting and potentially really helpful! I have been keeping my eyes open for something along these lines. As the author, how do you see this project's relation to Conda? And if you see it working alongside of, rather than in place of, Conda, what support does it have for recognizing and using Conda envs?
Aside : Can someone explain how HN rises topics to the top? (I'm genuinely curious)
As of writing this comment, this submission has 4 points in 3 hours, and is on the top half of the front page. (Not saying the project doesn't shouldn't be - looks pretty awesome tbh).
Does the reputation of the people upvoting it matter? (still surprising it'd rise given only 4 points in 3 hours). Does the reputation of the person submitting it matter?
Not sure but I feel they've fine tuned it more so that posts like this one don't go by completely unnoticed. I also believe the comments are factored into the ranking as well.
Docker might replace the need for the virtual environment, but doesn't manage the components of a Python project (building, testing, installing and updating dependencies, etc).
Plus, a virtualenv is a much simpler tool, it's essentially just a way to tell the Python interpreter from which directories to load modules, so it's less likely to give you problems.
It doesn't. Imagine you just want to run some utility written in Python (let's use youtube-dl since it is been on front page lately), why do I need to install Docker to run a simple command? For isolation? No thanks. Virtualenv was created to isolate overwrittkng and installing dependencies.
Also, want to emulate multi-sites on single machine deployment? You need virtualenv.
Docker does solve those problems, even if you and I prefer to use other solutions.
And by the way, you don't actually need virtualenv :) we run multiple webapps on a machine by just installing dependencies in a directory (pip -t) in the git repo, and then adding the appropriate PYTHONPATH in the service's startup config. As long as you don't need multiple versions of the Python interpreter, this works fine.
Is it really computationally expensive? Containers/jails/zones are essentially chroots "on steroids", isolating things other than the filesystem (networking, hostname, process IDs, users, etc.)
Once built a docker container is not expensive, but the idea with Docker is to create immutable, layered containers and rebuilding every time there is a change. Unlike with nodejs, it's harder to volume mount python packages installed in the docker container and so no one does it.
Change in requirements means rebuilding the container which is a very computationally and IO expensive process. So if you can get away with it virtualenv is better, but putting everything in a container so you can simply ship a container is also useful. It's a compromise.
A layer with your pip requirements being added and then installed is the most expensive part of building a python container so if you change your requirements you invalidate the cache for that layer (and possibly the other layers above).
Developing inside a docker image seems like needless complexity to me, especially when virtualenv takes care of the python environment isolation already without another layer of indirection. Virtualenvs are just file copies and PATH vars with some tooling.
It's still not /great/. I have had to switch from my mac to a new Win10 laptop for development because the app server in my docker container would take around 2 mins to detect a change and reboot the app server, which then would take a few minutes to reboot. Now it's much closer to realtime. I do hope that docker get's it sorted out soon, as the tooling on windows isn't as good as what I am used to on osx.
And by tooling, I mean command line. I am using Cmder with Linux subsystem for Windows, and it STILL isn't half as good as even the terminal built into osx, let alone iterm.
Maybe if what you are building runs comfortably in one app, on one server, with no important discrepancies between your dev system and production.
The environment I work with has docker containers for
Web App Server
Background Task Worker Server
Database
Redis Node
Cassandra Node
Getting all of these to work nicely together on linuxes/macOS/Windows in a repeatable fashion across multiple developers is a major headache. Being able to spin the whole cluster up with `docker-compose up -d` turns the whole mess into merely a minor headache.
Most of the time, I only need to work on functionality that depends on DB and WEB nodes, but if I ever need to debug some issue with the worker tasks, it's much easier to have a whole working system available.
I don't want to be suspicious and negative but here is yet another needless dependency without any real value. mkvirtualenv is the best tool to manage venvs, pytest with pytest-cov to test and there is so much benefit to know original instruments but not some inferior wrapper. If you bothered by bugs in pip it's better to fix it in pip. All community will be grateful. But another wrapper is a complete shame.
easy_install: Forget it existed, substitute with pip
virtualenv: A virtual environment, providing an isolated Python and libraries
mkvirtualenv: A script to create a virtualenv
pipenv: Combines pip and virtualenv into an integrated solution, comes with a Pipfile not unlike Ruby's Gemfile for example
pyenv: Script that helps to install and manage different Python versions on your system, so you can have the full range of Python 2.x and 3.x installed without them getting in the way of each other. Allows you to set local, default and system Pythons allowing you to switch your Python interpreter depending on which project/directory you're in
I use Python daily and have never heard of mkvirtualenv. I started using pyenv a couple years ago and it worked well enough that I stayed with it. It's really a shame that there is no source of truth for packaging in the Python world.
I don't understand this mindset. There are millions of python developers using this language in for everything from from small shell scripts, web scrapers, to apps and. machine learning models.
Yet there is a monolithic additude towards tooling for languages, so much so that experimentation is actively discouraged. In my experience, there is no better way to level up as a developer than releasing and supporting an actual product that solves what you see as problems. Sure, you can contribute to existing projects, but you're going to learn a lot more if you are the owner. This experience, in and of itself, makes it worth it. I don't understand the existential fear and admonishment since the 'superior' tool will be most widely used. Also, there are many more developers today of all stripes and skill levels, more than enough to support the existing toolset, and others that like to play around and experiment with different projects for fun. This is somewhat how biological evolution works... lots of experiments(mutations), almost all are dead ends, but sometimes, that experiment leads to a wildly successful adaptation.
I only have about 4 years experience, but I've never understood this mindset. I see the value of having a primary toolset for a language, but don't see the value of having onw and only one solution. Can someone please explain to me its value?
The README.rst documents the `egg` subcommand like this:
> Creates a new Python project. Think of an "egg" as a new idea.
How does this differ from the egg distribution format [1] introduced by setuptools, which wheel aims to replace? If it's completely different, couldn't a distinct term be used? Python packaging is confusing enough as it is!
From what I can tell, the egg subcommand doesn't create any kind of egg file, just creates a new project.
But I agree. Creating a new distinct keyword for that command (why not just `hatch new my-app`?) would be far better. I've tried to get into Python packaging, and it's the biggest PITA I've dealt with as a dev.
I've stopped using any other virtual env managers than asdf[0]- it has one simple API for pretty much everything.
You can define a .tools-versions file in the root of your project, which lets you install the correct versions with a simple `asdf install`. We even use this in CI for zero-config language version upgrades.
I read "asdf" and wondered, "Using Common Lisp to configure Python?" Then I followed the link and am curious whether or not it is a port of the widely used (in) Common Lisp tool.
68 comments
[ 3.2 ms ] story [ 132 ms ] threadI started reading the readme, and after getting past general "this is awesome", I got to the doc of the config option...
How about showing an example usage, which does something simple and cool?
If I had to say what is truly novel here, it would be the virtual env management and shell support.
Hatch can replace Conda as a virtual env manager but doesn't really try to compete with it in regards to binary installs.
Hatch does try to detect when a Conda env is in use and will adjust the shell prompt accordingly. https://github.com/ofek/hatch/blob/59cfdd4a929240dd82b2e4e25...
As of writing this comment, this submission has 4 points in 3 hours, and is on the top half of the front page. (Not saying the project doesn't shouldn't be - looks pretty awesome tbh).
Does the reputation of the people upvoting it matter? (still surprising it'd rise given only 4 points in 3 hours). Does the reputation of the person submitting it matter?
And don't use random topics to ask questions like these, that's what Ask HN can be used for: https://news.ycombinator.com/ask
Plus, a virtualenv is a much simpler tool, it's essentially just a way to tell the Python interpreter from which directories to load modules, so it's less likely to give you problems.
Also, want to emulate multi-sites on single machine deployment? You need virtualenv.
And by the way, you don't actually need virtualenv :) we run multiple webapps on a machine by just installing dependencies in a directory (pip -t) in the git repo, and then adding the appropriate PYTHONPATH in the service's startup config. As long as you don't need multiple versions of the Python interpreter, this works fine.
...with a very computationally expensive solution, and without improving other aspects like security.
A systemd unit on a "chroot" can do the same and provide isolation.
Change in requirements means rebuilding the container which is a very computationally and IO expensive process. So if you can get away with it virtualenv is better, but putting everything in a container so you can simply ship a container is also useful. It's a compromise.
And by tooling, I mean command line. I am using Cmder with Linux subsystem for Windows, and it STILL isn't half as good as even the terminal built into osx, let alone iterm.
The environment I work with has docker containers for
Getting all of these to work nicely together on linuxes/macOS/Windows in a repeatable fashion across multiple developers is a major headache. Being able to spin the whole cluster up with `docker-compose up -d` turns the whole mess into merely a minor headache.Most of the time, I only need to work on functionality that depends on DB and WEB nodes, but if I ever need to debug some issue with the worker tasks, it's much easier to have a whole working system available.
[0]: https://github.com/sdispater/poet
- comparison with pipenv
- how it handles things like `pip freeze` for, e.g., Heroku
More or less, hatch manages packages based on environments, whereas pipenv does so based on projects.
When pip supports Pipfiles, hatch will as well. At that point, project based package management will be supported too.
Eventually, yes.
pip
easy_install
virtualenv
mkvirtualenv
pipenv
pyenv
easy_install: Forget it existed, substitute with pip
virtualenv: A virtual environment, providing an isolated Python and libraries
mkvirtualenv: A script to create a virtualenv
pipenv: Combines pip and virtualenv into an integrated solution, comes with a Pipfile not unlike Ruby's Gemfile for example
pyenv: Script that helps to install and manage different Python versions on your system, so you can have the full range of Python 2.x and 3.x installed without them getting in the way of each other. Allows you to set local, default and system Pythons allowing you to switch your Python interpreter depending on which project/directory you're in
pyenv-virtualenv is a wrapper to allow pyenv and virtualenv to work together
https://conda.io/docs/
Yet there is a monolithic additude towards tooling for languages, so much so that experimentation is actively discouraged. In my experience, there is no better way to level up as a developer than releasing and supporting an actual product that solves what you see as problems. Sure, you can contribute to existing projects, but you're going to learn a lot more if you are the owner. This experience, in and of itself, makes it worth it. I don't understand the existential fear and admonishment since the 'superior' tool will be most widely used. Also, there are many more developers today of all stripes and skill levels, more than enough to support the existing toolset, and others that like to play around and experiment with different projects for fun. This is somewhat how biological evolution works... lots of experiments(mutations), almost all are dead ends, but sometimes, that experiment leads to a wildly successful adaptation.
I only have about 4 years experience, but I've never understood this mindset. I see the value of having a primary toolset for a language, but don't see the value of having onw and only one solution. Can someone please explain to me its value?
> Creates a new Python project. Think of an "egg" as a new idea.
How does this differ from the egg distribution format [1] introduced by setuptools, which wheel aims to replace? If it's completely different, couldn't a distinct term be used? Python packaging is confusing enough as it is!
[1] - https://packaging.python.org/glossary/#term-egg
But I agree. Creating a new distinct keyword for that command (why not just `hatch new my-app`?) would be far better. I've tried to get into Python packaging, and it's the biggest PITA I've dealt with as a dev.
But it's still a mess only Rube Goldberg could love.
[1] - https://www.youtube.com/watch?v=qOH-h-EKKac
You can define a .tools-versions file in the root of your project, which lets you install the correct versions with a simple `asdf install`. We even use this in CI for zero-config language version upgrades.
[0]: https://github.com/asdf-vm/asdf
https://common-lisp.net/project/asdf/
http://docs.pipenv.org/en/latest/
Very different tools, but they cover the same surface area, in spots.
I expect I'll be writing quite a few pull requests.