Well, pyenv does have rather good virtualenv integration, and in fact I use that to make it easy to spin up a virtualenv with whatever Python version I happen to need at the moment.
I was puzzling over that exact thing for maybe 15 minutes. I've never used either one. Neither is on my path currently, even though I just installed python 3.5 with that option ticked. I can find pyvenv.py in my file system. Perhaps I'm expected to add it to my system path manually? Maybe I'll figure this out another day.
I've switched to using conda for managing all of my local python projects and tools. Its an easier, more flexible interface and it can switch both the virtualenv and the python version. So no problem going back and forth from py3 to 2
Just install miniconda (without all the data science packages) and then use pip as normal
I'm running that in Vagrant anyway, but it would work just fine. Conda is just a well done wrapper for switching python versions and virtualenvironments.
You can also set ENVIRONMENT variables for each so that could be useful for specific django projects.
Most often you won't need to use pip, and `conda install ...` is preferable (for example, pretty much everything on PyPI is mirrored to anaconda.org, and if you ask them to support a well-known package that's not already available, it will be available extremely quickly, and chances are it's already available on some other anaconda.org channel anyway).
You can use pip from within conda as well if you want to, and you can refer to environment-specific pips directly, such as `~/anaconda/envs/my_env_name/bin/pip install ...` to install for the environment my_env_name.
conda can work with pip-styled requirements.txt files as well, and it's easy to either upload your own packages to anaconda.org, or create the conda build metadata (metadata.yaml) with instructions for how to get the package from PyPI if you prefer to only upload to PyPI.
There is an abundance of well-written tutorials and documentation for these things.
It'd be really great if conda knew how to fall back to pip, or if the conda skeleton -recursive / conda build / conda install cycle could be collapsed into one command
I'm not sure falling back to pip is a good default. You might rather be alerted to a package's absence on anaconda.org than to have it silently installed elsewhere, for example for use cases when you totally want zero pip installed packages.
But I agree there's a lot of opportunity for improvement around the configurability of this sort of thing.
While conda is newer than virtualenv, pip, and easy_install, it's not really "new." Conda also is orthogonal to the source repository. As I mentioned, you can easily let conda manage the environments of your pip-installed packages.
One reason anaconda.org is better is that it's also Python agnostic. There is a main channel for R and R packages, so you can actually manage R environments the same way and R dependencies in a Python project. This could conceivably grow to include many things that pip alone can't help you with.
Having built packages both by using the necessary extra setting in a setup.py file, coupled with `python setup.py register` and the PyPI upload commands, and also via the meta.yaml and the anaconda.org upload commands, the latter seems more sensible to me.
Putting info into setup.py is a bad thing generally, especially if you do real work in setup.py, like with a complicated Cython package. Placing that info into meta.yaml offers better separation between data and code, and is remarkably easy to use in a cross-platform way with just `conda build`.
There's a lot more related to configurability and advanced options, but the above examples have been enough for me to fully use conda and to support other packaging as secondary when necessary.
Hmm, can someone mention the advantages of conda? Right now I use virtualenv with autoswitching, and I don't know what conda provides that's better. You mention switching the Python version, but virtualenv does that seamlessly as well, so I never had a problem.
yes, virtualenv switches to whatever python you used to create that virtualenv. but where did that python come from ?
conda installs pythons as well. like pyenv, but I lost a bunch of time trying to get that to work correctly.
everytime I used virtualenv I had to go look up things and many things still didn't work with the same ease as nvm rvm etc. There's virtualenv, virtualenvwrapper (to make it somewhat usable ? I forget); pyenv; pyenv-virtualenv (what?)
conda has always worked flawlessly for me without fussing.
I currently use GNU Guix to make virtual environments [0] (sometimes using containers) for software written in any language, which I find much nicer than a Python specific solution.
Small disclaimer: I'm also the author of Pew ( https://github.com/berdario/pew ), an alternative to virtualenvwrapper... and I'm working on a small program to try to automate further what pypi2nix/guix-import do to automatically package Python stuff for Nix/Guix... Until that'll be really easy, I think that whipping up a new pew virtualenv to try something out quickly is still the easiest thing (especially for people not used to Nix... I know: otherwise you can also use mutable virtualenvs on top of Nix)
We've moved to Docker Compose for local Python (Django) development.
Oftentimes when you need package management (pip) and multiple environments (virtualenv) you likely also have several services necessary for your projects (Python, Postgres, Nginx, Celery, et al.) and it can be easier for your team to run `docker-compose up` and have the whole stack running in lieu of setting up services manually and then initializing a virtual environment.
I much prefer taking the time to setup Docker Compose even for a simple project. I don't have to initialize a virtual environment, install packages, make sure the DB is running, sync with the DB, run the web server, etc. All of that work becomes automatic and I can focus on development instead sysadmin stuff.
Frankly I just use Nix, which is a little like having virtualenvs for your entire distribution and as such can handle non-python dependencies properly.
"Virtualenv isn't supposed to handle non-Python dependencies."
Making it kinda useless for anything like numpy or scipy.
"Why use Nix instead of containers or VMs?"
Well, why use virtualenvs instead of containers or VMs?
Because it's lighter weight and more straightforward. And doesn't require you to swap out your whole working environment every time you want a slightly different package setup.
Containers and VMs are a bad excuse for "I don't have a hope of getting the dependencies of my setup under control so I'm just going to ball them all up into this great big binary image."
Nix packages are incredibly flexible and composable in comparison.
30 comments
[ 2.7 ms ] story [ 68.4 ms ] threadJust install miniconda (without all the data science packages) and then use pip as normal
http://conda.pydata.org/docs/using/envs.html
Autoswitching when you enter a directory:
https://github.com/chdoig/conda-auto-env
You can also set ENVIRONMENT variables for each so that could be useful for specific django projects.
You can use pip from within conda as well if you want to, and you can refer to environment-specific pips directly, such as `~/anaconda/envs/my_env_name/bin/pip install ...` to install for the environment my_env_name.
conda can work with pip-styled requirements.txt files as well, and it's easy to either upload your own packages to anaconda.org, or create the conda build metadata (metadata.yaml) with instructions for how to get the package from PyPI if you prefer to only upload to PyPI.
There is an abundance of well-written tutorials and documentation for these things.
But I agree there's a lot of opportunity for improvement around the configurability of this sort of thing.
One reason anaconda.org is better is that it's also Python agnostic. There is a main channel for R and R packages, so you can actually manage R environments the same way and R dependencies in a Python project. This could conceivably grow to include many things that pip alone can't help you with.
Having built packages both by using the necessary extra setting in a setup.py file, coupled with `python setup.py register` and the PyPI upload commands, and also via the meta.yaml and the anaconda.org upload commands, the latter seems more sensible to me.
Putting info into setup.py is a bad thing generally, especially if you do real work in setup.py, like with a complicated Cython package. Placing that info into meta.yaml offers better separation between data and code, and is remarkably easy to use in a cross-platform way with just `conda build`.
There's a lot more related to configurability and advanced options, but the above examples have been enough for me to fully use conda and to support other packaging as secondary when necessary.
Works perfectly well inside conda, and properly tracked. Packages don't have to come from anaconda/binstar.
conda installs pythons as well. like pyenv, but I lost a bunch of time trying to get that to work correctly.
everytime I used virtualenv I had to go look up things and many things still didn't work with the same ease as nvm rvm etc. There's virtualenv, virtualenvwrapper (to make it somewhat usable ? I forget); pyenv; pyenv-virtualenv (what?)
conda has always worked flawlessly for me without fussing.
[0] http://www.gnu.org/software/guix/manual/html_node/Invoking-g...
Just yesterday I added a few tasks to the ansible playbook for my workstation to automate its installation:
https://github.com/berdario/dotfiles/commit/4e92cdb00dbe135f...
Small disclaimer: I'm also the author of Pew ( https://github.com/berdario/pew ), an alternative to virtualenvwrapper... and I'm working on a small program to try to automate further what pypi2nix/guix-import do to automatically package Python stuff for Nix/Guix... Until that'll be really easy, I think that whipping up a new pew virtualenv to try something out quickly is still the easiest thing (especially for people not used to Nix... I know: otherwise you can also use mutable virtualenvs on top of Nix)
Oftentimes when you need package management (pip) and multiple environments (virtualenv) you likely also have several services necessary for your projects (Python, Postgres, Nginx, Celery, et al.) and it can be easier for your team to run `docker-compose up` and have the whole stack running in lieu of setting up services manually and then initializing a virtual environment.
I much prefer taking the time to setup Docker Compose even for a simple project. I don't have to initialize a virtual environment, install packages, make sure the DB is running, sync with the DB, run the web server, etc. All of that work becomes automatic and I can focus on development instead sysadmin stuff.
Why use Nix instead of containers or VMs?
Making it kinda useless for anything like numpy or scipy.
"Why use Nix instead of containers or VMs?"
Well, why use virtualenvs instead of containers or VMs?
Because it's lighter weight and more straightforward. And doesn't require you to swap out your whole working environment every time you want a slightly different package setup.
Containers and VMs are a bad excuse for "I don't have a hope of getting the dependencies of my setup under control so I'm just going to ball them all up into this great big binary image."
Nix packages are incredibly flexible and composable in comparison.
$ python3 -m venv myvenv $ source myvenv/bin/activate