Ask HN: Why Poetry did not become a mainstream package manager for Python?

Imho the python steering committee is too weak in taking action in the direction of improving the whole ecosystem.

They did mess up the python 2/3 and imho they just don't want to deal with such things anymore.

Deprecating stuff is necessary as well as it is necessary to provide an upgrade path (both documentation and tooling) and new stuff.

    python3 setup.py -q pytest

    WARNING: Testing via this command is deprecated and will be removed in a future 
    version. Users looking for a generic test entry point independent of test runner 
    are encouraged to use tox.

    python setup.py provisions dependencies, however, the way
    it does it is considered deprecated and unsupported going
    ahead. Instead, users are encouraged to use tox or ensure
    their own dependencies. There's no way around that unless
    you're willing to fork setuptools, fix the provision issues 
    and use that.

     python - m pytest

    pytest

Lots of people have a process that works well-enough. There was confusion and bad experience with there being multiple options, e.g. pipenv (which came with its fair share of drama), making people wary.

That's an oddly worded question. Poetry is a "mainstream" packaging, dependency and virtualenv management tool. What makes you think it's not popular?

for a start, python’s package manager is pip. Poetry is the leading “pip but also more” tool, and has been for a while.

Poetry has been usable for how long? Maybe two years? Python packaging has been a dumpster fire for that many decades. The developers who haven't left for languages with more sensible package management have been burned by the latest Python packaging fad many times. It is understandable that they would be willing to let others work out the bugs first.

Because venv/pip/requirements is good enough for most cases. Most developers are introduced early to these tools and learn them good enough to work fine for them.

If there's no real problem with the existing tools, then people will not change their tested and trusted way of working for some fancy new tool, no matter what that new tool promises!

There's an old engineering proverb about that: if it works don't mess with it.

Poetry is a mainstream package manager. Every company I know uses it.

And more importantly, the concepts it brings will eventually become Python standards (like pyproject.toml or the lock file): https://snarky.ca/what-the-heck-is-pyproject-toml/

Give it time. It's easily the best ome of the bunch, but python has tons of legacy.

I for one am happy with pip/requirements.txt and venvs, I guess if I wanted to publish packages I would look at a more comprehensive system, but for internal apps and scripts I don't see the benefit. I'd love to know why others prefer Poetry, though.

Nothing is holding it back. It is making progress but Python is a big community and poetry is still new. A new way of managing dependencies does not take off overnight (and nor should it! go slow, and break nothing!).

It is complicated I think.

For one, I think competition, there is Kenneth Reitz' pipenv and then of course conda's environment.yml. I think pipenv lost the popularity contest but somehow lead to a situation where people wondered on which horse to bet.

Conda also takes a lot of market share esp. in the machine-learning deployments while also being not a 'perfect' tool.

Pretty sure we will see more poetry in the coming years.

Last but not least Poetry sometimes also feels like a weird tool. For example, when its dumping colourful stack traces on you in case of (invalid) inputs, etc.

If pip-compile would support dev and production requirements it would still be my way to go I think.

Poetry strikes me as about 90% of what is needed.

I use it all the time because otherwise I’d have to write my own build system.

I am not sure if poetry’s approach to solving dependencies is really correct (Personally I like downloading the metadata for the projects, SMT solving, choosing a set of wheels and installing them together. It only works for wheels since you can’t extract the dependencies for eggs w/o running setup.py)

Pip is almost right if you install all deps at once but it can’t update wrong versions of old packages incompatible with new ones you add.

Practically poetry handles cases that pip doesn’t.

My beef(s) with poetry are:

   A. It is bad hygiene to copy files by default into your package; they should quit pulling wings off angels and just make a /src dir.
   B. Poetry itself gets corrupted and fails to update.  Delete and reinstall always gets me back on the road but I was anxious about it at first.

Can't speak for others, but in my own & work projects I'll use containers (not just for Python but other dependencies). That gives me the isolation I need (e.g. if I need to use a different Python version and/or set of libraries) and pip/requirements is adequate (maybe using pip-tools to keep the requirements.txt up to date). While you can of course set up Poetry in containers, it adds complexity and build time and doesn't give me much on top of what pip+containers provide.

Poetry does need help. I created a thread here: https://github.com/python-poetry/poetry/issues/4160

my primary concern poetry's development stagnating due to natural causes while many projects rely on it. switching costs are high. moving from pipenv -> poetry took me days across all my projects.

they're looking for developers, but also help with managing issues on the tracker.

i also raised the issue of them being funded. I think one possible outcome is seeing if PSF's Fiscal Sponsoree program (https://www.python.org/psf/fiscal-sponsorees/) would be a fit. Waiting to see what their maintainers say.

I looked at it a bit, but not long enough to really understand how it's better than modern setuptools w/ a fully declarative setup.cfg. I'm not sure I have whatever problems it's solving.

It's around 1k opened issues on github. If you want more wide adoption you can help with fixing them.

I can speak for myself— venv is good enough for most things and conda works for everything else (pandas/numpy/tensorflow on Mac m1).

For production code I’m more worried about minimizing surprises then elegance which means it’s really hard to replace something that works. Poetry only hit 1.0 in 12/2019, so maybe I’ll try it if the api remains stable for another few years. Contrast that to JS where we’re now on webpack 5 I think and there’s really no long term support available.

Its pretty good but you can get almost all the functionality with venv, make, and pip.

pip install -r requirements.dev, reference requirements.txt in it. It will install dev packages and the main ones.

I like to keep it simple. When I'm not working on a dockerized project I use virtualenvwrapper + pip and they do the job just fine.

The author of Poetry has an open Github sponsors account [0]. If you're using Poetry, or rely on it in the commercial context, I'd suggest investing into reliability and sustainability of your build tools :)

In my startup, I started to make sure that we allocate a small monthly budget to distribute between our open source dependencies, Poetry being one of them. It's always shocking to find how little support the projects are getting - the author of Poetry currently has 11 sponsors (with majority of those being individuals).

[0] https://github.com/sponsors/sdispater

I like to use it for applications, because of the lock files and easy virtualenvs / all in one experience.

However, I also maintain a number of packages with c extensions or other compiled components, and poetry has no documented way of building packages with binary components, it can only create non-any wheels.

Poetry just was too little too late, containerization is how people are fixing their dependency issues now. I obviously want have a dockerfile that really represents all my dependency then a good package manager that blames my OS when it can't install something. It's the best horse at the car show.

I think poetry is rally gaining traction. Poetry solves some problems in an easy way that are cumbersome to replicate with pip (like guarding against supply chain attacks with lock files) and teams are starting to realize that basic use of pip and requirements.txt no longer is enough to manage dependencies. I am starting to more an more projects adopting poetry and the uptake seems to look like the uptake I saw in FastAPI. Side note: I tried using pipenv, but is was just too slow for resolving dependencies. Curious what other people’s experience is on this with pipenv and poetry.

For me virtualenv+requirements is good enough. The only feature of Poetry that I miss is the ability to create wheels without a setup.py. But I don't create wheels when writing backend code. So I only use Poetry for working with packages.