I don't expect to have to click through to Github to get a summary of what your project's about. Consider putting some of the text from your readme onto the main project page, it's good copy and I think wants to be front and centre.
I went to the docs first to try to find this, but bounced off the terse three word descriptions and lists of verbs.
Disagree. The GitHub and Docs do a great job of explaining the project. No need to maintain more copy than necessary. The homepage is a jumping off point. I wish more projects did it like this, because that just means there is less potential for stuff to become outdated.
The copy is on the website though, it's just behind the docs link. I get that it's a jumping off point, but I'd like even just a sentence explaining what I'm jumping off to.
We love DepHell. I migrated some work repos from Pipenv (and plain ol' pip) to Poetry. However, we didn't want to have a flag day where we updated our build tooling to be 100% Poetry, so I made a Makefile target that builds requirements.txt and setup.py from pyproject.toml. Now developers can work with pleasant tooling, but the build system can use the old stuff it already knows.
We're close to having everything migrated to Poetry. When that day comes, we can throw out all the compatibility stuff, update the build server, and be happy. Until that day, DepHell gives us an easy compatibility layer so that we don't have to do the migration all at once. It's awesome.
I'm curious -- are you using Poetry for applications, besides just the libraries? If so, what is your workflow? I'm currently setting up some workflows with Poetry so I'm keen to learn of others' experiences.
We are. pyproject.toml is now our official record of dependencies, command-line scripts, etc. for all recently-touched repos. We've updated most built tooling to use `poetry install` instead of `pip install -r requirements.txt`. In places where we haven't yet, we have Makefile targets like:
# Generate setup.py and requirements.py until we can get off that treadmill.
# This *must* be done every time you update dependencies, as pyproject.toml is
# now the official source of project configuration and packages.
python_oldfiles:
dephell deps convert; dephell deps convert --env requirements
and pyproject.toml blocks:
[tool.dephell.main]
from = {format = "poetry", path = "pyproject.toml"}
to = {format = "setuppy", path = "setup.py"}
[tool.dephell.requirements]
from = {format = "poetry", path = "pyproject.toml"}
to = {format = "pip", path = "requirements.txt"}
Now when a dev does `poetry add foo`, they can run `make python_oldfiles` to autogenerate updated "compatibility" files.
This: "So, if you can’t install by the recommend way and have some conflicts with your global modules, install without it"
Maybe that's why it is called DepHell instead of DepHeaven.
If you don't have access to clean Python interpreters that don't have "global modules", including in the user's own "site" directory, eventually you will be praying that your builds work.
For a system like "dephell" to have a chance of solving the problem that it tries to solve, it has to have a "place to stand from where to move the Earth" -- without it, it's just a faster way to trash your Python install and have to reinstall it.
JFC, I twitch every time I see this. "Download my script from my website and pipe it to python/perl/ruby". I want to "rm -rf /" people's computers just to stop this trend.
Please never do this! Always read the scripts you're going to run on your computer.
It always icks me out when people recommend this. Like with `get-pip.py`... It's fine to download that and run it to install pip, but for the love of GNU please at least take a cursory glance at what you're running!
If "pip install" tries to install a system-wide package, it will break so many things, the author will get many bug reports. Unless this is an abandoned project, this will get fixed. And "pip install" does not need root access in most cases.
If "| curl" does this, it is basically normal to do anything. And root access is required in a lot of cases.
They are doing different things, and that is exactly the point! Here is a random example:
"meteor.js" was a first relevant hit for curl|sh query on google. is javascript app platform. You are supposed to use "curl | sh" to install it ( https://install.meteor.com/ ). This file:
- Hardcodes install location to ~/.meteor (and removes previous location of it).
- Uses "sudo" to write to /usr/local/bin/meteor
Compare it with "scipy", which can be called a scientific app platform. It tells you to install via pip to user dir ( https://www.scipy.org/install.html ). It installs itself only to this dir, and nowhere else (I know it because I use it at work a lot, and we do pre-packaged virtualenv here). It is also using standard mechanisms -- if you want to have many version side-by-side, it is trivial.
Can you write |sh script so it minds its own business and only writes to a single directory? Yes. Do people do this? Not very often.
-----
When I wrote this, I thought: maybe I am biased towards curl|sh method because I don't know of one? So I want to HN front page, looked at last 210 entries, chosen every one which looks like a a software installable on a PC, and tried to evalalate the system impact:
- Dephell: curl | python, no side effects other than forced install location.
- neo.mjs: installed via node, presumably no side effects outside of node/project dir
- huginn: manual steps, all manual -- or docker. No surprises either way.
- ponylang: docker or PPA. No surprises.
- Poetry: installed via "curl|python", modifies my .bashrc to set PATH (to be fair, it told me about this afterwards...)
- Uni the unicode database: use "go get", no side effects
- Qt 5.1 -- commercial, "installer app".
- Virtualbox 6.1 -- installed via .deb file (system-wide but expected)
- event-driven-shell -- run from repo, optional "make install"
We've had 2 "curl |" apps. One of them was modifying my ~/.bashrc.
We've had 4 "traditional" apps -- which used "checkout repo and run command" method. None of them were writing stuff outside of their checkout dir. Some of them explicitly recommended that users change their .bashrc.
We've also had some docker apps and .deb-installed apps. Of them, virtualbox and Qt could write all over the places -- but they are much more "heavy weight" compared to other ones..
----
This is not a very big sample, but I think it is pretty representative. Once piping things into shell, it seems people cannot help but install stuff all over the place. It is just like "make install" was -- except it is not optional this time.
If they don't have https that is one thing, but if you ever install a ruby gem or python library you're already giving others command over your system. I know it sucks, but it's true.
Is it really all that different from a site asking you to do:
python setup.py
Do people ever open up setup.py to check the code and all that imports for side effects?
As a matter of fact the construct makes it easier to inspect the code without running it:
curl -L dephell.org/install | more
I do understand the main concern though - that supposedly websites are easier to hack than a repository. It is also a valid concern that a hostile agent may serve different versions of the code to different people.But all those concerns apply when downloading code as well.
When one needs to checkout and run setup.py, the packages are almost always nice -- they install packages and that's it. If the compiler is missing it fails. The worst I have seen is super confusing error message because of missing dependencies.
On the other side, I have seen `curl | sh` do completely insane things -- like install packages system-wide, mess with global PATH, create new users on the computers, remove random databases from localhost.
I suspect there are two reasons:
- checked out code can be inspected by random person just by clicking around on github site, every commit has a author name and a message. Feedback is simple and immediate -- there is the "create issue" button is right there.
- If someone tries to update my .bashrc from setup.py, it is pretty clearly a bug. It is unlikely to happen in the first place, and if it does happen, there will be plenty of users who can confirm it is a bad idea.
If someone tries to update my .bashrc from | curl site, it is not usually a bug. The maintainer will likely refuse to fix it, and claim something about user experience.
Which, people should never ever do either. Maybe as a SWE I'm too removed from other people's use cases but I cannot recall the last time I ran an executable on my computer that didn't go through a centralized inspection. Be it me myself, or Debian team etc...
No, I can read the source code. I'd be 100% fine if they advertised people to run `curl xyz; nano xyz; python3 xyz`. The stupid&dangerous part is `|python3`. It allows people to run arbitrary code in your machine.
Honestly that’s a pretty good sign they’re probably not thinking through all the implications of what they’re doing and all they have is yet another incompatible package manager when what everyone wanted was an audited repository for one of the existing package managers.
I rate curl | * as low on the vulnerability scale; scale, as in compared to so many other forms of control we grant regularly to unreviewed code. Yeah, it's not great in practice to do this without reviewing the source, but how often do you "pip install" or "npm install" without reading all 1000 lines of code, plus 1000 lines for each required dependency? How often do you compare hashes (if even available" for binary images? I think in the modern state of software, most verification is done via reputation and critical assessment, vs technical review as in times past.
> Always read the scripts you're going to run on your computer.
I consider this harmful advice. You're promoting a false sense of security, which I think anyone that's spent 10 minutes looking through an underhanded code contest can see.
If your threat model includes the packages you're installing, then you need to do multi-person audits, sandboxing, etc. This applies to software obtained from package managers as well, which aren't typically pre-audited (and let's not forget popular packages that have been hijacked or acquired and injected with malicious code).
Python packaging is definitely in need of something, but my sense is that "yet another tool trying to be a catch-all package/environment management interface" is not it.
36 comments
[ 4.2 ms ] story [ 110 ms ] threadI don't expect to have to click through to Github to get a summary of what your project's about. Consider putting some of the text from your readme onto the main project page, it's good copy and I think wants to be front and centre.
I went to the docs first to try to find this, but bounced off the terse three word descriptions and lists of verbs.
Good luck with the project!
curl -L dephell.org/install | python3
* Trying 185.154.12.127:80...
* TCP_NODELAY set
* Connected to dephell.org (185.154.12.127) port 80 (#0)
> GET / HTTP/1.1
> Host: dephell.org
> User-Agent: curl/7.65.1
> Accept: /
> Referer:
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
you're begging to get owned at that point
https://news.ycombinator.com/item?id=21779191
https://news.ycombinator.com/item?id=21781421
We love DepHell. I migrated some work repos from Pipenv (and plain ol' pip) to Poetry. However, we didn't want to have a flag day where we updated our build tooling to be 100% Poetry, so I made a Makefile target that builds requirements.txt and setup.py from pyproject.toml. Now developers can work with pleasant tooling, but the build system can use the old stuff it already knows.
We're close to having everything migrated to Poetry. When that day comes, we can throw out all the compatibility stuff, update the build server, and be happy. Until that day, DepHell gives us an easy compatibility layer so that we don't have to do the migration all at once. It's awesome.
Maybe that's why it is called DepHell instead of DepHeaven.
If you don't have access to clean Python interpreters that don't have "global modules", including in the user's own "site" directory, eventually you will be praying that your builds work.
For a system like "dephell" to have a chance of solving the problem that it tries to solve, it has to have a "place to stand from where to move the Earth" -- without it, it's just a faster way to trash your Python install and have to reinstall it.
JFC, I twitch every time I see this. "Download my script from my website and pipe it to python/perl/ruby". I want to "rm -rf /" people's computers just to stop this trend.
Please never do this! Always read the scripts you're going to run on your computer.
In that sense curling is better as you can see the code beforehand just pipe it into a pager.
If "| curl" does this, it is basically normal to do anything. And root access is required in a lot of cases.
In practice, no. It is customary for "pip install" to not require root, and for "| curl" to require it.
"meteor.js" was a first relevant hit for curl|sh query on google. is javascript app platform. You are supposed to use "curl | sh" to install it ( https://install.meteor.com/ ). This file:
- Hardcodes install location to ~/.meteor (and removes previous location of it).
- Uses "sudo" to write to /usr/local/bin/meteor
Compare it with "scipy", which can be called a scientific app platform. It tells you to install via pip to user dir ( https://www.scipy.org/install.html ). It installs itself only to this dir, and nowhere else (I know it because I use it at work a lot, and we do pre-packaged virtualenv here). It is also using standard mechanisms -- if you want to have many version side-by-side, it is trivial.
Can you write |sh script so it minds its own business and only writes to a single directory? Yes. Do people do this? Not very often.
-----
When I wrote this, I thought: maybe I am biased towards curl|sh method because I don't know of one? So I want to HN front page, looked at last 210 entries, chosen every one which looks like a a software installable on a PC, and tried to evalalate the system impact:
- Dephell: curl | python, no side effects other than forced install location.
- neo.mjs: installed via node, presumably no side effects outside of node/project dir
- huginn: manual steps, all manual -- or docker. No surprises either way.
- ponylang: docker or PPA. No surprises.
- Poetry: installed via "curl|python", modifies my .bashrc to set PATH (to be fair, it told me about this afterwards...)
- Uni the unicode database: use "go get", no side effects
- Qt 5.1 -- commercial, "installer app".
- Virtualbox 6.1 -- installed via .deb file (system-wide but expected)
- event-driven-shell -- run from repo, optional "make install"
We've had 2 "curl |" apps. One of them was modifying my ~/.bashrc.
We've had 4 "traditional" apps -- which used "checkout repo and run command" method. None of them were writing stuff outside of their checkout dir. Some of them explicitly recommended that users change their .bashrc.
We've also had some docker apps and .deb-installed apps. Of them, virtualbox and Qt could write all over the places -- but they are much more "heavy weight" compared to other ones..
----
This is not a very big sample, but I think it is pretty representative. Once piping things into shell, it seems people cannot help but install stuff all over the place. It is just like "make install" was -- except it is not optional this time.
As a matter of fact the construct makes it easier to inspect the code without running it:
I do understand the main concern though - that supposedly websites are easier to hack than a repository. It is also a valid concern that a hostile agent may serve different versions of the code to different people.But all those concerns apply when downloading code as well.When one needs to checkout and run setup.py, the packages are almost always nice -- they install packages and that's it. If the compiler is missing it fails. The worst I have seen is super confusing error message because of missing dependencies.
On the other side, I have seen `curl | sh` do completely insane things -- like install packages system-wide, mess with global PATH, create new users on the computers, remove random databases from localhost.
I suspect there are two reasons:
- checked out code can be inspected by random person just by clicking around on github site, every commit has a author name and a message. Feedback is simple and immediate -- there is the "create issue" button is right there.
- If someone tries to update my .bashrc from setup.py, it is pretty clearly a bug. It is unlikely to happen in the first place, and if it does happen, there will be plenty of users who can confirm it is a bad idea.
If someone tries to update my .bashrc from | curl site, it is not usually a bug. The maintainer will likely refuse to fix it, and claim something about user experience.
I consider this harmful advice. You're promoting a false sense of security, which I think anyone that's spent 10 minutes looking through an underhanded code contest can see.
If your threat model includes the packages you're installing, then you need to do multi-person audits, sandboxing, etc. This applies to software obtained from package managers as well, which aren't typically pre-audited (and let's not forget popular packages that have been hijacked or acquired and injected with malicious code).