That has nothing to do with PHP and everything to do with that the webserver runs as a different user than the creator of the files.
Unless you have root it's rather difficult to make a directory that the webserver can write to.
It's not specific to PHP, except that PHP is used for web scripting.
The correct way to do it, if you don't have root is:
Make a directory 777 (yes, you do need to). Have the webserver create a subdirectory owned by the webserver. Change the parent directory away from 777 to 755 or 775.
Double check that no one else created something in there while it was wide open.
Be aware that ALL users on that machine who can run web scripts have access to that directory.
freenode #perl still periodically has to talk somebody trying to get a CGI script running out of using 777 to avoid having to figure out permissions properly.
It's an evergreen topic, I've seen this in so many contexts in almost 20 years in IRC help channels that it's a shibboleth for rookie status in configuring systems. Not only that, but it's a communicable weakness in that it's always passed on from person to person: if you only taught yourself using Google, it's actually hard to find 777 as advice. If you see an SO comment/answer recommending 777, it's invariably downvoted into oblivion.
This increased when devops came on the scene, and it was probably one of the first things that made me conclude that devops was about pushing sysadmin skills on unprepared developers. /rant
It’s also an interesting task: How do you configure a folder so that two users, who are in no common groups, can read, write and execute everything? Even if the other user uploaded it with 700 set?
If there's really no way to run these processes as the same group at least, there's always ACLs:
setfacl -m u:NON-OWNER:rwX,d:u:NON-ONWER:rwX,d:u:OWNER:rwX DIR
(Allows NON-OWNER to read, write, execute if already executable for anyone to that directory, and allows NON-OWNER and OWNER to do the same for newly created files.)
I’m not sure that solves the issue of a user (user:user) uploading and creating files via SFTP, but www-data:www-data having to execute, read, and modify them.
What I observed (even with newest npm 3.10.9) that it sometimes creates node_modules directories with permission 777. when doing the npm install multiple times the results vary, most of the time it results in 755 but sometimes in 777). This seems to have nothing to do with the source tarballs content (retrieved from registry.npmjs.org) but a more general issue. As mentioned, its not deterministic and the tarballs definitively don't contain any files/directories with such permissions.
If that's true, it may take a while to unearth the root cause.
I thought further up it mentioned that it starts with the permissions of the files in the tarball, then grabs process.umask (of the npm-tarball subprocess maybe?) and then overrides the execute bit. If that is correct, that is 2 layers of questionable permissions being applied leading to non-deterministicness.
Perhaps, but the author of the comment I quoted seemed to be performing the install over and over, on the same machine. I assume neither the tarball permissions nor the umask would be changing, so it's odd.
This bug was rediscovered 1 year later after a distro shipped a package with /usr/lib/node_modules/ being set to 777 in a deterministic build environment. This seems to be a race condition since it may or may not happen on multiple rebuilds.
Managing node with package managers feels like a a bit of a fools errand at the moment. They are going to be wrapping npm somehow, but when npm can't even do things right..
The problem is that there is no single package manager that covers every platform. If you are writing an inherently portable library in node.js, as an author, are you supposed to also provide packages for RPM, dpkg, pacman, brew etc? Or do you expect distros to package every tiny thing?
So in practice there is a niche that npm covers, that wouldn't be filled if we removed it.
On the question of whether the system package manager should wrap npm - the reason why you want to do so is because it lets npm dependency resolution work regardless of how packages are installed. If the system package manager just does its own thing, then next time you do need to npm install a package (because it's a relatively obscure package that's not in your distro), you don't want it to install copies (or worse, yet overwrite) all the dependencies that you've already installed by other means.
To be fair, that aptly describe being a package maintainer in general -- you spend all day adding workaround patches so build systems stop doing things the "wrong way".
edit: Where "wrong way" is defined as "differently from the target distro's best practices"
The usual way package managers do this is, at packaging time, to use the native build tool to install the software rooted in a temporary directory, then fix up those files as needed (including fixing permissions and ownership) and copy them into a tarball. Does pacman not do that?
Yeah, I reported https://github.com/npm/npm/issues/10463 (npm fails to install everything from a shrinkwrap sometimes -- the feature specifically for locking down the dependencies to make things deterministic/reproducible!) about a year ago and it hasn't received much attention. It was labeled "big-bug" once but got downgraded to "support" somewhere along the line.
I use npm in a continuous integration/deployment system for a project. The system used to do a git checkout, `npm install` to get the dependencies, and then a build, but then we ran into a few extremely confusing issues where we actually had a couple old dependencies tested and deployed into production because npm install failed to respect the shrinkwrap fully (yet it still exited successfully). We finally had to write a script which compares the node_modules directory against the shrinkwrap, and if there's a mismatch, it removes the entire directory and re-runs `npm install`. Thankfully `npm install` tends to install everything fine from a fresh slate, but it's just soo slow (which is also surprising to me, because shouldn't most everything be cached into ~/.npm if I've installed nearly all of the same dependencies recently? I would think that installing from a shrinkwrap of many things I've installed before would mostly be un-tarring things from cache)...
28 comments
[ 3.0 ms ] story [ 63.4 ms ] threadUnless you have root it's rather difficult to make a directory that the webserver can write to.
It's not specific to PHP, except that PHP is used for web scripting.
The correct way to do it, if you don't have root is:
Make a directory 777 (yes, you do need to). Have the webserver create a subdirectory owned by the webserver. Change the parent directory away from 777 to 755 or 775.
Double check that no one else created something in there while it was wide open.
Be aware that ALL users on that machine who can run web scripts have access to that directory.
An even better way is with mpm-itk
This increased when devops came on the scene, and it was probably one of the first things that made me conclude that devops was about pushing sysadmin skills on unprepared developers. /rant
The default ACLs make sure that any newly created files can be accessed by both users.
What I observed (even with newest npm 3.10.9) that it sometimes creates node_modules directories with permission 777. when doing the npm install multiple times the results vary, most of the time it results in 755 but sometimes in 777). This seems to have nothing to do with the source tarballs content (retrieved from registry.npmjs.org) but a more general issue. As mentioned, its not deterministic and the tarballs definitively don't contain any files/directories with such permissions.
If that's true, it may take a while to unearth the root cause.
( 95/121) installing grunt-cli [########################################] 100%
warning: directory permissions differ on /usr/bin/
filesystem: 755 package: 777
warning: directory permissions differ on /usr/lib/node_modules/
filesystem: 755 package: 777
:facepalm:
Package managers for scripting languages and similar should be left to manage user-wide packages, or virtualenv-wide for webapps.
So in practice there is a niche that npm covers, that wouldn't be filled if we removed it.
On the question of whether the system package manager should wrap npm - the reason why you want to do so is because it lets npm dependency resolution work regardless of how packages are installed. If the system package manager just does its own thing, then next time you do need to npm install a package (because it's a relatively obscure package that's not in your distro), you don't want it to install copies (or worse, yet overwrite) all the dependencies that you've already installed by other means.
edit: Where "wrong way" is defined as "differently from the target distro's best practices"
I use npm in a continuous integration/deployment system for a project. The system used to do a git checkout, `npm install` to get the dependencies, and then a build, but then we ran into a few extremely confusing issues where we actually had a couple old dependencies tested and deployed into production because npm install failed to respect the shrinkwrap fully (yet it still exited successfully). We finally had to write a script which compares the node_modules directory against the shrinkwrap, and if there's a mismatch, it removes the entire directory and re-runs `npm install`. Thankfully `npm install` tends to install everything fine from a fresh slate, but it's just soo slow (which is also surprising to me, because shouldn't most everything be cached into ~/.npm if I've installed nearly all of the same dependencies recently? I would think that installing from a shrinkwrap of many things I've installed before would mostly be un-tarring things from cache)...