297 comments

[ 3.0 ms ] story [ 185 ms ] thread
Yeah I see Permission denied messages everywhere now.

    $ brew update
    error: unable to unlink old '.gitignore' (Permission denied)
    error: unable to create file .travis.yml (Permission denied)
    error: unable to unlink old '.yardopts' (Permission denied)
    error: unable to unlink old 'README.md' (Permission denied)
    Error: Failure while executing: git pull --quiet origin refs/heads/master:refs/remotes/origin/master
Nice. I saw that I can't change also system icons ( like Automator, AppStore, Maps, Notes, etc. ).

Quick check on some forums suggest that I should do that in recovery mode. I guess it will be a mess with almost any other app that needs a bit more permissions and needs the raw UNIX filesystem.

The only problem is that after the update, the /usr/local belongs to root again. That's easily fixed with a chown though.

Apps still have access to the raw UNIX filesystem. The only restriction is that they can't write to /usr, /usr/bin, /usr/sbin, /System and into pre-installed Apple apps. Normally, Apps shouldn't need write access to these directories.

It's not so much anything that needs the raw UNIX filesystem, it's anything that touches files that Apple has deemed not needing to be changeable, which I think app icons safely falls under. Fortunately, you can turn the whole thing off if you want.
Will homebrew switch the default install location out of /usr/local for new installations then? I doubt most developers will want to dive into recovery mode just to install wget.
you don't need to. You only need to if you have manually deleted /usr/local for some reason. /usr/local is exempt from system integrity protection.

The only problem is that its owner gets reset to root on every OS update, whereas Homebrew wants its owner to be the Homebrew user.

The advantage of /usr/local as the installation root is that /usr/local/bin is in the default PATH of the OS and that setting the PATH in a way that it takes effect no matter what starts an application is a bit... tricky in OSX as there are many ways to launch an application that doesn't go through a shell, so .profile and friends aren't enough.

Are you sure it gets reset to root? I've been on since the first public beta, quickly fixed the homebrew problem, and haven't had any permission issues since. Admittedly I can't say if normal updates will behave different from the beta ones.
I can't be sure obviously, but if it's done once by the installer, there's no reason to assume it won't ever do it again. So in the worst case, you'll have to re-chown every time you install an update.

Or you don't chown and run brew as root.

But I've found that many GUI applications do not take /usr/local/bin into account. It's like only terminal applications, which are rightly affected by terminal environments, care about it.
I just did this (from [1]):

    sudo chown -R $(whoami):admin /usr/local
And not `sudo chown $(whoami):admin /usr/local`. Are both really necessary?

[1]: https://news.ycombinator.com/item?id=10307800

The -R was a bit overkill - what's inside /usr/local should already have had the correct owner - the problem is the owner of /usr/local itself.
The upgrade changes the permissions of all files within /usr/local to root:wheel.
Didn't for me, only /usr/local itself.
No, only command with `-R` is necessary.
I'd say the "-R" might actually be harmful; for example the mysql GPL distribution binaries from dev.mysql.com installs in /usr/local/mysql and you definitively don't want to mess with the database file permissions there.

It might not be "best practice" to let non-homebrew stuff put things in /usr/local, but it happens.

I'm starting to regret upgrading...
El Capitan inclues a HUGE number of security fixes that you do want to have.

https://support.apple.com/en-us/HT205267

Those fixes applicable to Yosemite will be available as a separate Security Update. Apple usually supports the current OS and the OS X version behind (for example, this for Mavericks was released in August: https://support.apple.com/kb/DL1834?viewlocale=en_US&locale=...). So security is probably not a reason to upgrade.
Are they back porting rootless?
No, and thank God for that. The only thing Rootless prevents is system modification like iOS jailbreak, i.e. prevents tinkering with the system with things like debugging running processes or modifying system files (for example, utilities that modify the UI theme don't work with Rootless). Most malware out there is not that sophisticated, does not try to modify the OS and can still happily live in any other location of the system, like any other program.
So I guess, for new installations it's better to put homebrew into $HOME?
I was just reading the Arstechnica article, which mentions this:

> Instead of allowing developers to put files wherever they want, El Cap offers four canonical safe locations for applications, support files, and drivers:

> /Library

> ~/Library

> /usr/local

> /Applications

> The local or system library directories are the preferred stand-in for /System, and /usr/local is the preferred stand-in for /usr, /bin , and /sbin. However, Apple strongly recommends that developers use /Applications for everything if possible, since that keeps all of an app’s files in a single location for ease of uninstallation.

Why does brew need to install everything owned by my user in /usr/local? I would honestly prefer if everything installed by brew was owned by root:wheel and brew would gracefully abort if I forgot to run it with sudo.

Is all that current mess just so that I don't need sudo for brew?

Then install brew with sudo.
This is what you used to see when you tried to use sudo with brew:

Error: Cowardly refusing to `sudo brew install`

You can use brew with sudo, but only if the brew executable is owned by root. However, this is both not recommended and completely unsupported so do so at your own risk.

I don't know if it's different now, but a quick Google search indicates that brew still behaves like this. I would like to use brew with sudo, but I don't want to use unsupported methods.

You can create an admin account and use that ad indirection for dealing With homebrew
Yes it is still like this.

When XCode needed some license approval after the recent upgrade to support iOS9, Brew warned me that license can only be approved ( programatically ) by root. I tried to sudo brew and received the warning you showed. That was 1 week ago.

>so do so at your own risk.

sudo su at your own risk, hehe.

What is /usr/local for, if its not for locally installed (by the user) sub-components?

Just curious whats wrong with using /usr/local. This is what its for, after all ..

Using /usr/local is just fine. But I feel uneasy when a system directory is owned by just my user. What if there is another user on my system that also wants to perform a brew update?

Using sudo would also be a security advantage since not every program running under your username would be able to modify all binaries in /usr/local.

/usr/local makes sense to me if you've installed brew with sudo. But as other comment says, it's weird to me to have files owned by login user in system directory. It's such a rare scenario now I guess of multiple people on same machine, but isn't user home directory place for files owned by login user?
Typically /usr/local relies on group permissions: the accounts that you trust to build and install stuff are added to, eg. 'admin' group, and /usr/local is group writable for admin with the group s-bit set.
/usr/local is for machine local packages vs. packages that are meant to be shared across multiple machines. /usr/local is still under the control of the local sysadmin though and not a user.

http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHI...

Really brew should always be installing somewhere under ~/ if you are installing packages as a user.

So you'd want all those programs to run as root? Isn't that against any notion of security?
Why would they run as root, just because they were owned by root? Only brew itself would need to run as root (just like most package managers do anyway).
>Why would they run as root, just because they were owned by root?

Because "sudo" (that OP wants to run them with) runs them as root by default?

OP wants to run it (ie, brew) with sudo, not them.
I'm curious to know if there's a reason everyone installs Homebrew in /usr/local (other than it being the default installation path). I've always chosen to install it in ~/.homebrew and haven't had any problems. Everything I install with Homebrew seems to handle an alternative prefix without issue.
Because it has never caused any trouble having it in /usr/local, and as another user said /usr/local/bin is part of the standard path.

Other than this one-liner, which is done once, there really isn't any extra hassle with using the default.

I'd be more interested in why you didn't want to install it there?

Actually it sounds like you will need to run this/restore permissions after every future OS X update.
Hmm, having been on El Capitan since the first beta, this hasn't happened. Do you have anywhere it states that?
https://github.com/Homebrew/homebrew/blob/master/share/doc/h...

"Apple documentation hints that /usr/local will be returned to root:wheel restricted permissions on every OS X update; Homebrew will be adding a brew doctor check to warn you when this happens in the near future."

Perhaps they should consider adding an option to install something into launchd that just does this.
Right. The OS X image now has to contain a /usr/local directory so that it exists unrestricted after you install the OS (otherwise you would be unable to create it yourself, because /usr is restricted). It has to ship with some permissions, so it rightly ships owned by root. The installer will apply these permissions each time it runs.

Aside: I really wish Homebrew didn't encourage having a single user own /usr/local. If they're going to insist on never needing sudo to install things, it should just default to installing in your home directory.

Mostly because of my (perhaps irrational) OCD in not wanting to touch global system paths or files, even though under FHS /usr/local is where you're supposed to install manually-managed libraries and binaries. I believe Homebrew likes to have its path owned by you instead of root, so I think it makes more sense to have stuff that's going to be owned by me to be in my home folder rather than /usr/local.
I installed mine in /usr/local/brew because I already had a ton of stuff in /usr/local managed with GNU stow before brew even existed. I don't have any problems with it.

Someone I know puts it in ~/brew and that works just fine, too. His reasoning is that /usr/local/ is for all users, and though he's really the only user on his laptop, it's just wrong to install a bunch of stuff that's just for him into a global user directory.

For me, because it is the default, and there is section in homebrew FAQ tell that many build scripts breaks if it isn't in `/usr/local/`.
That's a common Unix practice to put system-wide stuff manually managed (as opposed to managed by the OS/distribution) in /usr/local.
Yep, I agree and that's where I install stuff on everything other than OS X. One distinction though, I think, is that most of the stuff I install on OS X I don't want to be available system-wide. I'm (typically) not using Homebrew to install daemons that run all the time or things that serve critical system/network functions, so I've never seen a reason to make them available to the entire system. I agree that goes against the Unix way but I started preferring this way of using Homebrew after I had similar problems upgrading and even updating OS X.

Also, what if for some reason a single machine is shared by two people and they need different versions of some programs installed with Homebrew? Installing everything in /usr/local isn't going to look like a good idea then.

It's not going to look like a good idea even if they need the same version. You're not supposed to run homebrew as root; it runs as your own user instead. If two users try to install things with homebrew in the same directory, you're going to end up with some things owned by one user and some things owned by the other, and things will start failing pretty soon.

A while ago I floated the idea of having a separate, low-privilege "brew" user that installs things, with the brew command automatically switching to that user, but there was no interest.

Homebrew is not really "manually managed" though - I prefer just things I actually manually install in /usr/local
It is if by manually managed we mean _not managed by the system/OS, and not updated without user interaction_. If you update OSX and you had homebrew binaries in /usr/bin, they'd be gone. Homebrew also won't you that there are updates and never updates packages unless you decide to get a new version.
But not to 775 it and change it's owner from root:root (both of which Homebrew does)
brew supports pre-compiled binary packages (what they call bottles) only when you use the default install location /usr/local. If you use any other prefix brew will always compile all packages on your system, which might take a long time depending on the performance of your system. So using /usr/local saves time and energy since all brew needs to do is to download and install binary packages.
Actually, many bottles are marked as safe to install anywhere, so even if you don't use /usr/local you will still get a lot of things installed via bottles (but not everything).
Homebrew is great, but using /usr/local as the default install location just a bad idea.
> in ~/.homebrew

Not sure about OS X but common corporate Unix protocol is to make /home/ noexec.

Non-core ( i.e. outside central package management ) installations go to /opt/

That requires nvram manipulation... Would it work well on a Hackintosh?
IIRC, all of the "nvram" stuff on a hackintosh actually exists as a flat config file in /Extras read by whatever bootloader you're using.
Much like 'goto' statements in code, unless you really have a good reason to do you shouldn't disable SIP.
Homebrew sounds like a malware of EI Capitan. :)
Couldn't all this be avoided if Homebrew installed itself in a sub-directory of /usr/local (e.g. /usr/local/homebrew) instead of /usr/local itself?

Edit: For the avoidance of any doubt/confusion, this is what a virgin /usr/local/ looks like after Homebrew has been installed:

  $ ls -la /usr/local
  total 96
  drwxrwxr-x  13 root      admin    442 31 Aug 21:52 .
  drwxr-xr-x@ 12 root      wheel    408 31 Aug 21:52 ..
  drwxr-xr-x  14 jgavigan  admin    476 31 Aug 21:52 .git
  -rw-r--r--   1 jgavigan  admin    448 31 Aug 21:52 .gitignore
  -rw-r--r--   1 jgavigan  admin    291 31 Aug 21:52 .yardopts
  -rw-r--r--   1 jgavigan  admin   3161 31 Aug 21:52 CODEOFCONDUCT.md
  -rw-r--r--   1 jgavigan  admin   1103 31 Aug 21:52 CONTRIBUTING.md
  -rw-r--r--   1 jgavigan  admin   1241 31 Aug 21:52 LICENSE.txt
  drwxr-xr-x   9 jgavigan  admin    306 31 Aug 21:52 Library
  -rw-r--r--   1 jgavigan  admin   2319 31 Aug 21:52 README.md
  -rw-r--r--   1 jgavigan  admin  23801 31 Aug 21:52 SUPPORTERS.md
  drwxr-xr-x   3 jgavigan  admin    102 31 Aug 21:52 bin
  drwxr-xr-x   4 jgavigan  admin    136 31 Aug 21:52 share
That would break the unix convention they are trying to follow, so then they might as well put it anywhere else as well.
What unix convention are you referring to?
Yep, this is why I install brew into a subdirectory that I keep rsynced across computers.

    $ which brew
    /usr/local/brew/10.10/bin/brew
I'm thinking maybe its time to finish off ditching homebrew for nix.
Is this also related to why you can't rename /usr/local to /usr/local.old? (To do a fresh install of homebrew into /usr/local.)

  $ sudo mv local local.old
  mv: rename local to local.old: Operation not permitted
mv /usr/local /tmp/local.old might work (or just somewhere outside of /usr)
Installed El Capitan yesterday. Process took almost half an hour. Afterwards, brew just gave me a warning to do "chown" to my usr/local. That's it. Everything works fine.
I think the bigger headache will be the next time you buy a new Macbook with El Capitan or later installed on it, because you'll need to do a song and dance to create the /usr/local directory in the first place.
Not true. Even on a fresh install, /usr/local exists and is exempt from SIP
Yes. I did a fresh install and homebrew had zero problems.
I bought a new Air in mid September. I was a little annoyed that documentation seemed out of date, but it took all of five minutes to fix this. No song and dance needed, just a sudo mkdir.
But his comment refers to the new OS changes I believe thus it's going to be a pain going forward.
I just installed brew on a brand-new El Capitan install with 0 problems.
A simple 'sudo mkdir /usr/local' won't work with SIP?
Wow... I just re-read the page and saw what you have to do to create /usr/local and that is insane. Reboot into recovery mode?!? W.T.F?
Most Mac users will never need to do this, and the short reboot cycle is, to many, worth the extra security rootless brings. It's hardly WTF.
Being required to reboot my computer into recovery mode just to add a directory is certainly a WTF worthy situation.

BTW, thanks for the down vote?

To add a directory to a system-protected directory. It's all about protecting the integrity of the installed system. If you don't like/want it, you can reboot into recovery, disable it, and leave it that way.
The integrity of the installed system could be violated by creating a directory?

My system was plenty protected already - it had a root account with a password only I knew. This is just more annoying Apple gatekeeping.

Your system is much more secure with SIP in place because it makes big parts of your system immune to trojans and privilege escalation attacks, the two biggest ways malware creeps into your filesystem. It's not very difficult to disable (even permanently if you are inclined), and offers a big security boost.

And yes, it is Apple "Gatekeeping"... the same gatekeeping that's kept OS X almost completely free of major hacks for over a decade. I don't consider that annoying but to each his own.

> It's all about protecting the integrity of the installed system

Why is this important? It's trivial to reformat my drive and have a fresh install. The stuff I have that's actually sensitive is in my home directory and still unprotected =/

99.99% of users will never need to touch those system directories, (They aren't even required for homebrew), and, at the same time, writing to them poses a huge gaping security hole for all OS X users. Locking them down (particularly in a way that can be trivially reversed with a user who has physical possession of the device and the password) is the right thing to do.

For the 0.01% who need to screw around with the operating system directories, they can take the 90 seconds to hop into recovery mode.

This is a situation where 100% of the user community has a reasonable solution, 99.99% aren't inconvenienced in the slightest, and the security situation goes up significantly. It's like hiding ~/Library by default - 99% of users probably don't need to mess around there, and hiding it probably reduced a lot of failure modes. (and, once again, for the 1%, chflags solves the problem in 5 seconds)

Honestly - I've always been terrified whenever I've been asked for my password during installation (I do a gut check every time an ARQ upgrades asks me for my password - "Do I trust them one more time? Well, they are my backup software, so, ...) - What if it's dropping something in my system directory?

I usually follow up with a cd /; sudo find -cmin -5 to see if things look reasonable - but I'm guess not all users do that. Also - with all the extended attr stuff on OS X, it's not even clear that a find -cmin -5 would be sufficient.

So - count me in the group that is really, really pleased that Apple is locking down those system directories by default.

> cd /; sudo find -cmin -5

Sorry, but I can't resist mentioning that there's no need for that `cd /`; `sudo find / -cmin -5` would work just fine.

From the hn guidelines:

> Please resist commenting about being downvoted. It never does any good, and it makes boring reading.

(though personally I'm not sure why you were downvoted originally)

Yeah, I couldn't resist. I never understand down-voting something you disagree with so I'm shocked each time it happens to me as I rarely down-vote people myself. I just wish someone would require you enter a reason for down-voting.
No, it's not. Mainly because it's not just "adding a directory"
"mkdir /usr/local" is just making a directory. It's not like that suddenly triggers your battery to overheat and explode.
Again, no it's not. Unless you just want to ignore everything about where you're making that directory.
"T.F." is that /usr is immutable under SIP, so if you somehow manage to nuke /usr/local, you have to disable it to put it back.

This is a good feature, and it's not a new one: the BSDs have done it for over a decade with securelevels and immutable mounts --- where you also had to take the system down to make alterations.

If you think about it for a second, you'll see that anything that would have allowed you, a normal user typing at the keyboard, to create directories under /usr would probably render the security of SIP meaningless. It's the fact that you do have to go into recovery mode to hack up your filesystem that indicates that there's actually a meaningful security check happening here.

Just think of SIP's filesystem protections as literally just being "you have to boot the system into a special mode to change certain parts of the filesystem". I want that feature, and I hack up my filesystem all the time! The overwhelming majority of Mac users don't even know they want the feature and benefit enormously from it regardless. A major win.

Here's what burned me: El Capitan will let you remove /usr/local, but you can't re-create it (without the recovery console) due to SIP.
I had the same experience in the betas so I've been a little confused by all the problems people are experiencing.
The first thing I did after the upgrade was disabling SIP. Now my system acts exactly the same way before with all the goodies (except rootless) el capitan has to offer.

Having root not able to write system stuff is good for people that like to blindly execute shell scripts from the internet but I think apple should add something that makes it easier for developers to disable it.

I don't know if there is a new method yet, but before you had to boot into recovery and execute `csrutil disable`. In the early days there was a boot flag you could set but that got removed quickly.

It has to be a method that can not be accessed by malware - the very thing SIP is meant to protect from. I believe that is the reason the option ended up on the recovery partition.
This is one of the reasons I love HN: a grapevine of important things to help developers fix their dev environments, security things we need to be aware of for our servers, new APIs to take advantage of, and other important stuff.
Well, if you use OS X, write against Node.JS, work at a SV startup and are interested in big data / machine learning. It is quite a bubble.
Isn't using the word "bubble" against official HN rules?
I live in a rural part of France, use OS X, Ruby/Elixir, I'm not working for any SV startup, and indeed what the OP describes in one of my favorite things in HN!
I use OS X, write Node.js, live in Northeast Iowa and I too find the information great, like OP said.
There hasn't been a clean upgrade in ages, yet developers eat that shit up and keep showing up with their shiny MacBooks in conferences.

It must be great being a second class citizen.

Worst case is a half-day of cleanup once in a year to keep your computer secure and up to date.

Most years (like this one!) there's a one-liner for Homebrew upvoted to the top of HN, and that's it.

Why do you feel this way? I mean, seriously, at the end of the day we're talking about computers and operating systems. That's it. No reason to be upset. It's just stuff.
To me, the fact that you are directed to install Homebrew to /usr/local is wrong since there's no guarantee how /usr/local is going to treated by Apple in the future and this is now an example of that. The only directory that Apple, probably, doesn't have control is your home directory.

I've been using homebrew for a long time without issues because, in my opinion, I installed it on ~/homebrew. I wonder why is that so difficult for others to use.

Because the default is in `/usr/local` and because the installation warns against changing prefix for possible software breakage.

Now since many people say they never run into problems with nonstandard homebrew path, I'd like to give it a try too.

Users should have full control over their own machines. System Integrity Protection is obnoxious paternalism at best, and at worst, it's just another step toward iOS-ification. If Apple keeps this up, I expect a revival of Linux desktop distributions.
Yeah, iPhone a very very unpopular for this reason, right? You have the same amount of control, this is an option you can disable if you need to. I think the amount of people who need that is very small, even among technical crowd.
They do have full control - see the bit about being able to disable/enable via recovery mode. I for one am very happy to have moved on from an era where desktop security was abysmal and compromises lurked around every corner.
Isn't the goal to prevent malware from modifying core system files? How different is this than a simple mandatory access implementation? As long as it is possible to disable it, should I choose to, I am not outraged.
Yes, 2016 might very well be the year of the Linux desktop.
The common problem of today OSes is that shall malware breach security at least once, it can hide in the system and you will have no idea that you have it. SIP is a good step towards fixing that problem.
> Users should have full control over their own machines.

Agreed.

> System Integrity Protection is obnoxious paternalism at best

Couldn't disagree more. The user maintains full control: you can disable it at will, temporarily or permanently. Protections like this are becoming essential as a result of current malware threats. Thanks to System Integrity Protection, we're now much more protected against rootkits and similar attacks. This is a good thing.

I feel like I have to keep defending it because it's perceived as this removal of user freedoms, when it's factually just not the case–you can still do everything you could do before, you just have to restart into recovery mode to do so (to be able to do this without the restart into recovery mode would undermine the security this technology provides). System Integrity Protection is closer in spirit to something like SELinux than anything.

Many of the "I want full and complete control of my system at all times!" people mostly just mean "I want to continue doing it the way I have done it for the last fifteen years and will resist any change to that traditional method on principle".

See also: Android

Obnoxious changes ought to be resisted. For me as a sysadmin who doesn't need their hand held, this is just one more annoyance to deal with.
Well, if we somehow knew for sure that you'll always be able to disable it, sure, this feature is fine.

But Apple can't be trusted -- they sell two very popular lines of computers (iPhones and iPads) that are completely locked down. It's not unreasonable to be worried that they'll try to move towards locking down their other lines also. Not to mention that they're a remarkably developer-hostile company and probably wouldn't care much about the outcry it would raise from the HN crowd.

It seems a little unfair to judge a company against their hypothetical future actions. I agree that if they do remove the option to disable it in future then that's a problem, of course.

> they sell two very popular lines of computers (iPhones and iPads) that are completely locked down.

Note that iOS is less locked down now than it has been in the past. Previously, you had to enroll in the $99/year developer programme to run your own code on your device, but now you just need to register for a free developer account. So they're moving in the right direction, even if this is still too restrictive for some.

I'm not judging them based on hypothetical future actions. I'm not really judging them at all; I'm just uneasy and worried. I'm predicting that the probability of hypothetical future actions for which I'd judge them is too high for my comfort.

I'm also judging the hell out of them for their past and current action of keeping iOS locked down.

The developer program thing, IMO, is a bit of a red herring; everyone who can buy an iPhone can also pay $99, so pragmatically there is nothing you can do now that you couldn't before, unless you're a student and your parents bought you your phone. Sure, it's a step in the right direction or at least not negative, but I don't think it's particularly meaningful.

> unless you're a student and your parents bought you your phone

It's a bit tangential, but speaking for myself this actually matters. I was a student until recently, I bought my own phone, but I couldn't afford the developer programme (especially when it was two separate programmes, one for OS X and one for iOS). Now, I can build apps and only need to buy the developer programme when I want to deploy them, and then I only have to subscribe to the one programme for both OS X and iOS too. So honestly it does matter to some people.

I hear you about your concerns though. I'm hopeful, with the iPad Pro, that they're actually going to open up iOS more in future, not less. It'd be nice to do proper development on an iOS device. A BSD-like jail environment (e.g. a full sandboxed unix environment with shell access, with maybe iCloud Drive mounted in the filesystem) would be lovely. If they want to sell 'pro' iOS devices, they must be thinking about these sorts of problems.

"But Apple can't be trusted"

You haven't given a reason why. And no, the existence of iOS is not a reason.

Immediately upon reading 'csrutil disable' I thought of 'setenforce 0' and the fact that this is usually the first measure taken to deal with SELinux by many users, usually against better judgement. However this, like SELinux, begs the question: is this a good user interface design? Or simply a small stumbling block for malware authors? I suppose the devil is at least partly in the details: can a kernel exploit at runtime easily flip a few bits somewhere and disable SIP?
I think SIP gives me more control over my machine, not less. I don't want programs installing junk into my system directories. Don't mess with what my distributor shipped me. I've had this same complaint on Linux and OS X for years: to install system wide, you are typically asked to become root, and then some random installer has free reign to wreak absolute havoc.

It turns out Haskell Platform was installing symlinks in /usr.[0] Why? There was no need for that and no easy way to get it out or even know it was doing that in the first place. So as a solution people had "uninstaller" programs that would look for gunk like this left by other programs? Ridiculous.

I'm glad Apple is locking this down. If you don't like it turn it off.

[0] https://mail.haskell.org/pipermail/haskell-cafe/2015-October...

It's an unusual linux installer these days that requires root. Programs aren't going to write junk into your system directories on linux unless you give them permission to. I'm not sure how adding an additional layer of complexity on top of this will solve anything. Now, rather than asking for root, your random installer would be asking you to open up SIP permissions for it.
Privilege escalation is a real problem. SIP fixes it.
A couple of year ago I made a prediction that Apple is going to try to throw out OSX long term. While I thought it would proceed at a greater pace, gradually locking down things because of "security" can easily have double utility. I hope I'm wrong, I like my OSX machines ...
> System Integrity Protection is obnoxious paternalism at best

As is ASLR? Users should always know what addresses their libraries are loaded at in their own machines! While we are at that, let's remove file permissions. That's paternalism - users should know what they are doing.

(comment deleted)
osx just got selinux?
My MacTex (Latex) installation broke for the same reason. Luckily, all I had to do was set put "/Library/TeX/Root/bin/universal-darwin/" in my $PATH and change the path of "pdtlatex" and "bibtex" to the GUI tools I use. But still, I lost about an hour or so trying to figure out what is happening.
Homebrew troubles aside, does System Integrity Protection means the end of OpenVPN and FUSE on Mac OS? Those need to add kernel modules and such.
Yosemite already prevented unsigned kernel modules, so I don't think anything will change here.
No, I have used OpenVPN throughout the betas without issues, and OSXFUSE added support in version 2.8.
No. The official installers are signed, and are thus not prevented.
When I first read this I was a little pissed at Apple, but what if brew were to install everything to /brew? It doesn't need to install into /usr/local.
You'd run into exactly the same problem. Apple doesn't provide /brew in the default image, so something would have to create a new inode in /, and that something presumably requires SIP be disabled (assuming / is protected in SIP at least).

Even if it worked, you'd be no better off than just using ~/.homebrew or something like it. Their whole reason for wanting /usr/local is that (a) /usr/local/bin is in the default $PATH, and (b) it can be written to by a normal user. Using /brew gives up (a) anyway, so you may as well just put it in a user's home directory somewhere.

> You'd run into exactly the same problem.

I think you misunderstand the problem. /usr/local exists in OSX, and is one of the recommended install locations for user software. But if you change the permissions on /usr/local, OSX will interpret that as damage and fix it. Homebrew does this, which is in conflict with all uses of /usr/local in Unix history.

> and (b) it can be written to by a normal user.

Critically, no. This is a security risk. A larger one back when multiuser systems were the only Unix systems, but still a risk. Violates all standards, even the ones OSX doesn't attempt to comply with.

Oh, I know it's a huge problem on multi-user systems. I was just pointing out that homebrew treats it that way so they can have sudo-less access. Technically I guess it's not world writable in a default homebrew install though, just owned by a normal user. Which also defeats the purpose of /usr/local really, but I've never found that design decision from Homebrew to be very good.

It is entirely possible I misunderstood SIP. The first thing I did was disable it, and I haven't bothered any more about it. My impression though was that it locked down all "system directories", however Apple chooses to define that. Which would disallow changing the permissions on /usr/local, but also would disallow creating new files or directories under a protected directory. Is that not what it does?

If you only need a Unix-style package manager for things like coreutils, nodejs, tmux, irssi, vim, etc, pkgsrc runs on OS X and installs to /opt/pkg. There is no Google Chrome, or fonts, etc in it though, so if you enjoy installing those from Homebrew then this is not for you.

http://pkgsrc.joyent.com/install-on-osx/

To be fair, homebrew shouldn't need to change perms, it should follow common unix standards or at least fit in with OSX rather than require a hack. For years I've felt homebrew should really use either /opt/ or ~/.homebrew/ by default.
What makes you think /usr/local isn't a 'common unix standard'?
Never used Homebrew, but taking ownership of /user/local as I understand it does by default, certainly isn't a 'common unix standard'.
> certainly isn't a 'common unix standard'

Here I google.com it for you.

http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY

EDIT: Why is it okay to NOT look up a Standard and be uncertain in your comments when it can be found in 20 seconds tops on the Internet?

I don't see anywhere in there where it says you should take ownership of /usr/local
That is why I said I searched and got the standard for the person who said "certainly it isn't the standard." We have the ultimate resource to know the answer to any question that does have an answer.

I gave this down voted "I googled it for you" because people need to just look up what the Standard is and than speak from a position of knowledge and not some vague non-answer.

For the still lazy the Unix File Hierarchy Standard for /usr

/usr is the second major section of the filesystem. /usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere.

Large software packages must not use a direct subdirectory under the /usr hierarchy.

You are being downvoted because it is a GNU/Linux specific standard, not something that has been adopted by the UNIX standard, POSIX.

And no, I am not the one downvoting you.

Well I feel like an idiot since I absolutely know that but when I was searching I was looking at Unix with no /usr and than went brain dead and thought it was showing /usr as a Linux issue with no standard for /usr. Thank you for correcting me..
Confusingly, it also says:

> The /usr/local hierarchy is for use by the system administrator when installing software locally. It needs to be safe from being overwritten when the system software is updated. It may be used for programs and data that are shareable amongst a group of hosts, but not found in /usr.

> Locally installed software must be placed within /usr/local rather than /usr unless it is being installed to replace or upgrade software in /usr.

http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHI...

It says "The /usr/local hierarchy is for use by the system administrator when installing software locally."

That seems to kind of conflict with what you quoted "Large software packages must not use a direct subdirectory under the /usr hierarchy.", unless they mean a vendor-specific directory rather than "/usr/local".

I dunno, all I can say is that in my experience on various unixes, it is a "common unix standard" to install software that is not controlled by the OS package manager to /usr/local. I don't interpret "common unix standard" as "FHS" though -- if you mean FHS, say FHS. To me "common unix standard" just means "something that is commonly done on unix". And installing locally-installed (not using OS distro) software to /usr/local is pretty commonly done, as many people can attest. That's why homebrew chose it, they didn't just make it up.

(comment deleted)
Filesystem Hierarchy Standard is a GNU/Linux standard.

'common unix standard' is whatever POSIX states as standard.

(comment deleted)
It doesn't take ownership by default IIRC - just recommends that you do so. It will work with sudo if you want to go that way.
I run my Mac with a non-admin account. When I want to install or update Homebrew applications, I su to my admin account and use sudo. Just checked my /usr/local and it's owned by root. I've never had a problem running Homebrew apps from my non-admin account.
This sounds like the best suggestion/solution in this thread. But if you run brew from admin, why can't admin take ownership , and run brew without sudo?
So that Homebrew always requires my password to install an app, just like native OS X installs. It also preserves the default permissions. And it's not that hard to sudo.

    > Homebrew always requires my password to install an app
But isn't that _less_ safe?

Instead of trusting Homebrew with /usr/local, you have to trust every formula with `sudo`? Or does it only use sudo for part of it, a `mv` or something?

That's very clever. There is one problem though, now make clean all install is run as root (well maybe install stages now in homebrew). Imagine an env var not set (the way make vars interact give lots of way to shoot off a big toe here) and the wrong dir gets rm_-rf_-ed or it obliterates something in /bin instead.

If you don't run brew as root you at least protect places like /usr/lib, but you are protected from that and more with macports (where stages and uses nobody).

But for now your approach seems the best pragmatic course to take, thank you for the suggestion.

As I can't edit the comment, I wasn't referring to the name /usr/local as much as the permissions. It'd make more sense to store systemwide hierarchy in /opt/homebrew/* and user-specific hierarchy in ~/.homebrew/* instead.
Many unix tools are written to build looking for dependencies on default paths including /usr/local.

Putting packages installed in a custom /opt/homebrew location would make it harder to install and use software that often assumes a /usr/local possibility, but not an /opt/homebrew it's never heard of.

It is in fact a pretty common place to put non-distro installed software. And was not (and still is not?) used by OSX out of the box. It made a lot of sense for homebrew to use it.

I can't think of anything more standard than /usr/local/.
/opt/local is more correct
iuguy is not talking about use of /usr/local, but rather the fact that homebrew makes it owned by the user who installed homebrew with permissions so that the user can maintain homebrew packages without sudo. Usually /usr/local is owned by root.
Unfortunately, /usr/local is a total mess now. There is no internal package management in os x so applications could (and do) overwrite files in /usr/local. These conflicts are very annoying sometimes.
/opt? MacPorts uses /opt/local.
BSD and Linux use /usr/local, it's not universal.

But /usr/local has never been writable by non-privileged users.

Homebrew depends on that nonstandard and undesireable permissions model. Apple doesn't bend to accommodate an inappropriate use.

Hombrew doesn't want to use a location that is outside the default PATH, and they don't want to install with proper permissions, so they purposefully misinterpret the "standard".

Hijinks ensue, and people blame Apple, which is silly.

A. There is no real standard here B. Apple doesn't bend, period. They don't actually care whether the use is appropriate or inappropriate. C. Blaming this on homebrew is hilarious

Remind me again why i should need to reboot into recovery mode to create a directory?

A. /usr/local has never ever been writable by a nonprivileged user. Every standard that exists, including the principle of least surprise, is emphatic on that point. The security implications of "any other way" are massive.

B. If Linus or the FHS changed their ways in this, there would be outrage. It's inappropriate and just plain dumb. [edit: I shouldn't have said "dumb", please replace with "in violation of all guarantees of system integrity".]

C. See above.

You apparently have issues with SIP. That's a separate issue from the current thread, but OK. You should reboot to securely change nvram settings that will allow you to manipulate the protected zone of the file system (including /usr) because this concession to inconvenience saves you from privilege escalation attacks.

The real question is why you deleted /usr/local in the first place. It's standard OSX, and always has been. If you deleted it before SIP, but object to its absence now that SIP applies to the directory where it lives...I'm not sure how to help.

Where in FHS does it specify that system and user files should be put in

    /Applications
    /Library
    /Network
    /System
    /User Information
    /Users
    /Volumes

?
FHS does not apply to OSX. Never has, never will.
He's asking because you brought FHS up.
Under B, as an example of standards by another vendor who would be pilloried if they allowed the same lackadaisical approach to preserving system permissions integrity. In response to the comment that Apple was being unreasonable.

Apologies if that was not clear.

A. Single user systems are not the same as server systems, and plenty of single user systems have had the equivalent of /usr/local as writable. If you want to argue this security is worth it, that's a different argument. I suspect most people who just want to get shit done are going to find it a huge pain in the ass.

You can also argue these people are stupid.

As for standards, vendors follow FHS and friends exactly as far as it helps them justify whatever they want to do.

If you really want to argue from the perspective that "this is standard and that's the reason it's done", that seems awfully silly to me given the layout and other permissions of apple systems.

It's pretty non-FHS/etc to have /Applications be writable, for example.

B. let's be clear: there is outrage no matter what is changed and why, and the argument is always "it is inappropriate and dumb". So this statement is fairly independent of this change.

A. OSX is not a single user system. It inherits from various Unixes, and no Unix has ever had nonprivileged writability on any directory under /usr.

FHS does not apply to OSX in any way. Why do people think so?

Also, /Applications is not writable by nonprivileged users either!

Agreed on ~"changes yield outrage", but in this case, it's just a software application vendor violating obvious historical and specific guidelines from software OS vendor, and people blaming OS vendor when things break. This perplexes me.

You might just as productively argue that there is no such thing as a modern single-user system, since Windows isn't that either. His point is pretty clear.

OTOH: The debate here is a little confusing. What SIP is doing w/r/t/ /usr/local isn't unprecedented; securelevels and the immutable flag could have the same effect on OpenBSD all the way back in the 1990s.

> You might just as productively argue that there is no such thing as a modern single-user system, since Windows isn't that either. His point is pretty clear.

Correct, and nonprivileged users can't install software globally on Windows either. Because it's now a modern OS, and that's how things are done. For very good reasons.

His point, as far as I can tell, amounts to: applications should be able to trash permissions on convenient directories and not have the OS complain or repair their damage.

Perhaps I'm being ungenerous here? I apologize if so.

On re read, I realize that tptacek is responding only to the SIP part of the discussion. OK, SIP is surprising to desktop computer users. But it's a good feature that requires a little adjustment to work with. But really not much, and none for the vast majority of users who don't muck with system directories.

He's arguing that users should be able to elevate their privileges by supplying root's credentials and then create a simple directory without having to reboot.

I more agree with you than with him, but his argument seems pretty straightforward.

Well, if you unwind far enough, I think the original discussion was about why Homebrew has trouble using /usr/local. Where "using" means "owning, adding git repos to, changing permissions on, etc".

OSX suggests /usr/local as a good location to install software, but insists that the directory itself be owned by root, and protects the parent directory /usr under SIP.

My assertion is that Homebrew is using /usr/local incorrectly (and in contravention to all expectation), and that blaming Apple for the problems thereby created is inappropriate.

I also think SIP is a good thing.

The "single user systems aren't the same as server systems" argument cuts both ways. It is also not a big deal to reboot a single-user systems.
I see what you are saying but please step back and take a wider view for a moment.

Even assuming OSX is a single user system (which is arguable), it is 2015 and should that single user be able to accidentally damage /usr/local? I think not, that by today an OS should help protect you from broken postinstall script or what not.

Moreover please think about this: homebrew is not wanting /usr/local/ root:admin but instead /usr/local/ `/usr/bin/whoami`:admin.

Further even on our laptops where there is just one user, I use a non-admin user. I have no problems using FUS, GUI prompted temporary admin permissions, or /usr/bin/login -p and sudo day-to-day, but homebrew has a problem with sudo.

Apple SIP has no problem with /usr/local/ 0:0 but homebrew does. Because of shortcuts homebrew has forced a number of poor decisions on it's users.

You need to boot into recovery mode to create /usr/local (or to disable SIP altogether) because /usr is immutable, so that code executing under an admin account can't inject itself into binaries (or into PATHs) under /usr.

I use Homebrew and will probably keep doing that because Homebrew is really convenient and just generally works better than MacPorts for me, but what Homebrew is demanding --- ownership over /usr/local --- is not in fact reasonable. It makes a lot of sense to me that OSX will want to keep perms on /usr/local (which is in everyone's PATH) locked down, and so it seems extra hacky that they're going to add a 'brew doctor' check to see if Software Update has "broken" those privileges.

The commenter upthread that suggested Homebrew should use ~/.homebrew or something by default is right.

And I contend that homebrew did not want to used ~/.homebrew because that would mean modifying files to add it to PATH and be additional testing for people that write recipes.
And that's totally fine, and I let Homebrew own /usr/local too. What I'm not going to do is freak out when a totally sane and useful security feature breaks that install mode, especially when I could, if I cared that much about convenience, just take 30 seconds to disable the security feature.
I just want to make clear I'm not freaking out (and I know you're not either).

I would hope though that out of this someone would think, "Hmm... oh drat... yeah I guess this is just one of those things that nobody predicted at first. Shot I think it was a bad idea to have a user own /usr/local/. But you know, it's a good time to improve things... hmm... I know let's build and stage as nobody and then copy under /usr/local/! We still can have binary packages always work and nobody has to mess with PATH, man pages, or dynamic linker trivia. We might have to write something though that asks for a username and password, but whoa... OSX makes that surprisingly easy... I should do that!"

It's not going to be me cause I don't use homebrew, but that would be way better than these hacks that will ammount to people being encouraged to just disable SIP cause of the headache after ever software update.

Keep in mind that you can re-enable SIP right after you create the directory, if you had to create it at all.
There is a standard: http://news.stanford.edu/pr/2015/pr-worms-digest-plastics-09....

This is from RedHat. Not implemented by everyone, but modern OSes (e.g. CoreOS) seem to follow the patterns. CoreOS also do everything in their power to prevent you from mounting /usr as writeable.

Perhaps that doesn't pass your bar for 'real' though since it's not globally implemented ;)

I think that's the wrong link? (It's to a press release from Stanford's engineering department about plastic-eating worms...)
It's been a while since I looked into it, but I opted to stick with MacPorts rather than Homebrew in large part because I got the impression that Homebrew wanted to own /usr/local rather than sharing it with other software. (Maybe that's inaccurate?) I still occasionally compile software by hand rather than through a package manager, and /usr/local has been the default target for a system-wide install on every Unix-style system that I've used in the past 20-ish years.

(Also, my wife and I both have active accounts on my machine, which makes me uneasy having files in a systemwide directory like /usr/local owned by one specific user. That also feels distinctly non-standard to me.)

> I got the impression that Homebrew wanted to own /usr/local rather than sharing it with other software

This is still accurate. Homebrew creates/manages /usr/local/.git. /usr/local/.gitignore starts with /* which is … not conventional.

As others have mentioned, /usr/local may be a fairly standard place to put stuff, but homebrew's permissions model changes the way /usr/local works.

Additionally, /usr/local is used by other applications, and there's no way to ensure that it isn't written over by them or by homebrew when installing or upgrading software.

Putting homebrew in /opt/homebrew means that you can have a systemwide directory that is self-contained, can have a working unix hierarchy underneath and should work.

Putting homebrew in ~/.homebrew/ lets you have a user-specific directory that is self-contained with a hierarchy underneath.

In an ideal world, homebrew would use something like /opt/* for systemwide hierarchy and ~/.homebrew for user-specific hierarchy. We don't live in an ideal world, so we just have to make do with fudges to /usr/local permissions for now.

It is not standard for /usr/local to be writeable by users.
Where am I supposed to put stuff like my shell scripts?

I'm now using /usr/local/bin.

I looked for a guide, no joy. The more opinionated, the better, because I just want to follow the norm.

"Fit in with OSX" is why I put it in ~/Library/Homebrew/. Obvs.

    >  it should follow common unix standards
Isn't /usr/local?

    > or at least fit in with OSX
It did - before now /usr/local was available unprivileged with ownership.

    > rather than require a hack
It only needs that now, since requirements to "fit in with OSX" have changed. That's not Homebrew's fault - they need a fast solution. Kudos in my book for responding before it's a problem.
No, /usr/local is normally 755 root:root (or whatever the effective equivalent is). Homebrew wants it to be 775 root:admin, so that users with the admin bit can write to it. That's not standard.

People keep writing here as if El Cap removes /usr/local entirely. It does not. /usr/local is there by default. You only have to go into recovery mode if you've somehow removed it yourself.

> Homebrew wants it to be 775 root:admin

It's worse than that. Homebrew wants 775 $(whoami):admin.

(comment deleted)
quesera's reply points-out just how bad it is, but even for just 775 root:admin I would have to use sudo (plus edit the sudoers or /usr/bin/login -p first) anyway:

  $ /usr/bin/env -i /bin/sh -c \
    'id | grep -lw admin >/dev/null && echo admin || echo non-admin'
  non-admin
The problem is I have no problem using sudo, but homebrew does (like it wants make all to happen as me).
Yeah, the more I read this thread, the more I'm thinking "why is that?" - `apt-get` et al have no problem with you using sudo, why does `brew` scare you into an alternative? 'because Mac'?
(comment deleted)
If I have already installed a bunch of packages, how do I move everything to something like ~/.homebrew? (1) Change the HOMEBREW_PREFIX (2) move the existing aliases from /usr/local/bin (3) add the directory to my path (4) ???