Great article, and it's nice to know there are more choices for package management on the Mac.
But, sadly:
Install Xcode from the Mac App Store...
This is the main reason why I've stopped using OS X and am installing Linux on the Macs I use personally. I know it's free and I have an Apple ID, but it really bothers me that Xcode is no longer included with the OS and is unavailable as an anonymous download.
But it still requires an Apple ID, and that's what bothers me. Xcode should be an optional install included with the OS or downloadable anonymously. After all, I need it to install free software with MacPorts/Homebrew/pkgsrc, so I don't think my position is unreasonable, even if it seems extreme or impractical. It's just the line I wouldn't cross myself. With Linux, I don't have to, so I'm enjoying the alternative while it still exists.
Seems somewhat over the top and unreasonable to me to be honest. If it was included only on the install dvd or disk image, it could be out of date or have known bugs that are fixed in other versions. Keeping it downloadable means llvm can be updated independent of the OS. Not sure why an id is so offensive to you.
Before Software Update was integrated with the Mac App Store, Xcode was an optional install from the DVD that was also freely and anonymously updated via Software Update (beginning with Snow Leopard, I believe). It allowed me to freely and anonymously use my computer in a way that I wanted, so requiring an Apple Id makes it feel like that freedom was taken away. Everyone has a line somewhere, and that was the one I couldn't cross. Why do I need Apple's permission to install Mutt? It seems crazy and, yes, offensive to me.
Where do you get apples permission to install mutt? They're only going for a developer id. Put in bogus information if you want.
I guess I don't see this as being a freedom issue at all. From my vantage it seems over the top. And this is from someone thats used linux since 1997ish and osx since whenever 10.3 was. Seems over reactionary to me.
You pretty much need an Apple ID to use OS X these days, so if you don't trust Apple to not use the information it can/may be track(ing) against you I guess it would be a good idea to not use OS X. Personally, I don't really care...I already use the App Store so I'm already signed in, I guess...and getting automatic updates through one clean interface is way nicer than the previous Software Update / App Store cross, which was always annoying.
However, jackalope, I'm hoping you've seen the recent articles on using ChromeOS + Ubuntu on a Chromebook and you'll stop buying overpriced Mac hardware =D
> pkgsrc has several advantages over other Mac-specific package managers like MacPorts or Homebrew, especially in regards to security.
I'd love to hear a bit more about what those advantages are. Other than portability (which isn't a concern for me, since 99.9% of the non-OS X flavors of Unix I use have apt), the only argument the author gives for pkgsrc over something like Homebrew is that it has "important security features". Furthermore, its setup and use looks a fair bit more complex than Homebrew, making it seem less appealing without having the context the author clearly does.
I enjoyed the article, and think it's awesome to know that alternatives to MacPorts/Brew exist, but I'm much more interested in the promised follow-up article about the actual security features of pkgsrc and what makes it worth installing.
Agreed. Also, as a long-time NetBSD user, I'd be much more interested in having a package manager that's as easy and flexible as Homebrew on NetBSD rather than running pkgsrc on my Mac.
Thanks for the feedback. I'll be writing that follow-up post in the next week or so. To sum them up, the security features I alluded to are: signed packages; automated security checks; build options; and compile-time hardening.
HFS+, the file system used on OS X system by default, is not case-sensitive, but some packages from pkgsrc assume a case-sensitive filesystem (because they may have two files, e.g. one called "MAKE" and one called "Make" in the same directory).
It's easy to just install one's system on a case-sensitive HFSX filesystem from the get-go by making that choice at install time, but that's not the default.
Never use a case-sensitive file system for the OS volume. The OS itself is fine with it, but it breaks too many badly written third-party apps, including Adobe apps and Steam. I strongly recommend creating a read/write .dmg instead.
Keeping in mind that case-senstivie HFS+ breaks Adobe CS5 & 6, [Unity3D](http://unity3d.com/) and a host of other software that has been poorly tested.
It's not been poorly tested, it's just not designed to run on case-sensitive HFS+. It's unreasonable to expect a developer to test their software on every possible configuration, especially as something as fundamental as changing the configuration of the file system.
On my Mac, I currently install software in at least 8 different ways: pkgsrc, homebrew, GNU stow (for software that is in neither of the first two), gem, pip, drag-and-drop to /Applications, running installers, and using the App Store. It is becoming a bit hard to keep track and keeping things updated, but I've found it really is necessary to use all of these. Frequently a package from pkgsrc will not build, so I'll revert to homebrew. If it's not in homebrew, I have to build it manually or find a binary distribution somewhere.
On my Arch Linux system, in constrast, everything is installed using the Package Manager or pip.
I very strongly advise to NOT ever use a case-sensitive file system. While in this case we have an example of software the NEEDS case sensitivity, a lot of applications folks use will BREAK (or would historically) when installed on a case sensitive file system.
Examples include: Dropbox, Steam, Photoshop.
I'm sure more recent versions of these software tools can handle case sensitive file systems (one hopes), but I can say with certainty that there is plenty of other software in the wild, especially if its origins date back using FAT or NTFS on WIN32.
In an ideal world, software would be written in a way that works fine on both Case and Case-Insensitive file systems, but we live in a world were software engineering is rarely so disciplined.
point being: if you want to use any interesting desktop software that at some point may have been targeted in part at WIN32 systems in some past era, please make sure you stay with a case INsensitive file system
Why? Case Sensitive file systems are a truly ridiculous idea. I can't imagine anything more annoying than having README and Readme being two different, independent files...
You are correct. In practice, case-sensitive filesystems on OS X are impractical, but boy does this destroy my faith in developers... how hard is it make sure that you use the same cases in your paths throughout your program?
Since a lot of the comments here about the case-sensitive file system, note this comment on the blog[1]:
> pkgsrc hasn't required a case-sensitive file system for some years now. If you've run across documentation that told you otherwise, could you point me at it so it can be fixed?
Assuming that's correct, then using pkgsrc on OSX is very simple: simply download pkgsrc, bootstrap it and then use it.
(1) OS X is different enough from NetBSD and pkgsrc's assumptions about how the OS is set up and what kind of base installation is available [compilers, linker and what not), so that I can't really expect that the average package will actually work; and
(2) It might not have the packages I need (possibly because of licensing issues). Homebrew, which I am using, is license-agnostic, whereas I think NetBSD excludes (L)GPL code?
A casual browse through the package list [1] shows me that it has many things I use, but also lacks many things I use. For example, it lacks Discount (the Markdown processor), rbenv (what I use to easily switch between Ruby versions), rbbuild (what I use to easily install Ruby versions), Node, Elasticsearch, Redis, Riak and Proctools (pkill etc.). Not too bad, but it means that any switch away from Homebrew will involve a lot of work with very little upside.
(Also, I notice NetBSD makes the same mistake as some Linux distros in creating packages for Ruby gems.)
As a side note, I know this is the *BSD way and all, but remembering a path to "cd" to in order to run "bmake && bmake install" seems silly in this day and age. Is there really no command-line tool that integrates this process, as well as the task of keeping the ports tree synced? "brew install" is so much simpler.
So far, I haven't seen any OS X-specific problems with the build process, everything works fine. The pkgsrc documentation says OS X is officially supported.
The pkgsrc-wip project has packages for some of those projects you're looking for:
In general the entire system seems rather... antique. It doesn't give me a happy, fuzzy feeling inside. The reason is that I'm not a Unix geek, not in the ways this system wants me to be.
Oh, sure, I know the tools; I've written hundreds of bash scripts, I've done my share of C/POSIX programming, I know my way around Make and GCC and so on. But I don't want to do those things.
What's so great about Homebrew is that the recipe files are built against an API, which coincidentally is a simple programming language (Ruby). Pkgsrc seems to be based on makefiles, which is not really an API as much as a set of conventions based around Make and bash, where I need to sit down and read a manual to understand what plugs into where. The same problem exists with MacPorts, which uses Tcl; it's just so much more complicated than it needs to be.
The fact that pkgsrc and its ecosystem is is still entirely based on CVS, not just as a sync protocol but as the version control system for everything, is not inspiring, either. I'm sure it works for the people behind this project, but it simply won't lead to me submitting packages.
I'm sure I could use pkgsrc simply to install new packages, and treat it like a black box as a "user". But then one day I will need a package that's not in the tree, or I will have to fix a broken package, and I don't want to be left in a situation where I will have to do a lot of work to accomplish something trivial. In 2001 I would have used CVS and I would have hacked packages, not knowing any better. Not in 2012. :-)
That said, pkgsrc looks interesting, and I will be keeping an eye on it.
I am not a programmer, and I've used Macs since 1987. As far as I know, I haven't ever used a package manager and don't really know what they are beyond something that lets you install and delete apps.
If I want an app, I download it. If I don't want it any more, I move it to the trash. If I really don't want it anymore, I use AppZapper. Can someone explain to me why the issue of package managers comes up so often around these parts and why they're so important to people?
Primarily, package managers solve the problem of dependencies. [1] For example, if you want to install a piece of software X -- which depends on A and B, and A in turn depends on C -- then the package manager will install all of these for you. A package manager will also keep your software up to date automatically. There are other advantages, but basically a package manager saves you time.
> If I want an app, I download it. If I don't want it any more, I move it to the trash. If I really don't want it anymore, I use AppZapper. Can someone explain to me why the issue of package managers comes up so often arouand these parts and why they're so important to people?
You're talking about apps in a relatively recent sense of the word: a wholly self-contained piece of software that you drop into the OSX /Applications folder.
Package managers, in the relevant sense, are for installing software (often called 'packages') into places that OSX normally hides from its users (/usr/local for example). A lot of this software is command line software, though on other operating systems, a package manager would also manage GUI packages. (On OSX there is this somewhat artificial distinction between /usr/local and /Applications.)
In any case, the kind of software pkgsrc, MacPorts or Homebrew installs requires more work than just "download it". You can download the source code easily enough, but you can't simply download the precompiled software itself, unzip it and drop it somewhere. You have to build it, and this will require lots of other packages and also a whole bunch of knowledge and or scripts to configure and build it correctly. Package managers like pkgsrc help to automate and simplify all of this.
If you've never needed anything like this, no great surprise, and likely you never will. But if you do need these things, then OSX poses some extra challenges. (Although as far as that goes, every os has its own issues, no doubt.)
Tried building it, making sure that I used a clean bash environment not polluted with Homebrew or anything else. Bootstrap failed with no particular error, just a useless "exited with status 1".
Traced it to bmake/boot-strap, which after building bmake will run "bmake -r -m / test". Apparently bmake doesn't understand the arguments but, in typical Unixy fashion, will not say what it doesn't understand, but will instead output a "usage: bmake ..." and exit with a non-zero exit code, which causes everything to abort.
So I added some debug output to bmake's main.c and found that even though boot-strap invoked it with those arguments, bmake itself parsed them as "-j4 -w". Which is what I have MAKEFLAGS set to in my environment. I unset MAKEFLAGS and everything builds.
I love Unix, but I just hate how everything exists with so little separation and so much obscure brittleness, ending up in a situation where a tool called "bmake" will use options I intended for GNU Make (which doesn't have a .gnumakerc and so requires a global environment variable).
41 comments
[ 2.8 ms ] story [ 72.6 ms ] threadBut, sadly:
Install Xcode from the Mac App Store...
This is the main reason why I've stopped using OS X and am installing Linux on the Macs I use personally. I know it's free and I have an Apple ID, but it really bothers me that Xcode is no longer included with the OS and is unavailable as an anonymous download.
Like before MAS, it's still available at: https://developer.apple.com/downloads/index.action
The latest version listed is Xcode 4.6 (1.6GB DMG) with Feb 21, 2013 release date.
I guess I don't see this as being a freedom issue at all. From my vantage it seems over the top. And this is from someone thats used linux since 1997ish and osx since whenever 10.3 was. Seems over reactionary to me.
However, jackalope, I'm hoping you've seen the recent articles on using ChromeOS + Ubuntu on a Chromebook and you'll stop buying overpriced Mac hardware =D
[1] http://llvm.org/releases/download.html
I'd love to hear a bit more about what those advantages are. Other than portability (which isn't a concern for me, since 99.9% of the non-OS X flavors of Unix I use have apt), the only argument the author gives for pkgsrc over something like Homebrew is that it has "important security features". Furthermore, its setup and use looks a fair bit more complex than Homebrew, making it seem less appealing without having the context the author clearly does.
I enjoyed the article, and think it's awesome to know that alternatives to MacPorts/Brew exist, but I'm much more interested in the promised follow-up article about the actual security features of pkgsrc and what makes it worth installing.
It's easy to just install one's system on a case-sensitive HFSX filesystem from the get-go by making that choice at install time, but that's not the default.
You either have the case sensitive fs, or you don't. Sounds like you're making up excuses for the developer laziness.
On my Arch Linux system, in constrast, everything is installed using the Package Manager or pip.
Examples include: Dropbox, Steam, Photoshop. I'm sure more recent versions of these software tools can handle case sensitive file systems (one hopes), but I can say with certainty that there is plenty of other software in the wild, especially if its origins date back using FAT or NTFS on WIN32.
In an ideal world, software would be written in a way that works fine on both Case and Case-Insensitive file systems, but we live in a world were software engineering is rarely so disciplined.
point being: if you want to use any interesting desktop software that at some point may have been targeted in part at WIN32 systems in some past era, please make sure you stay with a case INsensitive file system
that was my PSA for the day/week/month
Typing LS and having it work is... odd.
> pkgsrc hasn't required a case-sensitive file system for some years now. If you've run across documentation that told you otherwise, could you point me at it so it can be fixed?
Assuming that's correct, then using pkgsrc on OSX is very simple: simply download pkgsrc, bootstrap it and then use it.
[1]: http://www.eliteraspberries.com/blog/2013/03/pkgsrc-on-os-x....
(1) OS X is different enough from NetBSD and pkgsrc's assumptions about how the OS is set up and what kind of base installation is available [compilers, linker and what not), so that I can't really expect that the average package will actually work; and
(2) It might not have the packages I need (possibly because of licensing issues). Homebrew, which I am using, is license-agnostic, whereas I think NetBSD excludes (L)GPL code?
A casual browse through the package list [1] shows me that it has many things I use, but also lacks many things I use. For example, it lacks Discount (the Markdown processor), rbenv (what I use to easily switch between Ruby versions), rbbuild (what I use to easily install Ruby versions), Node, Elasticsearch, Redis, Riak and Proctools (pkill etc.). Not too bad, but it means that any switch away from Homebrew will involve a lot of work with very little upside.
(Also, I notice NetBSD makes the same mistake as some Linux distros in creating packages for Ruby gems.)
As a side note, I know this is the *BSD way and all, but remembering a path to "cd" to in order to run "bmake && bmake install" seems silly in this day and age. Is there really no command-line tool that integrates this process, as well as the task of keeping the ports tree synced? "brew install" is so much simpler.
[1] http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/README-all.h...
The pkgsrc-wip project has packages for some of those projects you're looking for:
Discount: http://pkgsrc.se/wip/discount Node: http://pkgsrc.se/wip/node Redis: http://pkgsrc.se/wip/redis Riak: http://pkgsrc.se/wip/riak
More about pkgsrc-wip: http://pkgsrc-wip.sourceforge.net
To simplify the build process, there is a pkgsrc "front-end" called pkgin:
http://pkgin.net
Hope that helps.
In general the entire system seems rather... antique. It doesn't give me a happy, fuzzy feeling inside. The reason is that I'm not a Unix geek, not in the ways this system wants me to be.
Oh, sure, I know the tools; I've written hundreds of bash scripts, I've done my share of C/POSIX programming, I know my way around Make and GCC and so on. But I don't want to do those things.
What's so great about Homebrew is that the recipe files are built against an API, which coincidentally is a simple programming language (Ruby). Pkgsrc seems to be based on makefiles, which is not really an API as much as a set of conventions based around Make and bash, where I need to sit down and read a manual to understand what plugs into where. The same problem exists with MacPorts, which uses Tcl; it's just so much more complicated than it needs to be.
The fact that pkgsrc and its ecosystem is is still entirely based on CVS, not just as a sync protocol but as the version control system for everything, is not inspiring, either. I'm sure it works for the people behind this project, but it simply won't lead to me submitting packages.
I'm sure I could use pkgsrc simply to install new packages, and treat it like a black box as a "user". But then one day I will need a package that's not in the tree, or I will have to fix a broken package, and I don't want to be left in a situation where I will have to do a lot of work to accomplish something trivial. In 2001 I would have used CVS and I would have hacked packages, not knowing any better. Not in 2012. :-)
That said, pkgsrc looks interesting, and I will be keeping an eye on it.
If I want an app, I download it. If I don't want it any more, I move it to the trash. If I really don't want it anymore, I use AppZapper. Can someone explain to me why the issue of package managers comes up so often around these parts and why they're so important to people?
[1] http://en.wikipedia.org/wiki/Package_management_system#Chall...
You're talking about apps in a relatively recent sense of the word: a wholly self-contained piece of software that you drop into the OSX /Applications folder.
Package managers, in the relevant sense, are for installing software (often called 'packages') into places that OSX normally hides from its users (/usr/local for example). A lot of this software is command line software, though on other operating systems, a package manager would also manage GUI packages. (On OSX there is this somewhat artificial distinction between /usr/local and /Applications.)
In any case, the kind of software pkgsrc, MacPorts or Homebrew installs requires more work than just "download it". You can download the source code easily enough, but you can't simply download the precompiled software itself, unzip it and drop it somewhere. You have to build it, and this will require lots of other packages and also a whole bunch of knowledge and or scripts to configure and build it correctly. Package managers like pkgsrc help to automate and simplify all of this.
If you've never needed anything like this, no great surprise, and likely you never will. But if you do need these things, then OSX poses some extra challenges. (Although as far as that goes, every os has its own issues, no doubt.)
Traced it to bmake/boot-strap, which after building bmake will run "bmake -r -m / test". Apparently bmake doesn't understand the arguments but, in typical Unixy fashion, will not say what it doesn't understand, but will instead output a "usage: bmake ..." and exit with a non-zero exit code, which causes everything to abort.
So I added some debug output to bmake's main.c and found that even though boot-strap invoked it with those arguments, bmake itself parsed them as "-j4 -w". Which is what I have MAKEFLAGS set to in my environment. I unset MAKEFLAGS and everything builds.
I love Unix, but I just hate how everything exists with so little separation and so much obscure brittleness, ending up in a situation where a tool called "bmake" will use options I intended for GNU Make (which doesn't have a .gnumakerc and so requires a global environment variable).