I don't think that's particularly surprising. If what we'd call a standard mechanism today (like using an option) had been in use first, why would there have been the need to change it to something that is done elsewhere at most rarely (special form of argv[0]) would have been introduced later?
Arbitrary or not, the limits had to be there due to small amount of memory computer had back then. I don’t think this is specific to Unix/C culture (unless it was what influenced the design of MS-DOS with its limit on the length of file names, for example).
you don't need megabytes to allocate your strings dynamically, microsoft basic did it on the altair in 4k
"/bin/sh\0" or "\7\0/bin/sh" is less than 14 bytes, not more, even including a pointer or two
the real issue is that anything you try to allocate dynamically can fail, so you need a failure path, and your failure path needs to reliably free all the other dynamically allocated resources
early unix kind of sucked at handling failures
a fixed-size buffer isn't more efficient but it is simpler
ms-dos used cp/m's filesystem, which had fixed-size filename and ext fields, which i think it copied from rt-11, which i think got them from various pdp-8 systems
fixed-size string buffers make more sense on a word-oriented machine, or arguably in a filesystem, where pointers require some kind of hack
i'd like to blame radix50 but i don't think we can because then the limit would be 9.3 or 6.3 rather than 8.3. there have been some 6.3 filesystems (3 16-bit words per filename with rad50) but cp/m fat (and thus ms-dos) are not among them
A great and easy idea, as long as you never need to release anything. Otherwise you end up with holes in your memory and you need to do compaction, and reserve the memory you need to be able to do that.
Obviously permanently wasting memory on fragmentation is not an option on a device that typically has 24kb memory. Especially not just so you can specify long filenames for the shell, when the filesystem doesn’t even support those filenames.
the alternative they chose was to permanently waste memory on fixed-size struct fields, so i guess it was actually an option
malloc on the pdp-11 never compacted of course
the filesystem supported 14 characters for the basename, but evidently we're talking about code with a 14-character limit on the pathname, not the basename
You don’t need compaction to reuse fixed size structures. I presume their priorities were with creating a working system, not one that answers to the ‘requirements’ of 50 years into the future.
Perhaps I wrong, but FAT and CP/M filesystems, aren't the same. CP/M filesystem never had directories. When the original FAT even predates QDOS as was used by Microsoft on his BASIC in 1977
I'd argue that in practical cases, resizable arrays require less memory than fixed-size arrays, because the actual length is usually going to be shorter than the arbitrary maximum length. My guess is that C encouraged using arbitrary limits by not having first-class support for resizable arrays (or even a non-hacky type-safe way to implement them as a library).
Oh I bet fun also played a role. But I don't even know what exactly you are comparing against the PDP-11 here or and what point, because UNIX development happened on 11s for at least a decade. UNIX first ran on a PDP-11 in 1971 or so and a decade earlier there was only CTSS perhaps to compare it to, which had more memory than a PDP-11 at the time. Later timesharing systems like Multics and TOPS-10 also ran on bigger machines. And let's not forget it all began on a PDP-7, which was even smaller.
Agreed, I think it was just a matter of being originally developed at a time when memory was tight and CPU cycles were few. That is, if the original UNIX were developed today with otherwise the same development culture, I'm not sure that we'd see fixed length restrictions.
One of GNU's original goals was eliminating arbitrary fixed size limits in UNIX tools, in exchange for using more memory. It's discussed in https://www.gnu.org/gnu/thegnuproject.en.html under "Technical goals".
> But it was natural to apply the known standards of good practice to the work—for example, dynamically allocating data structures to avoid arbitrary fixed size limits, and handling all the possible 8-bit codes wherever that made sense.
> In addition, we rejected the Unix focus on small memory size, by deciding not to support 16-bit machines (it was clear that 32-bit machines would be the norm by the time the GNU system was finished), and to make no effort to reduce memory usage unless it exceeded a megabyte.
The other advantage of fixed size buffers is if you want to write the records to disk you can just write it directly. You don't need to serialize or instantiate the strings when reading, you can just copy the bytes straight to or from the file.
Looking at things like man utmp.5 should reveal what their thinking was.
It's good that we moved on from such limits, but it's also important to remember that limit served a practical purpose back then and to take a lesson from that.
argv[0] is independent of the executable path. In fact there’s a common “trick” on Unix systems where you offer one binary and it figures out what it is supposed to be by peeking at argv[0]. Also, you’re allowed to freely modify argv, so if anyone reads it by scanning your stack it’ll get the new value.
int execl(const char *pathname, const char *arg, ...);
Normally you would call it like execl("program", "program", "argument", NULL), duplicating the program's name/path, but it's possible to set the 2nd argument, which corresponds to argv[0] (in the case of execl), to something different. This behaviour just isn't always exposed in the shell and other languages.
The command line version of exec offers "-a name", and the various library/syscall forms allow you to specify argv[0] separately from the path of the binary to exec.
You can set it yourself within a running program as well, like:
cool, i remember spending hours trying to understand the section in the bash manpage about profile and login shells, and seeing if it has security implications and also why just the shell behaves bezerk when deciding what bashrc or whatever file to use. turns out i just needed to read some assembly of some obsolete proprietary OS on some architecture ive never used.
39 comments
[ 0.33 ms ] story [ 145 ms ] threaddefshell -/usr/local/bin/tcsh
Notice the dash :)
In my opinion, these arbitrary character limits are a big wart of the Unix/C culture.
"/bin/sh\0" or "\7\0/bin/sh" is less than 14 bytes, not more, even including a pointer or two
the real issue is that anything you try to allocate dynamically can fail, so you need a failure path, and your failure path needs to reliably free all the other dynamically allocated resources
early unix kind of sucked at handling failures
a fixed-size buffer isn't more efficient but it is simpler
ms-dos used cp/m's filesystem, which had fixed-size filename and ext fields, which i think it copied from rt-11, which i think got them from various pdp-8 systems
fixed-size string buffers make more sense on a word-oriented machine, or arguably in a filesystem, where pointers require some kind of hack
i'd like to blame radix50 but i don't think we can because then the limit would be 9.3 or 6.3 rather than 8.3. there have been some 6.3 filesystems (3 16-bit words per filename with rad50) but cp/m fat (and thus ms-dos) are not among them
compaction doesn't require a lot of memory but it's slow and intrusive; it has to know where all your buffer pointers are
malloc on the pdp-11 never compacted of course
the filesystem supported 14 characters for the basename, but evidently we're talking about code with a 14-character limit on the pathname, not the basename
wasted memory due to heap fragmentation is very similar to wasted memory due to slack space in fixed-size structure fields
thanks for the correction
https://www.informit.com/articles/article.aspx?p=25878&seqNu...
fat didn't have subdirectories in qdos or ms-dos pre-2 either
One decade older hardware was better than newly released PDP-11...
Thankfully we have encyclopedias.
One of GNU's original goals was eliminating arbitrary fixed size limits in UNIX tools, in exchange for using more memory. It's discussed in https://www.gnu.org/gnu/thegnuproject.en.html under "Technical goals".
> But it was natural to apply the known standards of good practice to the work—for example, dynamically allocating data structures to avoid arbitrary fixed size limits, and handling all the possible 8-bit codes wherever that made sense.
> In addition, we rejected the Unix focus on small memory size, by deciding not to support 16-bit machines (it was clear that 32-bit machines would be the norm by the time the GNU system was finished), and to make no effort to reduce memory usage unless it exceeded a megabyte.
Looking at things like man utmp.5 should reveal what their thinking was.
You can set it yourself within a running program as well, like:
The problem is, all of the replacements since then have been worse.