14 comments

[ 3.1 ms ] story [ 45.4 ms ] thread
I love a good history lesson. :)

In case it helps anyone: this is about a version 2 of the original LISP language. It's not about the distinction between a "Lisp-1" and a "Lisp-2", which describe two different families of Lisp languages (those with a single namespace, like Scheme; and those with separate namespaces for values and functions, like Common Lisp).

From the title I thought that a new version of common lisp was being made.
When your language has to include a built-in swapper to manage the memory you know you are suffering from the second system effect.

> The Q-32 version was so cramped for space (approximately 48,000 48-bit words) that an application-level swapper was written: If a function was needed but not in memory 1) if memory space was available, the function was paged in by LISP from an ordinary data file, 2) if space was not available, the memory manager tried to create space by shuffling the in-memory area holding binary code to make space, if this did not work some in-memory code was excised or the GC invoked then 3) the dynamic loader read the binary code into memory.

Turbo Pascal had a system like that called "Overlays" back in the 640 kB memory days. If you had multiple code modules of which just one was in place at a time, you could overlay them so just one got loaded at a time.

While that's before my time (Turbo Vision in 6.0 was when I started), TP 2 and 3 supported only COM files which could have only 64 kilobytes of code, making overlay extra important.

The original MacOS from 1984 (?) also had such a feature, IIRC: the segment loader.
Before demand paging or any other virtual memory it was very common technique. Lots of use on PDP-11 or IBM S/360.
Overlays were common on MS-DOS in the 80s.

A typical scheme was to have a configuration file / script for the linker. You would list one or more blocks, each of which had a list of mutually exclusive .obj files to be swapped in and out as needed. It could be challenging to track dependencies between objects to make sure you never called between modules which were supposed to be mutually exclusive, which was why you sometimes had multiple swap buffers.

Later linkers, or proprietary schemes like in Turbo Pascal version 5, would swap things per subroutine, rather than per compilation unit (.obj file), which was much easier to use.

(I did once see some people make a C function that was literally over 50 printed pages long, and the overlay manager didn’t help them much with OOM, among other problems that thing had)

When your language has to include a built-in swapper to manage the memory you know you are suffering from the second system effect.

I disagree.

Today, this functionality is so widely needed that it is built into every operating system. Therefore applications that rebuild it either have a specialized need or are over-engineered.

But back then, operating systems did not have that functionality. And it was pretty easy to hit the limits of RAM. So it then becomes reasonable to build the feature.

Is this not just a form of virtual memory?

Edit: never mind, maybe it's not. More like a specialized allocator.

It is, just more fine grained and integrated with GC - normal swappers have to work blind.
I wonder how far one could get combining the strengths of Haskell's bidirectional type system and Dlang's (http://dlang.org/) efficiency into Lisp to get a fast language.