Steel Bank Common Lisp version 2.6.7 (sbcl.org)

> "AVX512 instructions are now supported on X86-64"

This is the first news I've seen on HN in weeks that I am genuinely excited about! I have several AVX-2 hobby projects in Common Lisp, and an AVX-512 machine. It's an unexpected surprise to read this morning that this very useful ISA is suddenly unlocked. I'll be trying it out right away.

(edit: Looks like they mean *compiler* support for AVX-512, but not SB-SIMD definitions (yet). So I believe the only way for end-users to call AVX-512 instructions right now is to write custom VOP's).

> "Or are these intrinsics you have to explicitly ask for?"

There are language SIMD types and you explicitly use SIMD functions that operate on them. I believe it's essentially the same idea as C intrinsics. You can for example write (interactively)

    (u32.8+ (make-u32.8 0 1 2 3 4 5 6 7)
            (u32.8 10))

    ;; => #<SB-EXT:SIMD-PACK-256   10   11   12   13   14   15   16   17>

And that's VPADDD under the hood. Or reading an array

    (loop with sum = (f32.8 0.0f0)
          for index below (* 8 (floor length 8)) by 8
          do (setq sum (f32.8+ (f32.8-aref array index)
                               sum))
          finally (return sum))

which compiles down to a small loop around the vector insts

    VMOVUPS YMM1, [RDX+RCX*2+1]
    VADDPS YMM0, YMM1, YMM0

I much prefer it to writing intrinsics in C (and the results are just as good). It's an interactive, exploratory, coding: I write small modular functions, SBCL compiles them on the fly, I glue them together with high-level language constructs.

> Is this at the codegen layer?

On implementation level, it is a codegen layer. It uses a system of macros to generate instructions from a database of instructions. The database is specified manually:

https://github.com/sbcl/sbcl/tree/master/contrib/sb-simd/cod...

At compile-time, they are converted into "VOPs", i.e. intrinsic functions, which are used by the compiler to emit the actual machine instructions.

> can it auto-vectorize or anything like that?

Unfortunately, it can't.

> are these intrinsics you have to explicitly ask for?

Yes, more like higher-level intrinsics. You get quite some automation, but you are requesting manually what you need. More like a DSL, than pure intrinsics. This is how you can use it (as an example):

(defun count-lines-and-words-ascii (sap size ws-init-state) (declare (type fixnum size) (type (unsigned-byte 8) ws-init-state) (type sb-sys:system-area-pointer sap) (optimize (speed 3) (safety 0))) (loop with loop-end of-type fixnum = (logandc2 size 127) for i of-type fixnum from 0 below loop-end by 128

    with 0x0  of-type u8.32 = (u8.32 #x00)
    with 0x20 of-type u8.32 = (u8.32 #x20)
    with 0x0A of-type u8.32 = (u8.32 #x0A)

    with wa of-type u64.4 = (u64.4 0)
    with la of-type u64.4 = (u64.4 0)

    with ws-prev of-type u8.32 = (u8.32 ws-init-state)

    for c1 = (u8.32-sap-ref sap (+ i 0))
    for c2 = (u8.32-sap-ref sap (+ i 32))
    for c3 = (u8.32-sap-ref sap (+ i 64))
    for c4 = (u8.32-sap-ref sap (+ i 96))

    do
       (flet ((process-chunk (curr prev)
                (let* ((ctrl  (u8.32-sat- (u8.32- curr 9) 4))
                       (ws (u8.32-or (u8.32= ctrl 0x0) (u8.32= curr 0x20)))

                       (ws-shift (u8.32-alignr ws (u8.32-permute128 prev ws #x21) 15))
                       (wmask    (u8.32-andc1 ws ws-shift))

                       (lmask (u8.32= curr 0x0A)))
                  (values wmask lmask ws))))

         (multiple-value-bind (wm lm prev) (process-chunk c1 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c2 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c3 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c4 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev)))
    finally
       (return
         (loop for j from loop-end below size

               with words of-type fixnum    = (sum-lanes wa)
               with lines of-type fixnum    = (sum-lanes la)
               with prev-ws of-type boolean = (logbitp 31 (u8.32-movemask ws-prev))
               with tlines of-type fixnum   = 0
               with twords of-type fixnum   = 0
               for byte of-type fixnum      = (sb-sys:sap-ref-8 sap j)
               for curr-ws of-type boolean  = (or (= byte 32) (<= 9 byte 13))

               do
                  (when (= byte 10) (incf tlines))
                  (when (and (not curr-ws) prev-ws) (incf twords))
                  (setf prev-ws curr-ws)
               finally
                  (return (values (the fixnum (+ lines tlines))
                                  (the fixnum (+ words twords))
                                  nil))))))

An extensible LLM agent (such as https://pi.dev/ or maybe hermes) written in common lisp could be interesting. Conditions and restarts and general debugging and repair of the live system, fast startup, native execution speeds, ability to add or replace or modify core functionality on the fly, solid multi-threading support, dynamic introspection including documentation, CLOS and multiple dispatch, saved images.

I bet you do, or you will. In my case, I had been getting movie recommendations from friends and also randomly. I'd look up the flick on IMDB, metacritic, and rotten tomatoes and I'd guess whether I would like it.

I wanted something more 'algorithmic' - and more accurate.

So me and Claude build an sbcl-based Film Recommendation system. Type in a new film name, it goes to open film database OMDB, grabs the scores, and then, using films I have already rated and with a built-in tiny Neural Net, gives me a personal recommendation. It uses the OMDB data and an algo that weights those with my personal values (in half a dozen areas: overall, acting, cinematography, etc), along with a 'comments' box to note for friends / others.

The code is very well-done CL, heavily commented. The film & ratings database is stored as S-exprs, naturally, all in one file. I don't use a database as I only have a few hundred films, but maybe later I will. And that's the point -- this is a living chunk of code that I from time to time bolt in new features, or try new things (e.g. the UI is localhost:8080)

I fretted about having a 'real' project to really dig into CL for a long time, and finally, found a meaty-enough project that ends up being really useful & quite a learning (continuing learning) experience. I start it up inside emacs/sly like this:

  (ql:quickload '(:hunchentoot :dexador :yason))
  (load "s:/filmrec.lisp")
  (filmrec:start)

I have to set the omdb key:

  (setf filmrec:*omdb-api-key* "fxxxx70")

Then, every so often, I retrain the NN:

  (filmrec:retrain)

Good Luck, you'll find a project. And with an LLM buddy, you'll succeed.

if you are pondering about that, then i think you'll like this:

https://ngnghm.github.io/

The worst thing that ever happened to Common Lisp was its ANSI standardization, which for many years killed almost all language innovation and improvement.

That would be a bad idea. In Smalltalk the "image" was the main thing and I think that was one thing that led to the commercail demise of Smalltalk. You can not combine two (or more) images. You can not build new things by using "images" as components. Instead as we know source-code modules with well-defined interfaces between them is what keeps productivity hhigh. At least before AI.

> How would a kubernetes optimized for lisp look like

I imagine it would look a bit like Erlang's BEAM. No need to stop and start the application, but write a script to do hot updates of a live image.

We would skip attrocity of JavaScript, JSON and XML.