Ask HN: How to deal with shipping AI-written code you don't understand?

Yep. I’d give the link but I think HN thinks I’m spamming because I’ve mentioned it too many times before in posts - hence immediate death of any submission, so I’m not mentioning it for a while to see if that tails off.

Bottom line: GPL3 for the compiler, standard library exclusions for derived works, so use it for anything but feed me back improvements to the compiler itself, and you don’t get to redistribute it as your own.

Upvotes don't appear publicly on HN so I'll comment to agree with this. I'm sorry if earlier information about this compiler got lost in the whirlwind. Data parallelism is a hard problem and these results look genuinely impressive so I for one would definitely be interested in hearing more about it and maybe discussing it further when there are interesting developments.

Ssshhh. I like to try things before promising them :)

I’m being a little cynical - I’ve submitted it twice (though the ‘C’ front end is new) and it bombed both times - the second time within seconds of submission.

I think it’s my own fault, I’ve posted the link to the compiler pages several times in what I thought were relevant posts, and I think HN thinks I’m spamming. So I’m not posting the home page for a while and we’ll see if a future submission “takes”

Well, the end product might be similar, but the approach is a bit different. 'xc' is a language in its own right, and it can be compiled (with xcc) to both the CPU and GPU (and I have plans to make it work on a Zynq too, a sort of HLS).

The C front-end I'm currently working on is something that looks at the LLVM IR that clang produces and converts it to xc-IR, then runs the data-parallel extraction through the same routines as the original xc language. Consider it to be a 'Y' branch on the inputs - you can feed it LLVM IR or xc source code, they both get convered to xc-IR and from then onwards it's all the same.

So yes, you get tiled work-units, overlapping in time between compute/data-transport and tuned so that actually works, and an opportunistic scheduler that sends them to whatever worker can handle it at the instant of being sent. That opportunistic nature means the faster cards (eg my 5080) will naturally get more work, because the scheduler just keeps all cards busy. The divide between the 5080 and the 3090 in my Windows test box is about 2:1.

But the scheduler is also aware of networked GPUs - my M4 max has a pretty fast GPU, but it's connected to my wife's M3 Max by thunderbolt, and I measured a 37 Gbit/sec connection, so my app can send data to be processed over that very fast link and get results back. If you have a 10GBit nic, then it will work pretty well too, if your nic is "only" 1Gbit then modern GPU speed means that unless your work-units are really computationally intensive, it'll be faster to do it locally - and xcd will work that out at runtime and adopt that faster approach.

     #use "Stdio.xc"
     #use "Par.xc"
   
     #define W 2048
   
     i32 main(void)
     {
         u32 total = 0;
         par mandel :reduce(+ total)
             {
             for (u32 i in 0..W * W)
                 {
                 float cx = (float)(i % W) * (3.0f / W) - 2.0f;
                 float cy = (float)(i / W) * (3.0f / W) - 1.5f;
                 float x = 0.0f;
                 float y = 0.0f;
                 u32 k = 0;
                 while (k < 256 && x * x + y * y < 4.0f)
                     {
                     float xt = x * x - y * y + cx;
                     y = 2.0f * x * y + cy;
                     x = xt;
                     k = k + 1;
                     }
                 total = total + k;
                 }
             }
         printf("%u iterations\n", total);
         return 0;
     }

Darn. Good catch.

I was all primed to give an explanation about some of the internals of the language, and after checking ... that's not whats going on here. What I thought was happening (and why I wasn't overly surprised by the saxpy results) was this:

- The above data is actually from my M4 Max MBP, not the PC/3090/5080

- xc has the concept of memory that is sufficiently aligned that it can be used directly with GPU data in a unified memory model. You just annotate the pointer with 'simd:', thus:

   simd:u8* ptr = new u8[4096]

And what I had assumed was happening was that the C front-end was annotating its buffer correctly, that the kernel didn't need to do any buffer-copying (which is what generally slows down saxpy) and that the GPU was just ripping through the data in-place and handing back control to the CPU.

That ... turns out not to be the case [grin]. The CFE is not marking buffers that get allocated on the heap with the simd: annotation (this is a bug, I will fix it), so you are absolutely correct and the numbers don't stand up for saxpy. Here's a more in-depth table for saxpy, and it shows where the error crept in:

     ┌───────────────────────────────────┬───────────┐
     │   saxpy, M4 Max (C, --par-auto)   │   Time    │
     ├───────────────────────────────────┼───────────┤
     │ clang -O2, one thread             │ 1,398 µs  │
     ├───────────────────────────────────┼───────────┤
     │ xcc, all CPU threads              │ 518 µs    │
     ├───────────────────────────────────┼───────────┤
     │ xcc, auto (picks all CPU threads) │ 593 µs    │
     ├───────────────────────────────────┼───────────┤
     │ xcc, GPU                          │ 11,240 µs │
     └───────────────────────────────────┴───────────┘

In fact the GPU-run's own report shows where the GPU time goes. Each saxpy pass takes about 0.8ms on the GPU itself, but just over 11ms end to end. Almost all of the difference is copying the two 64 MB arrays to and from the GPU-aligned buffer on every run. The 2.1× over clang is just memory bandwidth: one core can't saturate the M4 Max's memory bus and 16 can.

The 'auto' mode (the default unless you pin a kernel as must-use-this-execution-style) measures a few iterations of the work-tiles at runtime, and picks the fastest to run the full job with on the local hardware.

Bottom line: what is actually happening here is that the 'auto' mode (also in the original table header) is doing its measurement, and picking the fastest route - which is the "all CPU threads" mode. You can see the cost (I always run these tests after clearing out the persisted previous run, so there's always the measurement-cost in the 'auto' figure). What was misleading was the "(GPU)" in the original table header, I have to apologise for that.

Having said that, I'm glad you asked - in this case I don't think it'd make an actual difference (multi-threaded CPU still beats the non-buffer-transfer GPU) but potentially an integrated GPU (Mac or one of the "AI" Ryzens perhaps) might benefit from another, very slightly more complex compute, now that I know the automatic memory-alignment isn't working properly :)

It was already posted :) See https://news.ycombinator.com/item?id=50158272

Me. That would be me.

Based on a solid 6 months or so of testing, probing for weaknesses, and building a test to keep the system on the straight and narrow after finding a problem.

do I think the compiler is perfect ? No. No I don't. Is it as good as I can make it ? Why yes, yes it is.

Nothing is perfect. [1] is an issue I found that was 20 years old in gmime, no LLMs involved...

Vibe-coding actually might be the absolute best way to find vulnerabilities because the AI knows a lot more about them than you or I do, it can pattern-match against known anti-patterns, will do exhaustive searches without flagging halfway through, and is kept up-to-date by its ongoing weight-maintenance about new types of vulnerability.

What you have prompted me to do is add a pass to the 'make a release' script to ask it to search for vulnerabilities and report them, then put any found ones on the TODO list for the next pass.

1: https://github.com/jstedfast/gmime/issues/183