Thanks apfel looks useful! I have been experimenting with Apple's foundation models for almost a year and they are useful for embedded applications. I have been taking a deeper dive into local agentic coding tools (starting with 'little-coder --model ollama/gemma4:12b-it-qat') and I put together a tiny free book with some setup advice that might save people a few minutes of setup time: https://leanpub.com/read/local-coding-agents
I have been fairly much pissed off at the "hype in hyperscaler" AI growth (data center environmental and other societal costs) and I support anything we can do to promote local and private AI.
Agreed. The idea of a system wide (and platform wide) on device model being a core part of OS APIs is very appealing. I do like my software more piecemeal, generally, but when it comes to Apple, I really love a lot of the out-of-the-box offerings they have. Just giving software access to something they know exists on these platforms and can use for various small (and likely increasingly large) gen AI tasks is so appealing.
This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it
In the coding realm, I think we'll be seeing 35, 70, and 150B models sold where you pay a few hundred to a few thousand dollars up front and get a year of monthly/bi-monthly updates where they've trained it on new coding documentation and repos.
Is there something like this on Linux? For example, if I’m an application developer can I assume GNU Core AI (or whatever it is or would be called) will be there if the kernel is >= some particular version?
AI future is clearly local, and my recent pitch has been "infinite tokens." Because that's what my M1 MBP can do; and that's what my RTX3090 can do. I don't need to pay hundreds of dollars a month and no one else does either.
they are also working on activations (w4a8, w4a16 from what i know). if they deliver (and a big if), it means that given their market reach, they can dictate the way sub 100b parameter models are trained and served to a large extent, given their major usecase would be on device (macos and not ios for most of them).
Free server-size model access for apps with <2M downloads, getting the same privacy guarantees. Hopefully they scale this up to all apps in time (I assume hardware/cost constrained, but larger devs would pay).
something I haven't seen highlighted anywhere yet, while I find it very interesting, is the distributed inference across Macs (JACCL over Thunderbolt 5), an OpenAI-compatible mlx_lm.server, agentic-on-Mac.
Apple keeps MLX (bring-your-own-weights) separate from Foundation Models / Core AI.
17 comments
[ 3.7 ms ] story [ 48.6 ms ] threadDoes this completely replace the previous API, CoreML? [1]
Meet Core AI - https://developer.apple.com/videos/play/wwdc2026/324/
Dive into Core AI model authoring and optimization - https://developer.apple.com/videos/play/wwdc2026/325/
Integrate on-device AI models into your app using Core AI - https://developer.apple.com/videos/play/wwdc2026/326/
but i maintain https://github.com/Arthur-Ficial/apfel so i might be biased
I have been fairly much pissed off at the "hype in hyperscaler" AI growth (data center environmental and other societal costs) and I support anything we can do to promote local and private AI.
https://developer.apple.com/private-cloud-compute/
Apple keeps MLX (bring-your-own-weights) separate from Foundation Models / Core AI.