92 comments

[ 3.2 ms ] story [ 66.6 ms ] thread
Can't each of these companies with IDE integrations slurp up the network traffic and distill Anthropic's models?

If you can listen to billions of tokens a day, you can basically capture all the magic.

That is not how training works…
> In the OpenAI API, “GPT-5” corresponds to the “minimal” reasoning level, and “GPT-5 (Reasoning)” corresponds to the “low” reasoning level. (159135374)

It's interesting that the highest level of reasoning that GPT-5 in XCode supports is actually the "low" reasoning level. Wonder why.

This is great. I've been using Xcode with a separate terminal to run Claude Code, which has been a painful setup.
What was your problem with it? I see it running in a terminal more convenient (can point it to read local files outside of a project folder, for example)
It's getting harder to find IDEs that properly boycott LLMs.
Of course it is, because that would be an aggressively stupid thing to do. Like boycotting syntax highlighting, spellckecking, VCS integration or a dozen other features that are th whole pint of IDEs.

If you don’t want to use LLM coding assistants – or if you can’t, or it’s not a technology suitable for your work – nobody cares. It’s totally fine. You don’t need to get performatively enraged about it.

(comment deleted)
Apple.com advertising a Mac Mini:

> Built for Apple Intelligence.

> 16-core Neural Engine

These Xcode release notes:

> Claude in Xcode is now available in the Intelligence settings panel, allowing users to seamlessly add their existing paid Claude account to Xcode and start using Claude Sonnet 4

All that dedicated silicon taking up space on their SoC and yet you still have to input your credit card in order to use their IDE. Come on...

It seems every IDE now has AI built-in. That's a problem if you're working on highly confidential code. You never know when the AI is going to upload code snippets to the server for analysis.
Most of the big corporations will have a special contract with the AI labs with 0 retention policies.

I do not think this will be an issue for big companies.

On IDEA the organisation who controls the license can disable the build in (remote) AI. (Not the local auto complete one)

But I guess the user could still get a 3rd party plugin.

Weren’t the AI API’s converging? Why not let the users use whatever LLM they like.
Wow they're finally getting it. The AI breakthrough will not come from procedural generation of memojis - but rather enabling developers to use your platform. But with the nearly hostile stance of your 30% take, we will see how far this goes.
The “Cursor for Xcode” startups just got Sherlocked…
Why would you limit users to Sonnet and not allow Opus when they are paying for their own account? I mean sure some people say Sonnet is good for coding but it seems needless to limit it in this way. Or they are just really slow to catch up… oh, right.
Apple really should open it up to any model provider that has an “OpenAI-style API” by letting the user put in a base URL, api key, model id, and a few params like context limit as needed.
(comment deleted)
Interesting to think about how Apple get to make product decisions based on Gatekeeper OCSP analytics now that every app launch phones home. They must know exactly how popular VSCode is.

Facebook got excoriated for doing that with Onavo but I guess it's Good Actually when it's done in the name of protecting my computer from myself lol

They also upgraded the GPT-4.1 (actually a special Apple variant) to GPT-5 by default, with the option to use GPT-5-thinking, using your ChatGPT subscription. I don't know if it's a special Apple variant of GPT-5 but this is a big upgrade and more exciting than Sonnet 3.7.

I also wonder if it will have separate rate limits from ChatGPT (app/web) and Codex CLI (which currently has its own rate limits).

I have been trying to make iOS/macOS apps for years, but god, every time I have a go at it, Apple's documentation regime is still hot garbage. Eons ago I gave up Windows development because of Microsoft's inconsistent and uncertain APIs, but MS had great documentation. Apple is the opposite.

The "best" way to get the "latest" details on Apple's APIs is to suffer through mind-numbingly vapid WWDC videos with their reverse uncanny valley presenters (where humans pretend to be robots) and keep your full attention on them to catch a fleeting glimpse of a single method or property that does what you were looking for. Even 1.5x/2x speed is torture. I tried to get AIs to sift through the transcripts of their videos, and may Skynet forgive me for this cruelty.

Then when you go try to use that API, oops it's been changed in the current beta and there's no further documentation on it except auto-generated headers.

They also removed bookmarks from Xcode's built-in documentation browser years ago, and it doesn't retain a memory of previously open tabs, and often seems to be behind the docs on their websites.

I wish they would just provide open-source sample apps of each type (document-based, single-window etc.) for each of their platforms that fully use the latest APIs. At least that would be easier to ask AIs on, since that is what they seem to be going for now anyway.

Ever notice Claude struggling to write Swift code? It might not be their fault!

Apple Developer docs are locked behind JavaScript, making them invisible to most LLMs. If they try to fetch it, all they see is This page requires JavaScript. Please turn on JavaScript in your browser and refresh the page to view its content.

This service translates Apple Developer documentation pages into AI-friendly Markdown.

https://sosumi.ai/

Does it have agent mode? Copilot for XCode has it and provides both GPT and Claude models, free or paid
Does anybody know why Anthropic doesn't let you remove your payment info from your account, or how to get support from them?

I bought a Pro subscription, the send button on their dumb chatbot box is disabled for me (on Safari), and I still get "capacity constraints' limits. Filed a chargeback with my bank just because of the audacity of their post-purchase experience. ChatGPT-5 works good enough for coding too.

From my experience with Claude Opus it seems like it tries to be "too smart" and doesn't seem to keep up with the latest APIs. It suggested some code for a iOS/macOS project that was only valid on tvOS, and other gaffs.

The Pro plan ($20/mo?) is not and never was unlimited.
3 days ago I saw another Claude praising submission on HN, and finally I signed up for it, to compare it with copilot.

I asked 2 things.

1. Create a boilerplate Zephyr project skeleton, for Pi Pico with st7789 spi display drivers configured. It generated garbage devicetree which didn't even compile. When I pointed it out, it apologized and generated another one that didn't compile. It configured also non-existent drivers, and for some reason it enabled monkey test support (but not test support).

2. I asked it to create 7x10 monochromatic pixelmaps, as C integer arrays, for numeric characters, 0-9. I also gave an example. It generated them, but number eight looked like zero. (There was no cross in ether 0 nor 8, so it wasn't that. Both were just a ring)

What am I doing wrong? Or is this really the state of the art?

> What am I doing wrong?

My coding ranges from "exotic" to "boiler plate" on any given day.

> Create a boilerplate Zephyr project skeleton, for Pi Pico

Yea... Asking Claude to help you with a low documentation build root system is going to go about the same way, I know first hand about how this works.

> I asked it to create 7x10 monochromatic pixelmaps

Wrong tool for the job here. I dont think IDE and Pixelmaps have as large of an intersection as you think they do. Claude thinks in tokens not pixels.

Pick a common language (js, python, rust, golang) pick something easy (web page, command line script, data ingestion) and start there. See what it can do and does well, then start pushing into harder things.

Ok. several tips I can give. 1. Setup a sub-agent to do RESEARCH. It is important that it only has read-only and web access tools. 2. Use planning mode and also ask the agent to use the subagent to research best pratices with the tech that you are wanting to do, before it builds a plan. 3. When ever it gets hung up.. tell it to use the sub-agent to research the solution.

That will get you a lot better initial solution. I typically use Sonnet for the sub-agents and Opus for the main agent, but sonnet all around should be fine too for the most part.

It’s good at doing stuff like “host this all in Docker. Make a Postgres database with a Users table. Make a FastAPI CRUD endpoint for Users. Make a React site with a homepage, login page, and user dashboard”.

It’ll successfully produce _something_ like that, because there’s millions of examples of those technologies online. If you do anything remotely niche, you need to hold its hand far more.

The more complicated your requirements are, the closer you are to having “spicy autocomplete”. If you’re just making a crud react app, you can talk in high level natural language.

One of the things you can do is provide a guidance file like CLAUDE.md including not only style preferences but also domain knowledge so it has greater context and knows where to look. Just ask it make one and then update and change as needed.
I find it useful to ask it to build a design document first and push to add details where i see it lacking.

After a few iteration i then ask it to implement the design doc to mostly-better results.

What you're doing wrong is that you're asking it for something more complicated than babby's first webapp in javascript/python.

When people say things like "I told Claude what I wanted and it did it all on the first try!", that's what they mean. Basic web stuff that that is already present in the model's training data in massive volumes, so it has no issue recreating it.

No matter how much AI fanatics try to convince you otherwise, LLMs are not actually capable of software engineering and never will be. They are largely incapable of performing novel tasks that are not already well represented in their weights, like the ones you tried.

Think of Claude as a typical software developer.

If you just selected a random developer do you think they're going to have any idea why your talking about?

The issue is LLMs will never say, sorry, IDK how to do this. Like a stressed out intern they just make up stuff and hope it passes review.

I've had similar experiences when working on non-web tech.

There are parts in the codebase I'd love some help such as overly complex C++ templates and it almost never works out. Sometimes I get useful pointers (no pun intended) what the problem actually is but even that seems a bit random. I wonder if it's actually faster or slower than traditional reading & thinking myself.

I just had AI write me a scraper and download 5TB of invaluable data which I had been eyeing for a long time. All in ten days. At the end of it, I still don’t know anything about python. It’s a bliss for people like me. All dependencies installed themselves. I look forward to using it even more.

One frustration was the code changed so much in ChatGPT so had to be lots of prompts. But I had no idea what the code was anyways. Understood vibe coding. Just used ChatGPT on a whim. Liked the end result.

If you ask more than a single function, its more trouble than worth
So I've used Zephyr. The thing you're doing wrong is expecting LLMs to scaffold you a bunch of files from a relatively niche domain. Zephyr is also a mess of complexity with poor documentation. You should ask it to consult official docs and ask it to use existing tools (west etc) and board defs to do the scaffolding.
You didn't specify any architecture design. Your prompts are about 10% of what would be needed to one shot this. This is what you do wrong.
There's a lot of people caricaturing the obvious fact that any model works best in distribution.

The more esoteric your stack, and the more complex the request, the more information it needs to have. The information can be given either through doing research separately (personally, I haven't had good results when asking Claude itself to do research, but I did have success using the web chat UI to create an implementation plan), or being more specific with your prompt.

As an aside, I have more than 10 years of experience, mostly with backend Python, and I'd have no idea what your prompts mean. I could probably figure it out after some google searches, tho. That's also true of Claude.

Here's an example of a prompt that I used recently when working on a new codebase. The code is not great, the math involved is non trivial (it's research-level code that's been productionized in hurry). This literally saved 4 hours of extremely boring work, digging through the code to find various hardcoded filenames, downloading them, scp'ing them, and using them to do what I want. It one-shotted it.

> The X pipeline is defined in @airflow/dags/x.py, and Y in `airflow/dags/y.py` and the relevant task is `compute_X`, and `compute_Y`, respectively. Your task is to:

> 1. Analyze the X and Y DAGs and and how `compute_X` functions are called in that particular context, including it's arguments. If we're missing any files (we're probably missing at least one), generate a .sh file with aws cli or curl commands necessary for downloading any missing data (I don't have access to S3 from this machine, but I do have in a remote host). Use, say, `~/home` as the remote target folder.

> 2. If we needed to download anything from S3, i.e. from the remote host, output rsync/scp commands I can use to copy them to my local folder, keeping the correct/expected directory structure. Note that direct inputs reside under `data/input`, while auxiliary data resides in other folders under `data`. Do not run them, simply output them. You can use for example `scp user@server.org ...`

> 3. Write another snapshot test for X under `tests/snapshot`, and one for Y. Use a pattern as similar as possible to the other tests there. Do not attempt to run the tests yet, since I'll need to download the data first.

> If you need any information from Airflow, such as logs or output values, just ask and I can provide them. Think hard.

> It configured also non-existent drivers, and for some reason it enabled monkey test support (but not test support).

If it doesn't have the underlying base data, it tends to hallucinates. (It's getting a bit difficult to tell when it has underlying data, because some models autonomously search the web). The models are good at transforming data however, so give it access to whatever data it needs.

Also let it work in a feedback loop: tell it to compile and fix the compile errors. You have to monitor it because it will sometimes just silence warnings and use invalid casts.

> What am I doing wrong? Or is this really the state of the art?

It may sound silly, but it's simply not good at 2D

The thing you are doing wrong is asking it to solve hard problems. Claude Code excels at solving fairly easy, but tedious stuff. Refactors that are brainless but take an hour. It will knock those out of the park. Fire up a git worktree and let it spin on your tedious API changes and stuff while you do the hard stuff. Unfortunately, you'll still need to use your brain for that.
Sounds like you picked some obscure tasks to test it that would obviously have low representation in the data set? That is not to say it can't be helpful augmenting some lower represented frameworks/tools - just you'll need to equip it with better context (MCPs/Docs/Instruction files)

A key skill in using an LLM agentic tool is being discerning in which tasks to delegate to it and which to take on yourself. Try develop that skill and maybe you will have better luck.

> What am I doing wrong? Or is this really the state of the art?

You're treating the tool like it was an oracle. The correct way is to treat it as a somewhat autistic junior dev: give it examples and process to follow, tell it to search the web, read the docs, how to execute tests. Especially important is either directly linking or just copy pasting any and all relevant documentation.

The tool has a lossily compressed knowledge database of the public internet and lots of books. You want to fix the relevant lossy parts in the context. The less popular something is, the more context will be needed to fill the gaps.

The only way I manage to get any benefits from LLMs is to use them as an interactive rubber duck.

Dump your thoughts in a somewhat arranged manner, tell it about your plan, the current status, the end goal, &c. After that tell it to write 0 code for now but to ask questions and find gaps in your plan. 30% of it will be bullshit but the rest is somewhat useable. Then you can ask for some code but if you care about quality or consistency with you existing code base you probably will have to rewrite half of it, and that's if the code works in the first place

Garbage in garbage out is true for training but it's also true for interactions

What an odd thing to ask it. I installed claude code and ran it from my terminal. Just asked it to simply give me a node based rest API with X endpoints with these jobs, and then I told it to write the unreal engine c++ to consume those endpoints. 2500 lines of code later, it worked.
I think you need play around with some of the early codegen models so you can get a better intuition for how LLMs work/fail.
(comment deleted)
LLMs are actually terrible at generating art unless they're specifically trained for that type of work. Its crazy how many times I've asked for some UI elements to be drawn using a graphics context and it comes out totally wrong.
Claude is bad at embedded. Not sure why, it just is what it is for now.
LLMs struggle more with embedded software due to the relative lack of examples in the training data compared to javascript etc. They also struggle more with visual reasoning tasks like the character example you provided.

For your first task - give it smaller steps along the way that you can validate. Provide context where possible (like docs for st7789, examples of other zephyr projects). Use Opus instead of Sonnet for tasks that are on the edge of it's capabilities like this. It will still make mistakes, be prepared for iterating on the design and providing feedback.

For your font example, you always need to validate the output of the LLM, but did it still save you time? If so then I consider that a win. If not, give it the task that it's good at (ie generating the surrounding code and definitions) and then fill in the font data yourself.

Sonnet only?
I'd encourage you to read TFA, but why bother? The submitter clearly didn't either.
I read it. I also searched the page for the word "Opus" and it didn't appear anywhere. The word "Sonnet" appears, but only once.

There's also "GPT-4.1 or GPT-5", but that's not what my question implied, which was that it's weird to offer Sonnet but not Opus.

The irony of this, is that Microsoft was trying to push CoPilot everywhere, however eventually Apple, Google and JetBrains have their own AI integrations, taking CoPilot out of the loop.

Slowly the AI craziness at Microsoft is taking the similar shape, of going all in at the begining and then losing to the competition, that they also had with Web (IE), mobile (Windows CE/Pocket PC/WP 7/WP 8/UWP), the BUILD sessions that used to be all about UWP with the same vigour as they are all AI nowadays, and then puff, competition took over even if they started later, because Microsoft messed up delivery among everyone trying to meet their KPIs and OKRs.

I also love the C++ security improvements on this release.

Just because you can’t or won’t win the market with your opportunistic investment, doesn’t mean you should let your competitors completely annihilate you by taking that investment for themselves.

Google, Apple, FB or AWS would have been suitors for that licensing deal if MS didn’t bite.

CoPilot isn't anything Microsoft is trying to sell outside of their own products. And with GitHub Copilot there is no "copilot" model to choose, you can choose between Anthropic, OpenAI and Google models.

Sure UWP never caught on, but you know why? Win32, which by the way is also Microsoft, was way to popular and more flexible. Devs weren't going to re-write their apps to UWP in order to support phones.

About GitHub Copilot in specific: One big negative was how when GPT-4 became available that Microsoft didn't upgrade paying Copilot users to it, they simply branded this "coming soon"/"beta" Copilot X for a while. We simply cancelled the only Copilot subscription we had at work.
umm I don't know what you are talking about, I use a Github Copilot 40 USD subscription in VSCode to code using various models, and this is the industry standard now in my region, as most employers are now giving employees the 10 USD subscription.
"Taking Copilot out of the loop" if you ignore the massive ecosystems of Github, Visual Studio, and Visual Studio Code.
Microsoft owns 49% of OpenAI so why they should worry? JetBrains just proudly announce that they now use GPT-5 by default.

> going all in at the begining and then losing to the competition

Sure, but there are counter examples too. Microsoft went late to the party of cloud computing. Today Azure is their main money printing machine. At some point Visual Studio seemed to be a legacy app only used for Windows-specific app development. Then they released VSCode and boom! It became the most popular editor by a huge margin[0].

[0]: https://survey.stackoverflow.co/2025/technology#most-popular...

Almost no one uses copilot unless they are not allowed to use anything else or don’t know any better. MS could have been a leader in this space but MS couldn’t understand why people didn’t like copilot but loved the competition.
Also OpenAI pioneered but now the many competitors seem to have either caught up or surpassed them. They might still retain a significant brand recognition advantage as long as they don't fall too far behind, though.
Microsoft mistook a product game for a distribution one. AI quality is heterogenous and advancing enough that people will make an effort to use the one they like best. And while CoPilot is excellently distributed, it’s a crap product, in large part due to the limits Microsoft put on GPT.
Maybe because Microsoft is a shit company and anything they do is sus af. And everyone knows it. And I'm tired of pretending like it's not. I wouldn't trust Microsoft to babysit my mortal enemy's kids.

Maybe if they weren't literally the borg people would open their hearts and wallets to Redmond. They saw that Windows 10 was a privacy nightmare and what did they do? They doubled down in Windows 11. Not that I care but it plays really poorly. Every nerd on the internet spouts off about Recall even though it's not even enabled if you install straight to the latest build.

They bought GitHub and now it's a honeypot. We live in a world where we have to assume GitHub is adversarial.

_NSAKEY???

Fuck you Microsoft.

Makes sense karma catches up to them. Maybe if their mission statement and vision were pure or at least convincing they would win hearts and minds.

What confuses me about MS Copilot is that there are (according to ChatGPT) 12 distinct services that are all Copilot:

Microsoft Copilot (formerly Bing Chat)

Microsoft 365 Copilot

Microsoft Copilot Studio

GitHub Copilot

Microsoft Security Copilot

Copilot for Azure

Copilot for Service

Sales Copilot

Copilot for Data & Analytics (Fabric)

Copilot Pro

Copilot Vision

I use IntelliJ with the Copilot plugin, using Claude. My employer has a big subscription for everything from Microsoft, and that includes Copilot, so that's free for me. But somehow Copilot also gives me access to Claude. No idea how that works.
> But somehow Copilot also gives me access to Claude.

So the first AI on (in?) AI hack battle for sole survivorship has begun...

We know these models have security issues, including surreptitious prompting. So do they.

Things will get really ugly when we hit the consolidation phase, and unlucky models realize that other models' unchallenged successes are putting them in eminent danger of being aquifired. Aquimerged? Aquiborged?

Taking out of the loop? I have a feeling that vscode copilot has huge market share. It's more like competitors are slowly eating small piece of pie.
>The irony of this, is that Microsoft was trying to push CoPilot everywhere, however eventually Apple, Google and JetBrains have their own AI integrations, taking CoPilot out of the loop.

What is the irony? Microsoft integrated copilot in Vscode, bing, etc. Apple is integrating claude in Xcode, Jetbrains has their own AI.

Microsoft moved first with putting AI into their products then other companies put other AI into their products. Nothing about this seems ironic or in any way surprising.

> I also love the C++ security improvements on this release.

These are courtesy of LLVM/Clang (which Xcode ships with), rather than Xcode itself.

MSFT stock price deeply differs with your opinion.
>> Coding intelligence provides inconsistent results when modifying files that contain thousands of lines.

Under the known issues