They need to focus on fixing reliability first. Their systems constantly go down and it appears they are having to quantise the models to keep up with demand, reducing intelligence significantly. New features like this feel pointless when the underlying model is becoming unusable.
Does Microsoft Copilot do this already? Isn't it integrated into Windows and MSFT Office products? Has it been working out for Copilot? Is it helpful? Adoption rates of AI are interesting to say the least.
I noticed the other day that chatgpt started preferring to provide me with a download link for code rather than putting it up in canvas. It also started offering me diffs, but as I just write fairly basic data munging scripts for neuroimaging analyses, I don't like to dive too deep into the coding tool boxes/chains...copy paste is easy...although, I would like versioning without making copies of my script for backup
Now we see where these ai foundation companies are heading. They are literally building the next operating system to replace the old gatekeepers, similarly like netscape tried to do with microsoft in the 90's
If the final Claude goal is to remove human from the process (IA can do everything), what's the point of having these files? If they are going to be feed again to a model to interpret them, wouldn't be better to use something simpler/easier to parse?
I tested this feature out today, applying the same prompt and CSV data to both Claude Opus 4.1 and GPT-5-Thinking. They both chugged away writing Pandas code and produced similar output. It's nice to have another option for data analysis to act as a second opinion on GPT, if nothing else.
Is it able to process a prompt on each file in a folder-full of files and then return the collated results?
That's the functionality which I could use for my day job, but I'm not finding an LLM which directly affords that capability (without programming or other steps which are difficult on my work computer).
It looks to me like a variant of the Code Interpreter pattern, where Claude has a (presumably sandboxed) server-side container environment in which it can run Python. When you ask it to make a spreadsheet it runs this:
What's weird is that when you enable it in https://claude.ai/settings/features it automatically disables the old Analysis tool - which used JavaScript running in your browser. For some reason you can have one of those enabled but not both.
The new feature is being described exclusively as a system for creating files though! I'm trying to figure out if that gets used for code analysis too now, in place of the analysis tool.
Odds are the new container and old JavaScript are using the same tool names/parameters. Or, perhaps, they found the tools similar enough that the model got confused having them both explained.
This will either result in a lot of people being able to sleep more, or an absolute avalanche of crap is about to be released upon society.
A lot of the people I graduated with spent their 20s making powerpoint and excel. There would be people with a master's in engineering getting phone calls at 1am, with an instruction to change the fonts on slide 75, or to slightly modify some calculation. Most of the real decision making was, funnily enough, not based on these documents. But it still meant people were working 100 hour weeks.
I could see this resulting in the same work being done in a few minutes. But I could also see it resulting in the MDs asking for 10x the number of slide decks.
the way it edits powerpoints is by launching a command line environment, and then editing the OOXML directly using command line tools. this takes several minutes to do even simple changes.
to me it seems miraculous that it even "sort of" works, but also it's not a reliable product yet. OOXML is very complex and the formatting can get mangled.
On the other hand, if you use LaTeX/Beamer slides, LLMs can reliably make a lot of formatting tweaks etc. and it is an actual time saver. But only weird academics use Beamer.
I agree with Simon Willison that this feature is really about writing code in a container, using that capability to edit PPT presentations as if they were markup is an odd thing to make the primary selling point.
A smell of changing strategy? Claude has been the favourite of engineers and it seems it’s now trying to win back the general consumer market where ChatGPT has taken the majority. But at the cost of Claude code? Codex is like a shark chasing CC nowadays.
For the past two to three weeks I've noticed Claude just consistently lagging or potentially even being throttled for pretty minor coding or CLI tasks. It'll basically stop showing any progress for at least a couple minutes. Sometimes exiting the query and re-trying gets it to work but other times it keeps happening. I pay for Pro so I don't think it's just API rate limiting.
Would appreciate if that could be fixed but of course new features are more interesting for them to prioritize.
Same. My usage is via an internal corp gateway (Instacart), Sonnet 4. Used to be lighting fast, now getting regular slow downs or outright failures. Not seeing it with the various GPT models.
I use Claude Code at work via AWS bedrock, also personally subscribe to the $20/month Claude. Anecdotallt, Sonnet hasn't slowed down at all. ChatGPT 5 through enterprise plan, on the other hand, has noticeably slowed down or sometimes just not return anything.
I've run into similar issues too. Even small scripts or commands sometimes get throttled. It does not feel like a resource limit. It feels more like the system is just overly sensitive.
Not Claude specific, but related to the agent model of things...
I've been paying $10/month for GitHub Copilot, which I use via Microsoft's Visual Studio Code, and about a month ago, they added ChatGPT5 (preview), which uses the agent model of interaction. It's a qualitative jump that I'm still learning to appreciate in full.
It seems like the worst possible thing, in terms of security, to let an LLM play with your stuff, but I really didn't understand just how much easier it could be to work with an LLM if it's an agent. Previously I'd end up with a blizzard of python error messages, and just give up on a project, now it fixes it's own mess. What a relief!
Wasn't this already doable? Via instructing the llm to output as PDF xml or PowerPoint markup etc and writing (with AI assistance) the glue layer. It's not nothing but also not that difficult. I don't see how Claude's version of this can be much better
Anyone else having serious reliability issues with artifact editing? I find that the artifacts quite often get "stuck", where the LLM is trying to edit the artifact but the state of the artifact does not change. Seems like the LLM is somehow failing in editing the artifact silently, while thinking that it is actually doing the edits. The way to resolve this is to ask Claude to make a new artifact, which then has all the changes Claude thought it was making. But you have to do this relatively often.
I saw this yesterday. I was asking it to update an SQL query and it was saying, 'I did this' and then that wasn't in the query. I even saw it put something in the query and then remove it, and then say 'here it is'.
Maybe it's because I use the free tier web interface, but I can't get any AI to do much for me. Beyond a handful of lines (and less yesterday) it just doesn't seem that great. Or it gives me pages of javascript to show a date picker before I RTFM and found it's a single input tag to do that, because it's training data was lots of old and/or bad code and didn't do it that way.
This has been super annoying! I just tell it to make sure the artifact is updated and it usually fixes it, but it's annoying to have to notice/keep an eye on it.
It edits it for me but it tries to edit it "in place" where it messes up the version history and it looks very broken and often times is broken afterwards. Don't know why they broke their best feature while ChatGPT Canvas just works.
My experience is similar. At first Claude was super smart and get even very complicated things right. Now even super simple tasks are almost impossible to finish right, even if I really chop things into small steps. Also it's much slower even on Pro account than a few weeks ago.
I'm on the $200 / month account and its also slower than a few weeks ago. And struggling more and more.
I used to think of it as a decent sr dev working alongside me. Not it feels like an untrained intern that takes 4-5 shots to get things right. Hallucinated tables, columns, and HTML templates are its new favorite thing. And calling things "done" that aren't even half done and don't work in the slightest.
Also yesterday tried to use it to debug some AWS issue and it tried to send me down so many wrong paths, and suggested changes that were either plain wrong or had unintended consequences, that if I didn't actually know my stuff and had followed blindly, the results would have been pretty bad or at least a huge time waster. When I called it out it would quickly reverse course ("You're right of course!") and it did provide some helpful snippets but I was unimpressed.
What I find it excellent at is for throw-away scripts to do small jobs or automate little things--stuff I could do but would take me a lot longer (especially in bash).
Wow, that's like...a huge deal. It's a major feat of engineering when some software can create and edit files. That's like half of CRUD! Seems like they are really advanced, like magic!
I just published an extensive review of the new feature, which is actually Claude Code Interpreter (the official name, bafflingly, is Upgraded file creation and analysis - that's what you turn on in the features page at least).
I reverse-engineered it a bit, figured out its container specs, used it to render a PDF join diagram for a SQLite database and then re-ran a much more complex "recreate this chart from this screenshot and XLSX file" example that I previously ran against ChatGPT Code Interpreter last night.
These days, I spend time training people using this kind of tools. I am glad it's called as such. It's much comfortable to explain to a tech person that it's "badly named" and that it should have been named "Code Interpreter" instead than explaining to a non tech that the "Code Interpreter" feature is a new cool way to generate documents. Most people are not that comfortable with technology, so avoiding big words is a nice to have.
74 comments
[ 3.1 ms ] story [ 76.1 ms ] threadIt can actually drive emacs itself, creating buffers, being told not to edit the buffers and simply respond in the chat etc.
I actually _like_ working with efrit vs other LLM integrations in editors.
In fact I kind of need to have my anthropic console up to watch my usage... whoops!
That's the functionality which I could use for my day job, but I'm not finding an LLM which directly affords that capability (without programming or other steps which are difficult on my work computer).
It looks to me like a variant of the Code Interpreter pattern, where Claude has a (presumably sandboxed) server-side container environment in which it can run Python. When you ask it to make a spreadsheet it runs this:
And then generates and runs a Python script.What's weird is that when you enable it in https://claude.ai/settings/features it automatically disables the old Analysis tool - which used JavaScript running in your browser. For some reason you can have one of those enabled but not both.
The new feature is being described exclusively as a system for creating files though! I'm trying to figure out if that gets used for code analysis too now, in place of the analysis tool.
A lot of the people I graduated with spent their 20s making powerpoint and excel. There would be people with a master's in engineering getting phone calls at 1am, with an instruction to change the fonts on slide 75, or to slightly modify some calculation. Most of the real decision making was, funnily enough, not based on these documents. But it still meant people were working 100 hour weeks.
I could see this resulting in the same work being done in a few minutes. But I could also see it resulting in the MDs asking for 10x the number of slide decks.
to me it seems miraculous that it even "sort of" works, but also it's not a reliable product yet. OOXML is very complex and the formatting can get mangled.
On the other hand, if you use LaTeX/Beamer slides, LLMs can reliably make a lot of formatting tweaks etc. and it is an actual time saver. But only weird academics use Beamer.
I agree with Simon Willison that this feature is really about writing code in a container, using that capability to edit PPT presentations as if they were markup is an odd thing to make the primary selling point.
Would appreciate if that could be fixed but of course new features are more interesting for them to prioritize.
I've been paying $10/month for GitHub Copilot, which I use via Microsoft's Visual Studio Code, and about a month ago, they added ChatGPT5 (preview), which uses the agent model of interaction. It's a qualitative jump that I'm still learning to appreciate in full.
It seems like the worst possible thing, in terms of security, to let an LLM play with your stuff, but I really didn't understand just how much easier it could be to work with an LLM if it's an agent. Previously I'd end up with a blizzard of python error messages, and just give up on a project, now it fixes it's own mess. What a relief!
I'm on 100$ Max plan, I would even buy 2x 200$ plan if Opus would stop randomly being dumb. Especially after 7am ET time.
Maybe it's because I use the free tier web interface, but I can't get any AI to do much for me. Beyond a handful of lines (and less yesterday) it just doesn't seem that great. Or it gives me pages of javascript to show a date picker before I RTFM and found it's a single input tag to do that, because it's training data was lots of old and/or bad code and didn't do it that way.
I instruct artifacts to not be used and then explicitly provide instruction to proceed with creation when ready.
I used to think of it as a decent sr dev working alongside me. Not it feels like an untrained intern that takes 4-5 shots to get things right. Hallucinated tables, columns, and HTML templates are its new favorite thing. And calling things "done" that aren't even half done and don't work in the slightest.
Also yesterday tried to use it to debug some AWS issue and it tried to send me down so many wrong paths, and suggested changes that were either plain wrong or had unintended consequences, that if I didn't actually know my stuff and had followed blindly, the results would have been pretty bad or at least a huge time waster. When I called it out it would quickly reverse course ("You're right of course!") and it did provide some helpful snippets but I was unimpressed.
What I find it excellent at is for throw-away scripts to do small jobs or automate little things--stuff I could do but would take me a lot longer (especially in bash).
Hope not.
I reverse-engineered it a bit, figured out its container specs, used it to render a PDF join diagram for a SQLite database and then re-ran a much more complex "recreate this chart from this screenshot and XLSX file" example that I previously ran against ChatGPT Code Interpreter last night.
Here's my review: https://simonwillison.net/2025/Sep/9/claude-code-interpreter...