Claude:
I do not actually have a paid version. I am Claude, an AI assistant created by Anthropic to be helpful, harmless, and honest. I do not have different paid tiers or capabilities beyond my base training as an AI model.
They should at least train it with info on what the product actually is, can’t really get information on its current capabilities.
Still annoying that the app is not available globally, just prevent the creation of accounts by checking the location? Moved abroad where Claude is available, but still have an EU App Store account because you can't easily switch countries with existing subscriptions, especially music.
I'm frustrated with Gemini Advanced because of this. It's very good at interacting with calendar and stuff like a digital assistant IF you use text to talk to it. The speech to text is awful (and borderline dangerous).
The exceptional speech to text on the chatgpt iOS app is the main reason I use it. It’s so good. (I mean when you hit the microphone button in the bottom right, not the headphones / back and forth voice chat. Ie STT but no TTS.)
I’m assuming it uses whisper v3 plus gpt for post processing the whisper transcript.
I hope that with iOS 18 siri’s transcription is as good.
ChatGPT’s transcription is good enough that I can trust that even for a long 2 minute input, it’ll be 99-100% right. Whereas Siri I can’t trust that even for a single sentence.
At the risk of being spammy, I wrote a simple Matrix bot that replicates the entirety of their currently announced featureset, but also supports any other model too, that you can access from a normal Matrix client.
Stop re-building chat clients, I already have one!
Ideally they would just run their own chatbot on different existing chat platforms that you could verify your API key with, but with my project you can at least run that chatbot yourself.
Locally-run LLMs are very cool, but I think we have some way to go for smartphone performance to make datacenter-run (or beefy personal workstation for privacy!) ones obsolete.
Until then, I also really like the idea of making chatbots accessible via instant messengers. I find myself using the one in WhatsApp quite a lot these days, just because it's conveniently available on multiple platforms (with my history synchronized) and I can tag it in when chatting with friends. It also works on flights with zero-rated messaging.
Chaz is less mature, probably buggier, more difficult to setup, configured with yaml instead of Environment Variables (why would they do that?), and written in Rust. It also uses AIChat on the backend which isn't necessarily restricted to OpenAI compatible APIs.
I also can't tell if matrix-chatgpt-bot supports sending images, but Chaz does.
Chaz supports switching the model used at any time as well, including in the middle of the chat. So you can setup many separate models and switch between them as necessary for different rooms.
Oo being able to change models sounds great! The environment thing is pretty popular technique for matrix bridges since majority of them are docker based. You usually set environment variables to the docker compose.
That’s interesting, I found Opus to be underwhelming compared to GPT4. Hallucinations occurred as often, if not slightly more, and overall I found Opus to be not as “creative” when writing revisions. That and the rate limit resetting at midnight as compared to GPT4 being something like an hour or two made me return to old faithful.
I got into the habit of running every query I do through all of the SOTA models so I can directly compare the results.
This was originally GPT-4, Claude 3 Opus, and Gemini Advanced. I recently added Meta AI when they launched.
Right now I've sent 486 queries through the first three systems.
The clearest pattern to emerge is that Gemini is terrible, not on par with the other two. There hasn't been a single query that it was the only model who did well. Around 1/4 of the time it gives a clearly inferior answer to the others.
But between GPT-4 and Claude it's less clear. 31 of the 486 queries Claude provided a significantly better answer than the other two but 20 times GPT-4 provided the significantly better answer.
I do think that Claude is a slightly better model but right now it's not a clear enough advantage that I'd recommend it generally. I will say you can probably cancel you Gemini subscription if you're using it though.
One thing I would say about Opus over ChatGPT is that it can return code without placeholders whereas ChatGPT will always have placeholders. It wasn't always that way though and it seems like cutting corners.
Why isn’t the model offered by Google in the same category? Or did I misunderstand their latest naming fiasco and “Gemini Pro” is not actually their latest model?
EDIT: It looks like Gemini Ultra 1.0 > Gemini Pro 1.5 > Gemini Pro 1.0… is that true? What a horrible versioning scheme.
You might want to check out perplexity.AI. It has access to many of these models, and it’s trivial to switch models mid conversation and ask it to repeat itself, so that you can see the differences.
Their web crawler / scraper "ClaudeBot" is the worst semi-legitimate crawler I've ever encountered. Extremely aggressive scraping from a ton of IP addresses with a ton of concurrent accesses, often getting lost in an infinite maze of dynamic url path combinations. Do better, please.
Yes please! Hopefully it's as, or more, accessible to blind users as the ChatGPT app. The website at least is pretty accessible, just the messages aren't read as they come in, but more buttons are labeled than on the ChatGPT site.
And it's not. Messages don't automatically read, the text field is weirdly duplicated, like there's one field that's not able to be opened, and then an unlabeled field that is usable. So I'll stick with ChatGPT. Might even cancel my subscription since all I use is the API, and only through OpenRouter with the OpenAI NVDA addon (screen reader for Windows).
IMO the coolest thing with the iOS app from OpenAI is i can walk around with my headphones in and have a conversation with a realistic AI. Given the amount of funding Claude has, it’s a huge bummer they are lacking this game changing feature.
Edit: top iOS search result is imposter app asking for sub. Be careful
Hi, Anthropic eng here. Thanks for your feedback. You should be able to use your existing account, assuming you're signing in with email, and it should share your existing subscription too. If you're still having issues, you can contact our support at support@anthropic.com
Also we've had a couple reports of people downloading imposter apps instead. Could you verify you're on the official Claude app by following this link? https://apps.apple.com/us/app/claude/id6473753684
72 comments
[ 2.8 ms ] story [ 141 ms ] threadThey should at least train it with info on what the product actually is, can’t really get information on its current capabilities.
Not complaining, just curious.
https://www.anthropic.com/supported-countries
The only "proprietary" (as in you can't run them locally) models that are available in the EU are the models from OpenAI and Mistral.
So only 2/5 major (did I miss anyone) 'AI companies' make their services available in the EU.
My point still stands for Meta and Claude, as I'm talking about the hosted products and APIs. e.g. https://meta.ai and https://claude.ai
I’m assuming it uses whisper v3 plus gpt for post processing the whisper transcript.
I hope that with iOS 18 siri’s transcription is as good.
ChatGPT’s transcription is good enough that I can trust that even for a long 2 minute input, it’ll be 99-100% right. Whereas Siri I can’t trust that even for a single sentence.
Stop re-building chat clients, I already have one!
Ideally they would just run their own chatbot on different existing chat platforms that you could verify your API key with, but with my project you can at least run that chatbot yourself.
[0] - https://github.com/arcuru/chaz
[1] - https://jackson.dev/post/chaz/ (Blog Post)
Locally-run LLMs are very cool, but I think we have some way to go for smartphone performance to make datacenter-run (or beefy personal workstation for privacy!) ones obsolete.
Until then, I also really like the idea of making chatbots accessible via instant messengers. I find myself using the one in WhatsApp quite a lot these days, just because it's conveniently available on multiple platforms (with my history synchronized) and I can tag it in when chatting with friends. It also works on flights with zero-rated messaging.
Disclaimer: I’m a small contributor to that repo
I also can't tell if matrix-chatgpt-bot supports sending images, but Chaz does.
Chaz supports switching the model used at any time as well, including in the middle of the chat. So you can setup many separate models and switch between them as necessary for different rooms.
Interrupting output (or streaming responses in general) won't work in a non-custom chat client, but I don't tend to use that anyways.
Best ones to play with:
Midjourney
Suno AI (music)
Claude
ChatGPT
Perplexity (a better version of “ChatGPT with bing”)
Then some more that others seem to like but I haven’t used / haven’t gotten into: Character AI and Pika Labs
Is this supposed to be higher in the list or is that expected?
Claude’s context window for Opus is remarkable and I’d say that GPT4 is vastly inferior as a result.
The second Anthropic ship Claude with a sandbox that can use python (or equiv) to take care of tasks, I’d happily end my ChatGPT subscription.
GPT4 has been here for over a year and it shows. But until other assistants can manipulate Excel data as easily… I’m stuck paying for it.
Claude is amazing for summarisation, copywriting and reasoned thinking. I love it.
Only downside I’ve found is it doesn’t save chat history for privacy reasons, but you can download them.
https://help.kagi.com/kagi/ai/assistant.html
https://help.kagi.com/kagi/ai/assistant.html#tips-and-tricks...
Maybe I should give it another try.
I also frankly will not extensively use a model that I can't turn off data storage for.
This was originally GPT-4, Claude 3 Opus, and Gemini Advanced. I recently added Meta AI when they launched.
Right now I've sent 486 queries through the first three systems.
The clearest pattern to emerge is that Gemini is terrible, not on par with the other two. There hasn't been a single query that it was the only model who did well. Around 1/4 of the time it gives a clearly inferior answer to the others.
But between GPT-4 and Claude it's less clear. 31 of the 486 queries Claude provided a significantly better answer than the other two but 20 times GPT-4 provided the significantly better answer.
I do think that Claude is a slightly better model but right now it's not a clear enough advantage that I'd recommend it generally. I will say you can probably cancel you Gemini subscription if you're using it though.
EDIT: It looks like Gemini Ultra 1.0 > Gemini Pro 1.5 > Gemini Pro 1.0… is that true? What a horrible versioning scheme.
For everyday non-dev users, Msty for MacOS lets you run split chats and compare each model's answers.
https://msty.app
Edit: top iOS search result is imposter app asking for sub. Be careful