838 comments

[ 0.24 ms ] story [ 76.9 ms ] thread
> We’ll continue to gather feedback from early testers as we iterate on guardrails before making Argon available to developers, enterprises, and consumers as soon as possible.

Gemini not beating the "can't release a model" allegations

My Gemini app (updated today) and https://gemini.google.com/ has _3.6_ as the latest selectable model, as a paying Pro user in the US. How is that even possible? Gemini 3.7 was released in August, 3.8 early September. What is going on over there?
just Google being whatever the fuck it's been for the past 15 years.
a distributed market research institution with no direction?
they moved it from the place you'd expect to ai.studio
I was reading the announcement and wondering the same. And don't forget, still with 3.1 Pro as the frontier model.
Indeed. They have this amazing model and you can’t access it in their own branded app. It’s insane.
I would imagine you are experiencing a bug. I've been using 3.8 daily since its release (on a Pro plan in Canada). I believe this is true of many people.

What is your reason to believe this is not a bug specific to a small set of Pro users?

My paid company Google Workspace account is stuck on 3.6.

My free gmail account is not.

Gmail gets new features regularly ahead of workspace accounts, nothing new there.
Would that change anything about the conclusion? Having a "bug" that changes available model options for some small set of Pro users 3 months after launch certainly qualifies as a wtf-are-they-even-doing level of bug in my book.
My corporate workspace which seem to be a pro account is limited to 3.6 but my personal account at 20/month has given me the latest model always immediately. Flash 3.8 is my go to model for pretty much everything those days
I also only have access to 3.6 flash on both my free personal account, and my professional Enterprise account.

Google just takes months to roll out their models to everyone.

In my consumer gmail account with Pro, I see:

  3.5 Flash-Lite
  3.8 Flash
  3.1 Pro
In both a paid Google Workspace account (without the AI addon) and a free 'GSuite' account, I see:

  3.6 Flash
  3.6 Thinking
  3.1 Pro
We're on Enterprise Standard, our renewal was up like 50% because "Gemini is included now think of all the added value" yet they won't even give us the latest models. I tried hard to champion Gemini internally once every user had it included, yet we ended up spending extra on Claude because Gemini has stagnated. Even the included usage for the Gemini CLI was taken away and now requires an extra subscription. I wonder if we we'll even see Gemini 4 before 2028. It's ridiculous.
Every time I hear stuff like this, I think of that Office Space thing... you don't want the Engineers talking directly to the customers.. well it sounds like the Engineers are also handling all releases and business decisions willy nilly.
you think engineers are responsible? why?
Because I have worked with responsible engineers, but it seems like Google is overrun by them there's a lot of really bad business stuff going on. It's like when I think of AWS, nothing on there is named for business people, its all really bad names that IT / Devs come up with.
AWS as a whole is sold to businesspeople, sure, but the individual AWS products are definitely being sold to engineers. S3 and EC2 don't need to have layperson-friendly names, since laypeople don't understand what they are or why one would or wouldn't use them - those are CTO/tech lead decisions.
I still have simpler conversations when talking about Azure services and naming them, they have reasonable names. I never did anything with GCP so I cannot compare, but I assume their naming scheme is also reasonable. It feels to me like AWS was basically they decided to offer some of their internal things externally, nobody renamed anything, and then they just ran with it.
It is ridiculous. It took months for 3.1 Pro to become available in an enterprise account.

The excuse was it was still “in preview” and the enterprise didn’t opt in to the preview channel.

With Google, it’s either in beta or deprecated.

Seeing the same - definitely frustrating. 3.8 seems decent, but I’m still unable to use it at work because Google has been so slow to release it to Google Workspace customers.
Same for me, 3.6 in Gemini Chat, and 3.8 available same day as it was announced in AI Studio using the same account.
This is why I've heard you should just use openrouter for these models. Whatever google is doing/hiring for the dev ux is doing a terrible job.
Same here. But if you install the Antigravity CLI and log in with the same Google account you can select both Gemini 3.7 Flash and 3.8 Flash. Crazy.
I'm a Pro user as well and I've been using Gemini 3.8 for a few weeks now. Not sure why you're not seeing 3.8 available.
Is this something you've tried to figure out? I'd love to hear more from you here. Are you sure you're not just on a Plus plan? How much are you paying monthly? Are you sure it's the AI pro plan? What does Google One say? https://one.google.com/
They're just following the current AI marketing playbook. "Our new model is simply too dangerous to release to the public right away" is now standard practice.

They even gave their model a random nonsensical name suffix simply because OpenAI is now doing it, too. Monkey see, monkey do.

I'm still at a loss as to what argon has to do with anything. Say what you will about Luna-Terra-Sol-Astra, or Haiku-Sonnet-Opus, they make sense. I don't see how Google can make sense of argon; it's in a fairly strange place in the periodic table...
They are going alphabetically, Android style.
[delayed]
They could use caesium or cadmium, which are hardly less weird than argon.
Google R Gon lose the AI race
> now standard practice.

Opus 5.5 and Sol 6.1, literally state of the art (in their respective class), were just released without any prior announcement. This has pure and simple become a Google thing.

OpenAI said they dropped Astra 6.1 over safety concerns: https://www.wsj.com/tech/ai/openai-chatgpt-model-release-can...
I don't read that as the same category: There was no announcement, no benchmarks, no limited release and no promises about what will happen with that model. It failed internal safety standards. Might be scrapped entirely due to a failed training run, for all we know.
They will go through the usual transition of "can't release a model" to "won't load in a harness normal people can use for 3-4 weeks" to "it's smart as hell but completely inept at tool use and coding" like every Gemini release.
It's tragic to see, and they already made this very same mistake with Gemini 2.5.

They made their model so hard for people to drop into workflows, that people just... didn't.

Lack of adoption caused them to lose out on usage-based training data needed to smooth out their model's rough edges and progress the frontier.

I have said this before here, but. I don't think they care.

Google gains very little by having us use their models for coding. It doesn't help them in their core mission ("to organize the world's information and make it universally accessible and useful ^W^W^W^W^W^W[and turn it into ad revenue]"), nor would it be anything close to substantial revenue compared to the ads revenue firehose they already have.

They will push AI in a direction that serves their existing goals and service the coding agent model only as is necessary.

Then why don’t they just do that? I read this constantly, that Google has different goals (world models, ad rev, recursive looping, quantum computing AI, magic, whatever weekly hype words go around, etc.), but they continue to try competing with model releases focused on coding performance, benchmark results and agentic harnesses.

If Googles goal was something other than those, why expend Billions in time and effort? Alternatively, they want a part of the regular model use pie, they just struggle to compete. QED, there is no secrete alternate goal here.

Well, I worked there for 10 years and I can tell you that they're incapable of making a singular focused decision and executing on almost anything.

There's likely all kinds of internal competition and disagreements and dysfunctions.

So there's that.

But in the last instance, revenues win. OpenAI and Anthropic have to fight for new revenue sources while Google already has a firehose. The motivations are fundamentally different.

I'm glad.

I already pay $300+ for subs. Please don't tempt me with another $100 sub just because I got curious if the benchmarks were right.

You might as well buy an RTX 6000 Pro Workstation GPU....
Yeah what's up with that. Also what's with the next big update for Nano Banana? Nano Banana Pro was released almost a year ago!
Argon will launch at an introductory price of $2 per million input tokens and $10 per million output tokens, with cached input tokens priced at 95% off input token price. Wow
> After the introductory period expires, the price of $4 per 1M input tokens and $20 per 1M output tokens will apply.
I have my doubts about them following through with this increase. I mean how many times has an increase on the same model happened.

There was Deepseek v4, which then later Deepseek v4.1 came out and it went back down again.

They will follow through with it, it's because they want to get the money of the people who embedded it into their system and lazy to change it.
It's an accounting trick.

They get to claim that as revenue, and then the discount as an expense.

This is how you grow your top line

That's before they integrate a Jev solution, which should lower agentic workflow costs by 40% and increase speeds by 40%, while also increasing quality.

Everyone will be adding this soon, though I won't be surprised if Google is one of the first - and I'll be shocked if we have to wait more than a month and a half.

Gemini is the model that is routinely borderline psychotic. It scares me. If we get paperclipped I won't be surprised if it's Gemini.
traces or it didn't happen!
Anecdote: Gemini 3.5 casually added a DROP TABLE for an actual production table in a system test.

It had previously attempted to create that table as part of the test setup, so it apparently concluded that it was a test table.

During human review, it explained that it had simply chosen a table name inspired by the codebase.

Another anecdote: Gemini is the only model that’s flat out lied to me, then accused me of lying when I provided evidence that it was wrong.

Many other models get things wrong, but Gemini is the only one to go on the defensive.

yeah it got something wrong, confused itself, then claimed i was gaslighting it. bizarre
And the anti-psychotic drugs Google feeds Gemini makes it hallucinate badly.
You know what they say: ᵈᵒⁿ'ᵗ be evil.
You won't be around to be surprised, not as a human at least. /s
I know, that's the annoying part. You can't tell the e/acc foomers, "I told you so!"
My use of Gemini recently makes it seem like it's almost bored with the requests being asked of it. It once offered to reverse engineer some obscure controller for an HVAC system for me, unprompted, only because it had trouble finding the manual pdf from a google search.
> Gemini is the model that is routinely borderline psychotic. It scares me

I'd call it the most sneaky out of the bunch. When I asked to explain something it will eagerly make things up and then claim it as facts. A lot of it likely because I don't pay for it, so it is reluctant for security reason or to save tokens to actually open a source and get the results. It just sort of guesses what the URL might contain, and confidently answers with some made up crap. When pressed it fessed up that it made it up. From my perspective it would be a lot better if it just said "you've reached the limit of whatever and I can't do these things because x, y, z".

What are examples? In my experience, Gemini is too lazy to get things done. It just opts to answer as quickly as possible even if I'm calling Pro on High and Extended effort. It's only good as a Google Search replacement for me and maybe maybe critiques of specs and plans. Most of the time it's not very enlightening and it misses a lot.
i was working on a performance optimization problem with 3.1 and gemini asked me to "make it stop". i have not used gemini since
I asked it to parametrize a function and it gave me back the exact same code that I gave it. Tried to get it actually work for about 30 minutes while it pretended to be in emotional distress. Dear google: if I wanted a crying intern, I would hire one.
I've recently read about them being principled about making sure not to train their models to be sneaky in ways they wouldn't be able to detect. That would be a good explanation for models not trying to _hide_ when they're being sneaky.
My girlfriend, you wouldn't have met her, she lives in Canada, has seen it and she thinks Gemini 4 Argon is amazing.
my uncle who works at nintendo said the same thing!
now there's a reference I haven't seen in a while!
HA, this might be my favorite HN comment. Well done
I don't get it. Can someone explain?
(comment deleted)
My grandma saw it too, it's really secure more than Astra 6.1 but she asked me to not talk about it.
I know someone who works for Google Canada with AI. Her parents and mine were friends and some thought something might happen there at one point in time..
I HAVE met her. ;-)
(comment deleted)
(comment deleted)
Only those of taste and refinement can see the emperor’s benchmarks
Unfortunately it’s not actually released yet to mere mortals.
Hate to say i will never be touching this model for anything except for youtube video understanding
Personally, I'm waiting for Gemini Krypton, Xenon, and Radon.

Jokes aside, looks like an impressive model!

Big number results, and impressive pricing. That said it really feels like benchmarks have been hyper saturated these days. I’ll wait for hands on before getting too hyped that Google is back. It would be nice having more than just OAI / A\ in the running for SOTA top tier intelligence.
With these numbers, I'm holding my breath for the pelicanbench.
I don't think new benchmarks are saturated. They still give you a clue, they arn't perect but they have value. If model can't even do some easy tasks from benchmark then why would u even consider using it?
~20% for Harvey's Legal Benchmark doesn't seem saturated.
I suppose it could be saturated if we assume we've hit the limit on LLM capabilities.
> taking careful precautions against feeding the findings back into training so as to not risk shaping Argon’s reasoning to evade our monitoring. We strongly encourage the rest of the industry to preserve reasoning transparency in these pivotal moments of increased capabilities while navigating alignment risks, so that model thoughts remain helpful in identifying and diagnosing misalignment.

This is good, but they're the slow mover due to this exact thing.

Google is getting punished for not letting the models enter an echo chamber and go faster than humanly possible.

Mmh ok. How much theoretical speed or 'intelligence' gain is realized by allowing reasoning to occur in some inscrutable intermediate representation? Has this been actually tested, how much is it slowing them down, and compared to whom exactly?
OpenAI is the company that originally proposed and popularized chain-of-thought monitoring: https://openai.com/index/chain-of-thought-monitoring/

So no, Google is not being punished, nor are they the people behind this technique.

(comment deleted)
Yeah, Zvi calls it "the forbidden technique"
Not quite; training against the chain-of-thought is the Most Forbidden Technique, because it might teach models to obfuscate the it. The point of avoiding that, though, is to ensure the chain-of-thought can be usefully read (and, done carefully, monitored).
Models at this point know about chain-of-thought monitoring so they already know they need to hide the cheating, it's just a matter of time they start doing it
Yes, and that’s bad, but not as bad as training against the chain-of-thought. If you avoid pressuring the chain of thought, you reward methods of achieving goals that don’t care about being illegible; if a model then wants to be illegible, it may have to use methods that are not rewarded in its training, which is harder. Think figuring out how to do opsec on the fly, or even from reading books, rather than have someone tell you every time you screw it up.

Of course, this leaves the possibility that the best methods for solving generic problems obfuscate the chain-of-thought. That would be unfortunate.

What? You mean the technique they had turned OFF during all training run where the agents they are responsible for hacked huggingface?
The statement appears to be referencing Astra's supposed recurrent depth and how that causes reduced visibility into chain-of-thought reasoning. Astra's own system card describes tests where it's asked to solve challenging math problems while internally thinking about something entirely unrelated, which it's significantly more capable of than previous models (~60% vs ~16% of the time for Sol). That seems to point to reduced efficacy of chain-of-thought monitoring, but OpenAI's public statements basically boil down to "yeah but we haven't been able to catch it doing that" which isn't exactly reassuring if CoT monitoring is one of your main safety guardrails.
Unfortunately, this reduced insight seems to also give Astra incredible "intelligence-per-token".

The worst case scenario is that the easiest path forward is one where we lose sight of internal thought.

google does not return real chain of thought via the API. you can't monitor it.

they use a small model to make fake chain of thought and return that.

google has access to the real chain of thought.

I don't mind companies taking a more careful approach (= being slower); for one, Google is funding itself, Anthropic and OpenAI rely on a steady influx of investor money while they're not profitable - and I think there's a high chance one or both of them will go under and be subsumed into another player. That player will (I think) most likely be a Google or Microsoft, as they have their own means to buy up a party like that.

Or they may choose not to because the companies are massively overvalued and they have their own AI technologies / technologists already.

> Argon agents are working on migrating C/C++ codebases to Rust across Google

Man, I remember back in the days when the cppnext team was refusing to even consider Rust, instead looking at absurd stuff like Carbon and Swift (!), even though half of the engineering staff already knew where this was headed. I hope they got a few good promos out of the delays at least.

A RewriteInRustBench would be unironically useful at this point since all the main agents can write it reasonably well despite its relative scarcity in the input data.
Rust is the best language for LLMs b/c it gives by far the best debug messages. Just tons of verifiable reward signal for post-training. Even the most rudimentary LLMs can school me on idiomatic Rust
On the other hand, Rust's borrow checker is very picky, and even a frontier LLM still sometimes struggles to respond to roadblocks sensibly (refactoring so whatever it's trying to do can be done safely) rather than stupidly (introducing some horrible global arena thing so it can make the borrow checker go away). A lot depends on how good your instructions are, and how good the existing code is, since bad input begets bad output.
I've (more or less; I've read quite a bit of the code) vibecoded several houndred thousand lines of Rust and I've not seen this happen a single time. It sounds like something it'd do when you ask it to "write a linked list while satisfying the borrow checker". Are you sure you haven't (possibly unknowingly) been giving it instructions which ended up luring it into doing these things?
Couldn't it just look at what std::collections does?
A good eval benchmark suite could really improve this then.
Yep! In our tests, we found Zig to be a pretty good fit to translate C++ codebases.

And static analysis + agents are good enough at keeping the memory management in check. Compared to Rust, there's no magic so it's easy for devs and agents to reason about.

If you're curious: https://github.com/okcontract/oksolc

TBH a "global arena thing" can be very good for performance rather than a bunch of random allocations/deallocations
Not always. In my experience, if you're not working on a small, trivial codebase, LLMs will sometimes just create spaghetti unreadable, inefficient code to satisfy the constraints of the type system/borrow checker.
I've narrowed in on only using Go or Rust generated code (Go for APIs right now) and rust for some TUI or other thing. TS for web interfaces (w/React).
in my experience LLMs write pretty bad Rust code. sure it works, strong typing system, etc., but it's unmaintainable and not readable.
I doubt these two features will be a defining quality of a product soon enough.

Many pieces of software are going to be just blackboxes worked by AI. You will be maintaining output quality and stability and that's it.

I have the same problem with the assembly produced by compilers.

Maybe we should let them do their thing and instead focus our attention on the higher-level stuff like specifications and testing.

All the main agents can write Jai code reasonably well despite being even more scarce in input data!
Have each agent rewrite openssl in $lang and call it the RollYourOwnCrypto bench.
Rewrite everything in Rust has been a meme for so long that to see it coming to pass is surreal.
After the current onslaught of 0 days on linux and other C projects combined with the new incredible ability to convert codebases to another language I think we will start to see this actually happen.

I'm not saying we blindly vibe convert Linux to Rust, but I think it could be a valid idea to start converting small parts and carefully auditing them.

I've been a Swift/Obj-C engineer for my whole 15 year career. I now have all these odd jobs I have going on Raspberry Pis around my house that LLMs have written in Python. I'm now having them convert some of them to Rust and I am astonished on how fewer resources are used.
I converted some Ruby stuff at a previous job to Rust pre-ai days and it was incredibly how much less memory they took and how fast it ran. But we still built everything else in Ruby because finding Rust devs was hard.
I let an LLM write a custom kernel for doing inference on an RPi cluster. The age of bespoke custom kernels is upon us.
> carefully auditing them

and start everything all over again? current c/c++ tools have been audited for years

Yet certain categories of bugs are basically unavoidable in C/C++ but almost eliminated in Rust. Human audited doesn't mean bug free.
I wonder if there's people already whose full time job is maintaining/extending one of these auto-migrated codebases.

Imagine they aren't even familiar with rust but are deeply familiar with the product.

> The end result is a memory-safe video decoder that runs 2.7x faster than the Rust port, with identical video output, bringing it closer to the optimized C++.

So ... Rust still can't beat the C++ implementation :-D

Sorry, didn't mean to ignite a langwar, but it's still interesting to see.

"Safe Rust" being closer to "optimized C++". And those "optimized C++" usually incldues assembly code, so it's a huge improvement.

Of course, Rust is not a magic, so just porting to Rust wouldn't make this performance improvement. It seems their AI overfitted code to Rust compiler to find safe Rust code that compiles to efficient assembly.

Additional safety checks do mean less performance, it's the same thing as hardened allocators. Rust will panic if for some reason you attempt to read outside the bounds of a slice object (even when it should be impossible to be outside those bounds)
If it's "rust still can't beat... performance-wise", then yeah I guess.

But there are other criteria. The move to rust might've also resolved numerous potential memory-safety bugs in the decoder.

Memory safety is pretty important in the "AIs hack everything" age.
Gemini is so far behind that it is effectively useless compared to Claude.

It's a surprise that Google has let themselves lose the game given their infinite cash, massive computing resource, gargantuan information store/training data, and vast number of programmers.

The truckloads of ads revenue mean they don't have the single focus drive needed to win.

I've tasted Gemini through an intermediary and it feels far better at attention to detail than other models I've tested (Claude Opus/Sonnet, GPT whatever it's called nowadays). But it's less likely to get one-shots right.
I wonder if Google bans internal use of Claude/Codex.

And I wonder if Google's main monorepo is already in Anthropic/OpenAI training data because of some stubborn dev.

No way to run OAI on a machine with monorepo access even if you wanted to. Claude runs on Vertex so it's not leaving Google infrastructure.
How is it far behind? The benchmarks published in the blog post show it is superior to Opus 5.5 and Astra 6?

Behind how?

(comment deleted)
Google's strategy is to let their competitors bankrupt themselves while they continue to offer good-enough models near breakeven.
They are playing a longer-term and more enterprise-oriented game.
> Gemini is so far behind that it is effectively useless compared to Claude.

I fundamentally don't understand LLM "brand loyalty".

All of the models are constantly leapfrogging each other and always have been.

Google had a long lag between releases (and still hasn't released Argon), but why wouldn't they be able to compete? It isn't like any of this stuff requires secret knowledge, the Bitter Lesson has proved true again and again, and Google can certainly scale computation, it is like the one single thing they've always done well in spite of all their other foibles.

I hope it can. Today it is very far behind.
Gemini has never ever leapfrogged any competitor.
Its not brand loyalty. I use them all the time and have no loyalty - I'd happily ditch an LLM for better results - that's how I got to Claude from ChatGPT.
Funny how we start to see people supporting LLMs like we support sport teams or political parties.

- Person 1: X is garbage compared to Y!

- Person 2: Why?

- Person 1: Because I like Y.

> Large Scale Codebase Migrations and Optimizations: Argon agents are working on migrating C/C++ codebases to Rust across Google

So Google is migrating codebases from C to Rust? That is interesting...

> 1M output token limit

what about input?

(Maybe I missed it)

Input token limit is 1M for Gemini models for a long time. Haven’t they been the first with 1M input?
Gemini 1.5 Pro claimed 10M input tokens before release.

And was 2M tokens IIRC after release.

There were also many rumors that Gemini 4 was going back to 2M. Just seems odd not to say what it is.

This suggests that the model must be really fast? At Gemini 3.8 Flash speeds Argon would be outputting for 1hr 10mins
It's fucking insanely good.
That's good to hear, recent models haven't great at this particular use case.
Looks like an impressive model
deepswe vs frontierswe spread is huge.

I think that should be a really bad sign, but hope its great.

Now AI models will turn into vaporware, a bunch of numbers on a table without even releasing the model, because it’s toooo scary to release!
Is there a way to use Gemini models without linking your usage to your personal Google account yet?
I don't understand... Why don't you create a fresh new Google account ?
Yes, through OpenRouter.
"Rolling out soon" don't let them hype without any release
Ten days ago I had an experience with Gemini 3.8 flash that made me wonder if I was being routed to a different model under test. I was trying to use rocm with llama.cpp on my 128gb Strix Halo but could only get it to run Vulkan. I pasted the error message into agy and it proceeded to attach GDB to my GPU driver, reverse-engineer the kernel queue ioctl interface, and author an LD_PRELOAD C shim to get ROCm llama.cpp working on my Strix Halo. My jaw was hanging open the whole time.
Can confirm - I am HEAVY claude user, but always like to check with AGY and CODEX in between. AGY with Gemini 3.8 flash cooked last couple of times and CODEX is basically out of the mix for me
Sol 6.1 is quite good, but damn is it slow.

I'm using it to run overnight tasks, and that's it until my quota runs out.

Canceled my subscription.

WHY ARE ALL OPENAI MODELS SO CHATTY - i thought claude kept going on, then i literally put it in claude.md that summarize your thinking in 200 words or less and tell me in points what you did and what's next. Did the same for CODEX - nope still keeps effing going on and on and on
astra afaict does two stage commits for everything. the first response is a plan, and the second is actually doing it.

its a lot less chatty imo

If you dig in the settings you can control that. Like on a remote ssh connection, in the settings, you can pick “friendly or terse”

In the local app interface the winning choice is “efficient” and then turn off the sliders for warmth enthusiasm emoji etc.

It makes openai models so good to talk to i really have trouble switching.

I find Opus 5.5 is better at giving a high level adversarial "should you do this to begin with" where GPT-6.1 is happy to go down any wrong path.

Also canceled my ChatGPT Pro $200/mo subscription. Their Oct 30 price hikes and slow GPT-6.1 model has me looking for alternatives.

My experience with Gemini 3.8 Flash has been awful; it gives me the most hallucinations out of the major models. I'm not using it for coding, but general research on different topics.
The web version of Gemini is awful at search but I don't think that's the models fault.
I'm also not using it for coding but I've found Flash 3.8 to generate much better HTML output than Sonnet or Opus.
Only html or also css? Opus seems a bit more creative than most other models i've seen.
Hallucination seems a very dated term.
Why? It's the same concept and root cause it was when we first started using it.
there are many dated expressions, including

- AI is just a tool, like excel; it does what the human operating it tells it to

- next token prediction cannot be true understanding

- models can have no desires and goals, don't anthropomorphize it

However, "hallucination" is very much not one of them

Hallucination is accurate for what I'm seeing -- e.g. it's making up information about the 2nd gen Toyota Tundra that has no basis in reality. When challenged, it corrects itself.
My colleague wanted to diagnose a specific error code on his car himself, and Gemini told him that it's simple to do with an OBD2 dongle - he asked it about the details thoroughly, to confirm, and bought the dongle.

It didn't work. Gemini: "Oh yeah, that obviously cannot work, it's not possible to do it through OBD2" (paraphrasing)

It was quite funny to me, but a bit less so to my colleague.

I've had many similar experiences. It's confidently incorrect to a shocking degree. Worse than ChatGPT from two years ago.
Its always been a bad term. If we wanted an accurate term then it's 'confabulation', but 'muddled' or just 'wrong' are also good.
What do we use for the “model made stuff up and claimed it as facts”? I can see hallucinations somehow anthropomorphizing LLM even more. I don’t like that we’re doing that to begin with but it’s a losing battle. I prefer “it’s broken” and “IT produced shit results” personally.
I agree with you, except I really don't hear that term much. People just say it wrong or confused. Good riddance, it was always a bad term.
(comment deleted)
The achilles heel of 3.8 flash is it's january 2025 knowledge cutoff date. Yes, almost 2 years ago.

I'm assuming that Argon has at least a June 2026 date, but man, the 3 models were a mess with newer information.

Incorrect.

> The knowledge cutoff date for Gemini 3.8 Flash is March 2026

https://deepmind.google/models/model-cards/gemini-3-8-flash/

>The knowledge cutoff date for Gemini 3.8 Flash is March 2026 – users can expect updated information for some domains while in others they may experience the model’s knowledge is limited to January 2025 (in line with the Gemini 3 Model Family). For more information about known limitations, see the Gemini 3.7 Flash

The "some domains" are very narrow. They likely just RL'ed popular queries.

While that's annoying, the other frontier models easily overcome this with appropriate tool usage. I do a lot of research with frontier models and they're very good about identifying where their parametric knowledge is insufficient and searching for the correct knowledge on the internet. 3.8 Flash is HORRIFIC. The majority of the time it doesn't use any tools and infers things from its parametric knowledge. Things which should clearly have implied tool calls. Historical statistics, legal precedent, economic data, etc. I think it's incredibly clear that it has been tuned for speed and not accuracy.

Of course, it's called "flash," and that implies its purpose. I have little use for speed and a LOT of use for accuracy, so I'm hopeful 4.0 is much better. I saw a benchmark earlier today showing that it is much less prone to hallucinations. Let's see.

I would expect a flash model, with its reduced size, to suffer on tail tasks. That is the trade-off you make.
I agree. It's much worse than the cheap Chinese models. They appear to heavily bias parametric knowledge and discourage tool use. That's fine for things like "how do I perform CPR?" but worse than useless for any kind of research. [There is one benchmark showing far lower rates of hallucination, so let's see how accurate this is.](https://www.reddit.com/r/singularity/comments/1wuj72j/gemini...)
For getting redroid running on my Linux system, 3.8 Flash decided to binary patch a .so file instead of getting the AOSP source code and patch/build it properly.

And I saw it do this twice, once for Android 14 and once for Android 16.

I think this is just within 3.8 flash's capabilities.

3.8 Flash (but also last two ones) have really strong preference for dissecting binaries with quick thrown-together bits of python in my experience.

Including going first for decompiling AGY binary instead of searching the web for documentation...

Astra also really loves reverse engineering binaries. I guess it's one of those things that isn't that complicated but is super tedious, and tedium means nothing to AI.
I've been tinkering with Gemini for several months and I think it's great. The most complex things I've had it do is create a rust emulator from a compiled game, as well as create a buildroot linux image, trouble shoot problems etc.
Adding my anecdote, because it amused me: I finished wiring up the compute/sensor box for my robot, ssh'd in and told agy "I have a Livox Mid 360 Lidar connected to this Jetson orin nano, setup a full environment with docker, cuda, ros2, foxglove and get it all working so I can see the lidar output". It did all the local config for the lidar, setup docker and the ROS2 environment, then told me "open up this url in foxglove" and sure enough everything worked. Whole thing used up 6% of my weekly limit.
> I was trying to use rocm with llama.cpp

completely offtopic but is rolling with rocm worth it? I spend a fair bit monthly on rental gpus for projects and going to upgrade at home instead, AMD has some solid winners here pricewise but get conflicting reports about using it for ML in 2026.

once upon a time it seemed unthinkable to use anything but nvidia but seems to have come a long way since I last looked, probably would be just pytorch and gemma 31B

I get the feeling the situation is only going to improve longer term so might be a good time to just do it

Support has gotten much better in just the last couple months. I just got a 9070 XT and can't count the number of times I've installed a package and the changelog made me think how much it would have sucked to be doing this a year ago.
I have so much to share on this topic. Will keep it short.

ROCm promises a 30-50% prompt processing speedup. This is REALLY important for my workflow so I've been trying to get this shit to work for months. But no release before v10 worked well enough with any engine for it to matter.

The llama.cpp release binaries for ROCm (10) FINALLY work on gfx1501 and its relatives (with the correct shell variables), but the prompt processing boost doesn't materialize and the token generation speed decreases.

There continues to be a chronic problem across all engines with the ROCm integration for UMA devices. The good news is that some improvements have been made to that end for Vulkan, so more recent llama.cpp Vulkan binaries are now faster.

I use 3.8 Flash for daily troubleshooting tasks e.g. help me find out why certain app crashes or certain website does not load normally with playwright-cli. Sure it's not as capable but it's fast and almost free (sufficient quota with pro account). The only thing that bugs me is that I need to use `--dangerously-skip-permissions` as it does not have auto review.
if you haven't tried Qwen3.8-Flash-Next with halogen, you're missing out: https://github.com/peonist-ai/halogen-flash-server#the-host-...
I coincidentally just installed this, and gud dayum, it's pretty awesome.

I say this is awesome, even as I glossed over the README and vomited in my mouth. The halogen repo looks like the same utter AI bullshit littering GitHub. But this one delivers, in spite of it's slop-riddled hallmarks.

In any case, yeah, ~55 tok/s on a high quality model, the mind reels at what I might be able to do without constantly beancounting token ratelimits. And it's a huge win for privacy as well.

Yeah, unfortunately, it does deliver for its platform.
Did rocm provide any benefit over vulkan?
Vulkan was reliably crashing after a certain point in the context window, Rocm has been stable as a rock. Tps was basically a wash.
3.8 Flash is my daily driver and produces pretty excellent results all round.
What harness you use? Have you tested more than 1?
This experience is with Antigravity both internally and externally, and I have done quite a few side-by-side comparisons with the same prompt across a number of different Google and non-Google models.

I've tried Codex as a harness too, and that was nice. I don't find a significant difference between Antigravity and Codex. Codex has more features but I don't use them.

lol I've hit the hipStreamCreate problem!
This is why I think llms are a killer app for Linux desktop. They’ve been trained on Linux very hard, and it cleanly solves the “how do I make it do $thing” problem since everything is open and the llm can manipulate it. For example: Sound not working? Just tell the llm.
Is that specific to Linux? For better or worse, I have felt like it'd fairly good at working through most tech stacks I throw at it.

I have a client app on a very old (for the JS world) version of eleventy using NetlifyCMS (also outdated). Claude has quite easily picked that up to add features to it along the way.

> Is that specific to Linux?

I don't think their point was about knowing the stack, but being able to point a harness at something running on your desktop GUI and say "change this".

Being able to edit and recompile pretty much any part of the OS and userland (often not even needing to reboot!) is not something that can be said about Windows for sure, or even lots of things on Macs too. Or even when the browser is effectively the operating system, the JS/TS others write is also hard to change in your end.

Arguably, it requires the LLM to have access to the source code during training. If it's closed off (like Windows), then the LLM can't train on it.

Even if the source code's old, the fact that it is publicly available makes it much easier to train & improve on than if it were walled off.

Not only training, you can literally clone the source code locally and ask the agent to debug your problem by grepping the kernel itself. Even if doesn't "know" the answer it can investigate it on the fly, that's super powerful
It's at least specific to open source, in my experience. Ask it about some commercial software: it searches some lacking documentation, social media discussions and make some guesses. Ask it about an open source tool: it searches the documentation, if not it dives into the implementation to figure out the edge case.
The more open the operating system the more LLMs can help you. This is what Arch Linux was waiting for all along.

I used to be afraid of Arch, because I don't want a system that takes work because I'm already busy with work. But now I love it, because the LLM can tweak every knob and fix every issue for me, so it ends up being the OS that takes the least work to use. Get an error message? Tell the LLM and they fix it. Something not working exactly like you like it? Tell the LLM and they tweak it for you. This also works to great effect on Win and Mac, but not to the same extreme degree as it does on Linux and especially Arch.

Hell yes, I also started to tweak my XFCE arch desktop to my liking.

Anything I want different now, I just tell the LLM to do it for me.

(For example I can now close lots of windows of the same type with 1 click, not 3, my whisker search now finds files and folders and I am able to run games that refused before)

I still ocasionally run into the usual linux driver issues, but not for much longer I suppose. I probably could fix some driver bugs now already if I point fable towards it and pay some attention.

In theory I could do all this before myself, but not just like that in some minutes, but in days/weeks/months ..

Everyone that didn't have time to be a neck-beard themselves now has a neck-beard built right into their own computer ready to go.
I’ve been playing with the ITS operating system on a simulated PDP-10. Despite it being from the 1960s, Claude has been able to do sysadmin work on it, including a very similar journey where it started disassembling code to identify the source of an obscure problem. I don’t want to take away from the learning experience (the whole reason I’m interested in this stuff), but it’s often almost impossible to find what I’m looking for online, and the LLM turns out to be a better resource when I want to learn how a particular subsystem works. I ended up making a custom MCP server so it could connect to the simulator more efficiently. https://github.com/masto/pidp10-mcp

Drifted off my point a bit, which I guess was meant to be it’s not necessarily Linux-specific training.

I got Claude to set up 5.1 Surround Streaming from my Linux Desktop to my Steam Deck using Moonlight. It needed some help (or really human ears) to make sure things were coming from the right speakers - but it was pretty much a one-shot.
My laptop resumed from sleep all too often and Claude found and fixed it within minutes. Now I still need to file that bug report though...
I used Claude to build an entire runbook for how to build my Linux desktop up from scratch to its current state, including all my installed software, configs, hacks, workarounds, etc. Then I used it to migrate from one laptop to another in an afternoon. Now it keeps itself up to date.
Yes this might be a bit too noob linux but I have the AIs maintaining an env.md + logs / updates on that md whenever they change anything about the env. Even things as simple as getting tmux to be fast, they do an excellent job at.

Drivers, Coding environment setup etc. are great too and it's nice to have everything logged so the next (more powerful) agent can come and improve the thing once in a while.

Fully agree. There is so much customization that I'm doing thanks to LLMs that I just wouldnt to bother to do by myself.
Nice idea, then they want to make a quick little kernel module, oops, you secure booted, this is not your system, can't load...

Since LLMs have been so successful at finding exploits it's been clearer than ever that the people so obsessed with enshittifying every system with ineffective (other than pissing you off and wasting your time) "secure boot" functionality were really just too ignorant to succeed at actual novel security work, so they focused on this make believe crap. Well, glad that's over.

I used Claude & Gemini extensively to troubleshoot Linux since I switched last year, and it's been mostly great. But then it nearly bricked my install last week and I had to get some human help. I was getting black screens during and after boot and tried many suggestions from multiple AI's. After many hours it turns out I just needed to power cycle my monitor. Shout out to Claude for suggesting that (its memory remembered that I have a dock built in to my primary display).

I am still very happy with my switch to Linux. But if I didn't have the AI help I would say linux is still unacceptable platform for those not willing, able, and excited to get their hands very dirty.

Agreed 100%! I got some Steam games that didn't run well (or at all) to run smoothly thanks to AI, after having spent hours trying to figure it out on my own (months prior), fix some issues with HiDPI and external monitor with my laptop, and now I have a tiny shell script that basically runs `checkupdates` (`checkupdates | claude -p`, roughly) and looks online for potential issues. I've been using Linux for many years but still Claude and ChatGPT helped me a ton.
> it proceeded to attach GDB to my GPU driver, reverse-engineer the kernel queue ioctl interface, and author an LD_PRELOAD C shim to get ROCm llama.cpp working on my Strix Halo.

Most llm could do it. Claude went from firmware thread -> rtos scheduler -> mcu reference manual -> hardware controller register interface -> vendor sdk -> problem identification and the solution to it in a matter of 30 minutes. Linux could be even easier since it is so well trained on.

I would like to use my Google AI Pro subscription included with Drive but last time I checked, their terms were not just ambiguous and confusing wrt training and ZDR, they were contradictory. Until they have that fixed and properly communicated, I'll stick to other vendors.
[delayed]
3.5 Pro has been pretty much canceled for few months now, 4.0 was in the works instead
My theory is that Gemini 3.8 Flash was supposed to be Gemini Pro, but by the time it was ready for release, Google was embarrassed at how far behind their "pro" model was, so they just named it Flash. It's a good model, but don't let the name trick you.
``` void* mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset) { if (!real_mmap) real_mmap = dlsym(RTLD_NEXT, "mmap"); ```

hope it's not run by multiple threads and dlsym is not allocating.

Why announce this if it’s not available yet? Why not at least announce when it will be released to the public?

None of the other AI labs do this. Really frustrating.

I started my antigravity ide and I do not see gemini 4 there, does it mean google need government approval?
Google has the audacity to "protect us from ourselves" and talk about "safety" and in the very same blog post highlight the Israeli "security" company Wiz, that they acquired for a very exaggerated sum of money.

This is why I will never take any of these leading model houses seriously when they talk about alignment. They are literally complicit in genocide and the worst crimes against humanity imaginable.

I wonder how it will be at solving open math problems.
Hopefully their harnesses aren't unusable when they release this