The Gemini 2 models support native audio and image generation but the latter won't be generally available till January. Really excited for that as well as 4o's image generation (whenever that comes out). Steerability has lagged behind aesthetics in image generation for a while now and it's be great to see a big advance in that.
Also a whole lot of computer vision tasks (via LLMs) could be unlocked with this. Think Inpainting, Style Transfer, Text Editing in the wild, Segmentation, Edge detection etc
Maybe some of these tasks are arguably not aligned with the traditional applications of CV, but Segmentation and Edge detection are definitely computer vision in every definition I've come across - before and after NNs took over.
I asked Gemini 2.0 Flash (with my voice) whether it natively understands audio or is converting my voice to text. It replied:
"That's an insightful question. My understanding of your speech involves a pipeline first. Your voice is converted to text and then I process the text to understand what you're saying. So I don't understand your voice directly but rather through a text representation of it."
Unsure if this is a hallucination, but is disappointing if true.
Edit: Looking at the video you linked, they say "native audio output", so I assume this means the input isn't native? :(
We’re definitely going to need better benchmarks for agentic tasks, and not just code reasoning. Things that are needlessly painful that humans go through all the time
Beats Gemini 1.5 Pro at all but two of the listed benchmarks. Google DeepMind is starting to get their bearings in the LLM era. These are the minds behind AlphaGo/Zero/Fold. They control their own hardware destiny with TPUs. Bullish.
No, and they haven't been for at least half a year. Utterly optimized for by the providers. Nowadays if a model would be SotA for general use but not #1 on any of these benchmarks, I doubt they'd even release it.
I've started keeping an eye out for original brainteasers, just for that reason. GCHQ's Christmas puzzle just came out [1], and o1-pro got 6 out of 7 of them right. It took about 20 minutes in total.
I wasn't going to bother trying those because I was pretty sure it wouldn't get any of them, but decided to give it an easy one (#4) and was impressed at the CoT.
Meanwhile, Google's newest 2.0 Flash model went 0 for 7.
I didn't see a straightforward way to submit the final problem, because I used different contexts for each of the 7 subproblems.
Given the right prompt, though, I'm sure it could handle the 'find the corresponding letter from the landmarks to form an anagram' part. That's easier than most of the other problems.
You're saying the ultimate answer isn't 'PROTECTING THE UNITED KINGDOM'?
Regarding TPU’s, sure for the stuff that’s running on the cloud.
However their on device TPUs lag behind the competition and Google still seem to struggle to move significant parts of Gemini to run on device as a result.
Of course, Gemini is provided as a subscription service as well so perhaps they’re not incentivized to move things locally.
I am curious if they’ll introduce something like Apple’s private cloud compute.
i don’t think they need to win the on device market.
we need to separate inference and training - the real winners are those who have the training compute. you can always have other companies help with inference
At what point does the on device stuff eat into their market share though? As on device gets better, who will pay for cloud compute? Other than enterprise use.
I’m not saying on device will ever truly compete at quality, but I believe it’ll be good enough that most people don’t care to pay for cloud services.
That makes no sense. Inference cost dwarf training cost if you have a succesfull product pretty quickly. Afaik there is no commodity hardware that can run state of the art models like chatgpt-o1.
> Afaik there is no commodity hardware that can run state of the art models like chatgpt-o1.
Stack enough GPUs and any of them can run o1. Building a chip to infer LLMs is much easier than building a training chip.
Just because one cost dwarfs another does not mean that this is where the most marginal value from developing a better chip will be, especially if other people are just doing it for you. Google gets a good model, inference providers will be begging to be able to run it on their platform, or to just sell google their chips - and as I said, inference chips are much easier.
Each GPU costs ~50k. You need at least 8 of them to run mid-sized models. Then you need a server to plug those GPUs into. That's not commodity hardware.
more like ~$16k for 16 3090s. AMD chips can also run these models. The parts are expensive but there is a competitive market in processors that can do LLM inference. Less so in training.
I don't know where did you get that price from but 1x RTX 3090 is $1,900. 16x is ~$30,000.
> The parts are expensive
Now that we invested ~$30k in GPUs, we only need to find a motherboard that can accommodate 16x pcie4 x16 GPUs, right? And we also need a CPU that can drive that many pcie4 x16 lanes?
Well, none of them exist, not even in the server parts sector let alone client commodity hardware. In any case, you'd need two CPUs so even with this imaginary motherboard we are already entering the server rack design space. And that costs 100's of thousands of $$$.
> but there is a competitive market in processors that can do LLM inference
Nothing but the smallest and smallish models. If that existed then why would you set yourself out building a 16x RTX 3090 machine?
Chip level is only a tiny part of the story. Training can happen with a big boy variant of "it works on my machine". Inference require a world wide network of GPUs. Chip level is the last thing you will be worrying about.
> i don’t think they need to win the on device market.
The second Apple comes out with strong on-device AI - and it very much looks like they will - Google will have to respond on Android. They can't just sit and pray that e.g. Samsung makes a competitive chip for this purpose.
I think Apple is uniquely disadvantaged in the AI race to a point people dont realize.
They have less training data to use, having famously been focused on privacy for its users and thus having no particular advantage in this space due to not having customer data to train on.
They have little to no cloud business, and while they operate a couple of services for their users, they do not have the infrastructure scale to compete with hyperscaler cloud vendors such as Google and Microsoft.
Most of what they would need to spend on training new models would require that they hand over lots of money to the very companies that already have their own models, supercharging their competition.
While there is a chance that Apple might come out with a very sophisticate on-device model. The problem here is that they would only be able to compete with other on-device models. The magnitude of compute needed to keep pace with SOA models is not achievable on a single device. It will take many generations of Apple silicon in order to compete with the compute of existing datacenters.
Google also already has competitive silicon in this space with the Tensor series processors, which are being fabbed at Samsung plants today. There is no sitting and praying necessary on their part as they already compete.
Apple is a very distant competitor in the space of AI, and I see no reason to assume this will change, they are uniquely disadvantaged by several of the choices they made on their way to mobile supremacy.
The only thing they currently have going for them is the development of their own ARM silicon which may give them the ability to compete with Google's TPU chips, but there is far more needed to be competitive here than the ability to avoid the Nvidia tax.
yeah i’ve never understood the outsized optimism for apple’s ai strategy, especially on hn.
they’re a little bit less of a nobody than they used to be, but they’re basically a nobody when it comes to frontier research/scaling. and the best model matters way more than on-device which can always just be distilled later and find some random startup/chipco to do inference
Theory: Apple's lifestyle branding is quite important to the identity of many in the community here. I mean, look at the buy-in at launch for Apple Vision Pro by so many people on HN--it made actual Apple communities and publications look like jaded skeptics.
Oh please, this is the classic “everyone who chooses differently than myself is <superficial/dumb/misinformed>” argument that a lot of people use when it comes to tech nerd identity politics.
Is it really that hard to imagine people have different viewpoints, and decisions than yourself without being painted as vapid, airheads?
"having famously been focused on privacy for its users and thus having no particular advantage in this space due to not having customer data to train on"
That may not be as big a disadvantage as you think.
Anthropic claim that they did not use any data from their users when they trained Claude 3.5 Sonnet.
sure but they certainly acquired data from mass scraping (including of data produced by their users) and/or data brokering aka paying someone to do the same.
It is likely Apple can get additional data by creating synthetic data for user interactions.
About 7 years ago I trained GAN models to generate synthetic data, and it worked so well. The state of the art has increased a lot in 7 years, so Apple will be fine.
For a while there I would have been in agreeance with you, but the thought that models can be trained purely on synthetic data has shown to be wrong on multiple levels.
Synthetic data needs to be reviewed by individuals to ensure data quality, significantly reducing the speed at which an organization can adopt training data.
Reasonable engineers would suggest that the answer to this is to have other language models review the synthetic data, but we have seen that this is what leads to model collapse due to compounding issues around hallucinations.
At best Synthetic data is a "slow follow" for training a model due to the need for human review, but a competitive model, it does not make.
For clarity, I was only talking about the hardware side, not the software one. I don't think the models matter too much, by the time the hardware is ready there will be open models that Apple can take and modify to their liking.
Besides, did Anthropic and e.g. Mistral inherently have such troves of data to train on that Apple doesn't? For the last 6 months, Anthropic has had the SOTA model for the average production usecase.
> Google also already has competitive silicon in this space with the Tensor series processors, which are being fabbed at Samsung plants today. There is no sitting and praying necessary on their part as they already compete.
Intel had a much bigger advantage with x86, and look where we are now. I find it hard to believe that creating a good AI chip isn't a much smaller challenge than it was to do Apple Silicon. The upcoming SE uses their in-house 5G modem, another huge hardware achievement that no one else has been able to do.
With that in mind, how can you bet against Apple when it comes to designing chips at this point? It's not like Amazon et al aren't producing their own AI chips too. Let alone all of the startups like Cerebras. That indicates the moat and barriers are likely much lower than Apple Slicion or the 5G modem.
There’s an easy solution here: Apple isn’t trying to compete with the big models everyone else is running. They’re betting in the opposite direction that many small models is a better value ad for their customers. And they can call out to other services as needed for the larger stuff.
I’m in the camp that this is the right call for consumers, instead of trying to compete on the large model side. They’ve yet to deliver on their full promise, but if they can, it’s the place where I think more of the industry will go (for consumers)
And regarding Google’s mobile tensor chips, they are infamously behind all other players in the market space for the same generation of processor. They don’t share the same advantages they do in the server space.
I don’t think the AI market will ever really be a healthy one until inference vastly outnumbers training. What does it say about AI if training is done more than inference?
I agree that the in-device inference market is not important yet.
Your first sentence has the fallacy that you’re attributing the cost of the device to a single feature against the cost of that single feature.
Most people are unlikely to buy the device for the AI features alone. It’s a value add to the device they’d buy anyway.
So you need the paid for option to be significantly better than the free one that comes with the device.
Your second sentence assumes the local one is dumb. What happens when local ones get better? Again how much better is the cloud one to compete on cost?
To your last sentence, it assumes data fetching from the cloud. Which is valid but a lot of data is local too. Are people really going to pay for what Google search is giving them for free?
I think it's a more likely assumption that on device performance will trail off device models by a significant margin for at least the next few years - of course if magically you can make it work locally with the same level of performance it would be better.
Plus a lot of the "agentic" stuff is interaction with the outside world, connectivity is a must regardless.
My point is that you do NOT need the same level of performance. You need an adequate level of performance that the cost to get more performance isn’t worth it to most people.
And my point is that it's way too early to try to optimize for running locally, if performance really stabilizes and comes to a halt (which may likely happen) then it makes more sense to optimize.
Plus once you start with on device features you start limiting your development speed and flexibility.
Yes, but I can't imagine situations where I "have" to run a model when I don't have internet at that time. My life would be more affected with the rest of the internet than having to run a small stupid model locally. At the very least until the hallucination is completely solved, as I need internet to verify the models.
You’re assuming the model is purely for generation though. Several of the Gemini features are lookup of things across data available to it. A lot of that data can be local to device.
That is currently Apple’s path with Apple Intelligence for example.
Poor reception is rapidly becoming a non-issue for most of the developed world. I can’t think of the last time I had poor reception (in America) and wasn’t on an airplane.
As the global human population increasingly urbanizes, it’ll become increasingly easy to blanket it with cell towers. Poor(er) regions of the world will increase reception more slowly, but they’re also more likely to have devices that don’t support on-device models.
Also, Gemini Flash is basically positioned as a free model, (nearly) free API, free in GUI, free in Search Results, Free in a variety of Google products, etc. No one will be paying for it.
Many major cities have significant dead spots for coverage. It’s not just for developing areas.
Flash is free for api use at a low rate limit. Gemini as a whole is not free to Android users (free right now with subscription costs beyond a time period for advanced features) and isn’t free to Google without some monetary incentive. Hence why I also originally ask about private cloud compute alternatives with Google.
Definitely not when it comes to LLM's, the larger more useful local models are not that fast and latency is not an issue, just look at this Google models voice function or even openai's advanced voice.
And yet these new models still haven’t reached feature parity with Google Assistant, which can turn my flashlight on, but with all the power of burning down a rainforest, Gemini still cannot interact with my actual phone.
Yeah they've been slow to release end-user facing stuff but it's obvious that they're just grinding away internally.
They've ceded the fast mover advantage, but with a massive installed base of Android devices, a team of experts who basically created the entire field, a huge hardware presence (that THEY own), massive legal expertise, existing content deals, and a suite of vertically integrated services, I feel like the game is theirs to lose at this point.
The only caution is regulation / anti-trust action, but with a Trump administration that seems far less likely.
Gemini in search is answering so many of my search questions wrong.
If I ask natural language yes/no questions, Gemini sometimes tells me outright lies with confidence.
It also presents information as authoritative - locations, science facts, corporate ownership, geography - even when it's pure hallucination.
Right at the top of Google search.
edit:
I can't find the most obnoxious offending queries, but here was one I performed today: "how many islands does georgia have?".
Compare that with "how many islands does georgia have? Skidaway Island".
This is an extremely mild case, but I've seen some wildly wrong results, where Google has claimed companies were founded in the wrong states, that towns were located in the wrong states, etc.
At first, this was true but now it has gotten pretty good. The times it gets things wrong are often not the models fault and just google searches fault.
Yes, LMArena shows Gemini-2.0-Flash-Exp ranking 3rd right now, after Gemini-Exp-1206 and ChatGPT-4o-latest_(2024-11-20), and ahead of o1-preview and o1-mini:
Big companies can be slow to pivot, and Google has been famously bad at getting people aligned and driving in one direction.
But, once they do get moving in the right direction the can achieve things that smaller companies can't. Google has an insane amount of talent in this space, and seems to be getting the right results from that now.
Remains to be seen how well they will be able to productize and market, but hard to deny that their LLM models aren't really, really good though.
>> hard to deny that their LLM models aren't really, really good though.
The context window of Gemini 1.5 pro is incredibly large and it retains the memory of things in the middle of the window well. It is quite a game changer for RAG applications.
Bear in mind that a "1 million token" context window isn't actually that. You're being sold a sparse attention model, which is guaranteed to drop critical context. Google TPUs aren't running inference on a TERABYTE of fp8 query-key inputs, let alone TWO of fp16.
BERT and Gemma 2B were both some of the highest-performing edge models of their time. Google does really well - in terms of pushing efficiency in the community they're second to none. They also don't need to rely on inordinate amounts of compute because Google's differentiating factor is the products they own and how they integrate it. OpenAI is API-minded, Google is laser-focused on the big-picture experience.
For example; those little AI-generated YouTube summaries that have been rolling out are wonderful. They don't require heavyweight LLMs to generate, and can create pretty effective summaries using nothing but a transcript. It's not only more useful than the other AI "features" I interact with regularly, it doesn't demand AGI or chain-of-thought.
I disagree - another way you could phrase this is that Google is presbyopic. They're very capable of thinking long-term (eg. Google Deepmind and AI as a whole, cloud, video, Drive/GSuite, etc.), but as a result they struggle to respond to quick market changes. AdSense is the perfect example of Google "going long" on a product and reaping the rewards to monopolistic ends. They can corner a market when the set their sights on it.
I don't think Google (or really any of FAANG) makes "good" products anymore. But I do think there are things to appreciate in each org, and compared to the way Apple and Microsoft are flailing helplessly I think Google has proven themselves in software here.
Stadia was a failure from the start. Microsoft and Nvidia are also laser-focused on this game streaming business, but I seriously doubt it will pan out ever. At least not in a profitable sense. In that regard, I think Google planned to be first-to-market, failed early, and killed their darling before anyone got a chance to love it.
The IMs post-Hangouts are less explainable, but I do empathize with Google for wanting to find some form of SMS replacement standard. The RCS we have today is flawed and was rushed out of the door just to have a serious option for the DOJ to endorse. This is an area where I believe the United States government has been negligent in allowing competing OEMs to refuse cooperation in creating an SMS successor. I agree it's silly, and it needs to stop eventually.
> Stadia was a failure from the start. Microsoft and Nvidia are also laser-focused on this game streaming business
Have you actually compared these services first hand? Stadia was miles ahead of the competition. The experience was unbelievably good and ubiquitous (Desktop, phone, TV, Chromecast...), and both mouse and gamepad felt like first class input methods.
Microsoft's Xbox game streaming is a complete joke in comparison. Last time I tried, I had to use my mouse to operate a virtual gamepad to operate a virtual cursor to click instruments in MSFS. Four levels of nesting. Development progress is also extremely slow. Not sure where you're seeing laser focus there.
> I do empathize with Google for wanting to find some form of SMS replacement standard
Why did Google out of all companies have to come up with an SMS replacement? Absolutely nobody asked for this! They started out with XMPP, which was federated and had world-class open source implementations, and after what feels like a double-digit number of failed attempts they arrived at SMS over SIP from hell that nobody other than themselves actually knows how to implement and only telcos can federate with (theoretically; practically, they just outsource to Google).
I find it really hard to believe that this is anything other than a thinly veiled marketing plot to be able to point at an "open standard" that Google is almost exclusively running via Jibe (not sure if they provide that for free or are charging carriers for it).
The contortions they went through to decouple their "Allo" and "Duo" brands from Google accounts (something almost everybody has anyway to send email!) for absolutely no benefit and even more significant customer confusion...
And now look at Gemini. It looks like the exact same story to me from the beginning: Amazing technology backed by a great team (they literally invented transformers), yet completely kneecapped by completely confused product development. It's unreal how much better it is queried through the API, but that's unfortunately not what people see when they go to gemini.google.com.
They have to not get blind sided by Sora, while at the same time fighting the cloud war against MS/Amazon.
Weirdly Google is THE AI play. If AI is not set to change everything and truly is a hype cycle, then Google stock withstands and grows. If AI is the real deal, then Google still withstands due to how much bigger the pie will get.
I actually wanted to know, real person proof
user: AH4oFVbPT4f8
created: August 12, 2013
I'm going back and forth between the different models seeing which works best for me but I'm trying to learn how to read and use other people's feedback in making their decisions.
Sora is a toy. No understanding of physics. Fairly expensive. Hard to tell stories with it. I work in video production. The industry is not big enough for you to invest billions and billions into. It’s currently in a total state of crisis.
Video production is just not big enough of a market to make a difference in the AI race. I don’t understand why any AI company would spend significant amount of resources matching Sora when I don’t really think it will be a 10 billion dollar product (yet).
Plus Google is well positioned to match it anyway, since they have YouTube data they can probably license to their AI gen video training.
Yet, google continues to show it'll deprecate it's APIs, Services, and Functionality at the detriment of your own business. I'm not sure enterprises will trust Google's LLM over the alternatives. Too many have been burned throughout the years, including GCP customers.
I read parent comment "grew 35% last quarter" as (income on 2024-09-30) is 1.35 * (income on 2024-07-01)
The balance sheet shows (income on days from 2024-07-01 through 09-30) is 1.35 * (income on days from 2023-07-01 through 09-30)
These are different because with heavily handwavey math the first is growing 35% in a single quarter and the second is growing 35% annually (by comparing like-for-like quarters)
In the financial industry you almost always compare over 12 months (year over year growth) to avoid noise like seasonality. It's so prevalent I didn't think I need to explain it but yeah I meant GCP grew 35% YOY.
It's indeed the typical interpretation of what I've said. I could have written YOY growth but it's so common that that's what everyone means people many times omit it.
Not sure why you're comparing both AWS + Azure to GCP - did they become one company ?
My point is GCP is growing very fast, AWS while being much bigger has slowed considerably to around 20% growth rate while GCP is at 35%. It has very good growth and many people seem to be adequately happy with it, in contrast to what many commenters here seem to think.
AWS is 2.5x the size of gcp. In fact AWS is growing faster than GCP based off of gross revenue added over last 12 months.
I used both azure and AWS to show that GCP has lost significant markshare because of its deprecation policy. Enterprises don’t trust GCP won’t deprecate their services.
> Remains to be seen how well they will be able to productize and market
The challenge is trust.
Google is one of the leaders in AI and are home to incredibly talented developers. But they also have an incredibly bad track record of supporting their products.
It's hard to justify committing developers and money to a product when there's a good chance you'll just have to pivot again once they get bored. Say what you will about Microsoft, but at least I can rely on their obsession with supporting outdated products.
> they also have an incredibly bad track record of supporting their products
Incredibly bad track record of supporting products that don't grow. I'm not saying this to defend Google, I'm still (perhaps unreasonably) angry because of Reader, it's just that there is a pattern and AI isn't likely to fit that for a long while.
I’m sad for reader but it was a somewhat niche product. Inbox I can’t forgive. It was insanely good and was killed because it was a threat to Gmail.
My main issue with Google is that internal politic affects users all the time. See the debacle of anything built on top of Android and being treated as a second citizen.
You can’t trust a company which can’t shield users from its internal politics. It means nothing is aligned correctly for users to be taken seriously.
> And that is a problem for a company with 20k employees?
It is for a company where the promotion culture rewards new initiatives and products and doesn't reward people maintaining products. Which was most certainly the company culture around the time reader was killed.
Yeah, either AI is significant, in which case Google isn't going to kill it. Or AI is a bubble, in any of the alternatives one might pick can easily crash and die long before Google ends of life anything.
This isn't some minor consumer play, like a random tablet or Stadia. Anyone who has paying attention would have noticed that AI has been an important, consistent, long term strategic interest of Google's for a very long time. They've been killing off the fail/minor products to invest in this.
The question is seldom "why" they kill it (I'd argue ultimately it doesn't matter), it's about how fast and what they offer as a migration path for those who boarded the train.
That also means the minute Gemini stops looking like a growing product it's gone from this world, where Microsoft backed alternatives have a fighting chance to get some leeway to recover or pivot.
Yeah, MS Azure DevOps is still alive, though stagnant. I thought everyone would be moved to GitHub in few years after MS acquired GitHub. Yet here we are, 6 years later.
I can guarantee that DevOps will still be around and functional in 2030. It won't have new features but it will still be supported.
Microsoft bought FoxBase in 1992. FoxPro never took the world by storm but it had a dedicated group of devs and ISVs who used it and it solved their needs. The last version was released in 2004, long after Microsoft had released.Net and C# and SQL Server. Microsoft officially ended support for it in 2015.
Google? If the product doesn't become an instant #1 or #2 in its market and doesn't directly contribute to their bottom line in a way which can be itemised in their earnings call, it's gone in less than 3 years guaranteed.
You are not wrong, but my irrational mind is unlikely to take your advice. Try treating it as a meme, rather than anything belonging to a sane discussion. For me, I don't think I'll ever get over it.
No we can't cause it was absolutely a turning point in Google's trajectory.
After Reader, it was Currents, Google TV, Picasa, Google Now, Spaces, Chromecast Audio,Inbox, GCM, Nest, Fusion Tables, Google Cloud Print, Google Play Music, Google Bookmarks, Chrome Apps, G Suite....
Reader keeps coming up because after Reader, Google's motto turned into "Do be Evil"
Yes. Imagine Google banning your entire Google account / Gmail because you violated their gray area AI terms ([1] or [2]). Or, one of your users did via an app you made using an API key and their models.
With that being said, I am extremely bullish on Google AI for a long time. I imagine they land at being the best and cheapest for the foreseeable future.
For me that is a reason for not touching anything from Google for building stuff. I can afford lossing my Amazon account, but Google's one would be too much. At least they should be clear in their terms that getting banned at cloud doesn't mean getting banned from Gmail/Docs/Photos...
That won't help. Their TOS and policies are vague enough that they can terminate all accounts you own (under "Use of multiple accounts for abuse" for instance).
Even if it is warranted on their part, the 1% false positive will be detrimental to those affected. And we all know there is no way to reach out to them in case the account is automatically flagged.
My buddy lost his Gmail account because of a heart attack followed by a string of events that google ‘AI’ considered too risky to allow the account to live.
If their fraud AI is wrong there is now human to talk to.
I used to be "thepimp@hotmail.com" in the early days of Hotmail, of course I was also a 6th grader (true story). One day they unceremoniously closed my account without any possibility to recover mails.
That day I learned an important lesson: pimpin' ain't easy.
I asked Gemini about banning risks, and it answered:
Gemini: Yes, there is a potential risk of your Google account being suspended if your SaaS is used to process inappropriate content, even if you use Gemini to reject the request. While Gemini can help you filter and identify harmful content, it's not a foolproof solution.
Here are some additional measures you can take to protect your account:
* Content moderation: Implement a robust content moderation system to filter out inappropriate content before it reaches Gemini. This can include keyword-based filtering, machine learning models, and human review.
...
* Regularly review usage: Monitor your usage of Gemini to identify any suspicious activity.
* Follow Google's terms of service: Make sure that your use of Gemini complies with Google's terms of service.
By taking these steps, you can minimize the risk of your account being suspended and ensure that your SaaS is used responsibly.
---
In a follow up question I asked about how to implement robust content moderation and it suggested humans reviewing each message...
So a convenient blah blah blah about all the nice things you can doto avoid Google's brainless algorithmic wrath, but which may simply not work anyhow because even by following all rules in good faith, you still get banned one day, as has happened to many, many people with zero recourse.
As an attacker, instead of DDoSing a service we could just upload a bunch of NSFW text so Google kills their infra for us.
Other providers, like OpenAI, at least provide a free moderation API. Google has a moderation API, that after the free 50k requests it is more expensive than Gemini 1.5 flash (Moderation API costs $0.0005/100 characters vs Gemini 1.5 flash $0.000001875/100 characters).
>Say what you will about Microsoft, but at least I can rely on their obsession with supporting outdated products.
Eh... I don't know about that. Their tech graveyard isn't as populous as Google's, but it's hardly empty. A few that come to mind: ATL, MFC, Silverlight, UWP.
Besides Silverlight (which was supported all the way until the end of 2021!), you can still not only run but _write new applications_ using all of the listed technologies.
That doesn't constitute support when it comes to development platforms. They've not received any updates in years or decades. What they've done is simply not remove the capability build capability from the toolchains. That is, not even the work that would be required to no longer support them in any way. Compare that to C#, which has evolved rapidly over the same time period.
Only because they operate different businesses. Google is primarily a service provider. They have few software products that are not designed to integrate with their servers. Many of Microsoft's businesses work fundamentally differently. There's nothing Microsoft could do to Windows to disable all MFC applications and only MFC applications, and if there was it would involve more work than simply not doing anything else with MFC.
I can write something with Microsoft tech and expect it with reasonable likelihood to work in 10 years (even their service-based stuff), but can't say the same about anything from Google.
That alone stops me/my org buying stuff from Google.
I'm not contending that Microsoft and Google are equivalent in this regard, I'm saying that Microsoft does have a history of releasing technologies and then letting them stagnate.
Not only has MFC been recently updated to support HiDPI, it is still the best Microsoft GUI C++ development experience on Visual C++.
And even if C++/CX and C++/WinRT aren't that great, with worse tooling than MFC and in maintenance mode, you can easilly create an application with them today.
Hardly the same can be told of most Google technologies.
You're absolutely right. I wasn't thinking about it when I wrote the comment, but I really should have included it. I'm still pissed about that, and I don't like how Core releases are deprecated in 1-2 years. For my personal projects that's a breakneck pace.
> Google is one of the leaders in AI and are home to incredibly talented developers. But they also have an incredibly bad track record of supporting their products.
This is why we've stayed with Anthropic. Every single person I work with on my current project is sore at Google for discontinuing one product or another - and not a single one of them mentioned Reader.
We do run some non-customer facing assets in Google Cloud. But the website and API are on AWS.
Just to drive the point home. I made the parent comment five days ago. Today I searched Google for "google-cloud-skd vs google-cloud-cli", the top result was this page from two years ago:
> CLI is the new name for the SDK.The reasoning and strategy was explained in great detail in this podcast:
> https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5mZWVkYnVybmVyLmNvbS9HY3BQb2RjYXN0/episode/NTI5ZTM5ODAtYjYzOC00ODQxLWI3NDAtODJiMTQyMDMxNThj?ep=14
So I click that link, and I'm greeted with:
> Google Podcasts is no longer available
> Listen to podcasts and build your library in the YouTube Music app.
This is why AWS and Anthropic are getting our money. We can not trust that Google projects will survive as long as our business needs.
Yea the only stories I ever see are ones that bubble up to HN. Often they are very one-sided as well. Not saying it hasn't happened, but let's not pretend it's rampant.
> But they also have an incredibly bad track record of supporting their products.
I don't know about that: my wife built her first SME on Google Workspace / GSuite / Google Apps for domain (this thing changed names so many times I lost track). She's now running her second company on Google tools, again.
All she needs is a browser. At one point I switched her from Windows to OS X. Then from OS X to Ubuntu.
Now I just installed Debian GNU/Linux on her desktop: she fires up a browser and opens up Google's GMail / GSuite / spreadsheets and does everything from there.
She's a happy paying customer of Google products since a great many years and there's actually phone support for paying customers.
I honestly don't have many bad things to say. It works fine. 2FA is top notch.
It's a much better experience than being stuck in the Windows "Updating... 35%" "here's an ad on your taskbar" "you're computer is now slow for no reason" world.
I don't think they'l pull the plug on GSuite: it's powering millions and millions of paying SMEs around the world.
> and Google has been famously bad at getting people aligned and driving in one direction.
To be fair, it's not that they're bad at it -- it's that they generally have an explicit philosophy against it. It's a choice.
Google management doesn't want to "pick winners". It prefers to let multiple products (like messaging apps, famously) compete and let the market decide. According to this way of thinking, you come out ahead in the long run because you increase your chances of having the winning product.
Gemini is a great example of when they do choose to focus on a single strategy, however. Cloud was another great example.
I definitely agree that multiple competing products is a deliberate choice, but it was foolish to pursue it for so long in a space like messaging apps that has network effects.
> hard to deny that their LLM models aren't really, really good though
I'm so scarred by how much their first Gemini releases sucked that the thought of trying it again doesn't even cross my mind.
Are you telling us you're buying this press release wholesale, or you've tried the tech they're talking about and love it, or you have some additional knowledge not immediately evident here? Because it's not clear from your comment where you are getting that their LLM models are really good.
> but hard to deny that their LLM models aren't really, really good though
Although I do still pay for ChatGPT, I find it dog slow. ChatGPT is simply way too slow to generate answers. It feels like --even though of course it's not doing the same thing-- I'm back to the 80s with my 8-bit computer printing thing line by line.
Gemini OTOH doesn't feel like that: answers are super fast.
To me low latency is going to be the killer feature. People won't keep paying for models that are dog slow to answer.
I'll probably be cancelling my ChatGPT subscription soon.
Maybe I've just hit a streak of good outputs, but I've also been noticing the automatic gemini search when doing google searches to have been significantly more useful than it was previously.
About a year ago, I was saying that Google was potentially walking toward its own grave due to not having any pivots that rivaled OpenAI. Now, I'm starting to think they've found the first few steps toward an incredible stride.
It's easier for consultants and sales people to sell to enterprise if the terminology is familiar but mysterious.
Bad
1. installed Antivirus software
2. added screen-size CSS rules
3. copied 'Assets' harddrive to DropBox
4. edited homepage to include Bitcoin wallet address link
5. upgraded to ChatGPT Pro
The beauty of LLMs isn’t just these coding objects speak human vernacular but they can be concatenated with human vernacular prompts and that itself can be used as an input, command or output sensibly without necessarily causing error even if a series of inputs combinations weren't preprogrammed.
I have an A.I. textbook that has agent terminology that was written preLLm days. agents are just autonomous ish code that loops on itself with some extra functionality. LLMs in their elegance can more easily out the box selfloop just on the basis concatenating language prompts, sensibly. They are almost agent ready out the box by this very elegant quality(the textbook agentic diagram is just a conceptual self perpetuation loop), except…
Except they fail at a lot or get stuck at hiccups. But, here is a novel thought. What if an LLM becomes more agentic (ie more able to sustain autonomous chain prompts that do actions without a terminal failure) and less copilotee not by more complex controlling wrapper self perpetuation code, but by means of training the core llm itself to more fluidly function in agentic scenarios.
a better agentically performing llm that isnt mislabeled with a bad buzzword might not reveal itself in its wrapper control code but through it just performing better in an typical agentic loop or environment conditions with whatever initiating prompt, control wrapper code, or pipeline that initiates its self perpetuation cycle.
Aside from sounding "dumb as shit", I also think it's the completely wrong abstraction. Every time I've split up work between multiple specialized LLM "agents", they've done an WAY worse job than one monolith LLM that had access to all the tools it needs and can "reason" about the whole task.
All the common tricks, like creating a list of steps that are then executed by specialized agents in order, for example, fall flat as soon as one agent returns a result that contradicts the initial steps. It's simply a bandaid for short context sizes and LLMs that can't remain focused past the first few thousand tokens of prompt.
I've been using Gemini Flash for free through the API using Cline for VS Code. I switch between Claude and Gemini Flash, using Claude for more complicated tasks. Hope that the 2.0 model comes closer to Claude for coding.
Agreed - tried some sample prompts on our data and the rough vibe check is that flash is now as good as the old pro. If they keep pricing the same, this would be really promising.
OT: I’m not entirely sure why, but "agentic" sets my teeth on edge. I don't mind the concept, but the word itself has that hollow, buzzwordy flavor I associate with overblown LinkedIn jargon, particularly as it is not actually in the dictionary...unlike perfectly serviceable entries such as "versatile", "multifaceted" or "autonomous"
That's not necessarily a good thing because they are overloaded while novel jargon is specific.
We use new words so often that we take it for granted. You've passively picked up dozens of new words over the last 5 or 10 years without questioning them.
I'm personally very glad that the word has adhered itself to a bunch of AI stuff, because people had started talking about "living more agentically" which I found much more aggravating. Now if anyone states that out loud you immediately picture them walking into doors and misunderstanding simple questions, so it will hopefully die out.
To play devil's advocate, the correct use of the word would be when multiple AIs are coordinating and handing off tasks to each other with limited context, such that the handoffs are dynamically decided at runtime by the AI, not by any routine code. I have yet to see a single example where this is required. Most problems can be solved with static workflows and simple rule based code. As such, I do believe that >95% of the usage of the word is marketing nonsense.
I actually have built such a tool (two AIs, each with different capabilities), but still cringe at calling at agentic. Might just be an instinctive reflex.
I think this sort of usage is already happening, but perhaps in the internal details or uninteresting parts, such as content moderation. Most good LLM products are in fact using many LLM calls under the hood, and I would expect that results from one are influencing which others get used.
You nailed an interesting nuance there about agents needing to make their own decisions!
I'm getting fairly excited about "agentic" solutions to the point that I even went out of my way to build "AgentOfCode" (https://github.com/JasonSteving99/agent-of-code) to automate solving Advent of Code puzzles by iteratively debugging executions of generated unit tests (intentionally not competing on the global leaderboard).
And even for this, there's actually only a SINGLE place in the whole "agent" where the models themselves actually make a "decision" on what step to take next, and that's simply deciding whether to refactor the generated unit tests or the generated solution based on the given error message from a prior failure.
This is what other replies are missing - I've been following AI closely since GPT 2 and it's not immediately clear what agentic means, so to other people, the term must be even less clear. Using the word autonomous can't be worse than agentic imo.
We need something to describe a behavioral element in business processes. Something goes into it, something comes out of it - though in this case nondeterminism is involved and it may not be concrete outputs so much as further actioning.
No, we need a scientific understanding of autonomous intelligent decision-making. The problem with “agentic AI” is the same old “Artificial Intelligence, Natural Stupidity” problem: we have no clue what “reasoning” or “intelligence” or “autonomous” actually means in animals, and trying to apply these terms to AI without understanding them (or inventing a new term without nailing down the underlying concept) is doomed to fail.
They don’t mean anything precise, and don’t need to. They describe a bag of behaviors with overlapping properties that we observe animals/people doing.
‘Intelligent’ is exactly as precise as ‘funny’ or ‘interesting’. It’s a label for a cluster of observations of another agent’s behavior. It entails almost nothing about how those behaviors are achieved.
This is of course only an opinion, but it’s my professional opinion after thirty five years in the AI business.
"GVP stands for Good Pharmacovigilance Practice, which is a set of guidelines for monitoring the safety of drugs. SVP stands for Senior Vice President, which is a role in a company that focuses on a specific area of operations."
Seems lot of pharma regulation in my telecom company.
Anecdotally, using the Gemini App with "Gemini Advanced 2.0 Flash Experimental", the response quality is ignorantly improved and faster at some basic Python and C# generation.
Considering so many of us would like more vRAM than NVIDIA is giving us for home compute, is there any future where these Trillium TPUs become commodity hardware?
Power concerns aside, individual chips in a TPU pod don't actually have a ton of vRAM; they rely on fast interconnects between a lot of chips to aggregate vRAM and then rely on pipeline / tensor parallelism. It doesn't make sense to try to sell the hardware -- it's operationally expensive. By keeping it in house Google only has to support the OS/hardware in their datacenter and they can and do commercialize through hosted services.
Why do you want the hardware vs just using it in the cloud? If you're training huge models you probably don't also keep all your data on prem, but on GCS or S3 right? It'd be more efficient to use training resources close to your data. I guess inference on huge models? Still isn't just using a hosted API simpler / what everyone is doing now?
Also FYI for anyone else, I can't actually see the gemini-2.0-flash-exp list in the model directory for Vertex AI in GCP Console, but passing in google/gemini-2.0-flash-exp to the vertexai Python SDK does actually work.
(you'll probably also need to increase your quotas right away, the default is only 10 requests per minute).
With the accelerating progress, the "be ready to look again" is becoming a full time job that we need to be able to delegate in some way, and I haven't found anything better than benchmarks, leaderboards and reviews.
FWIW I've found the 'coding' 'category' of the leaderboard to be reasonably accurate. Claude was the best, o1-mini then was typically stronger, now the Gemini Exp 1206 is at the top.
I find myself just paying a la carte via the API rather than paying the $20/mo so I can switch between the models.
poe.com has a decent model where you buy credits and spend them talking to any LLM which makes it nice to swap between them even during the same conversation instead of paying for multiple subscriptions.
Though gpt-4o could say "David Mayer" on poe.com but not on chat.openai.com which makes me wonder if they sometimes cheat and sneak in different models.
Leaderboards are not that useful for measuring real-life effectiveness of the models atleast in my day-today usage.
I am currently struggling to diagnose an ipv6 mis-configuration in my enormous aws cloudformation yaml code. I gave the same input to Claude Opus, Gemini and ChatGPT ( o1 and 4o).
4o was the worst. verbose and waste of my time.
Claude completely went off-tangent and began recommending fixes for ipv4 while I specifically asked for ipv6 issues
o1 made a suggestion which I tried out and it fixed it. It literally found a needle in the haystack. The solution is working well now.
Gemini made a suggestion which almost got it right but it was not a full solution.
I must clarify diagnosing network issues on AWS VPC is not my expertise and I use the LLMs to supplement my knowledge.
Yes. I find it a bit funny how much people care about leaderboards. I see models going up and down, winning this or that benchmark and yet, for me, Sonnet 3.5 still beats the crap out of all of them.
I too was puzzled by the response from Claude.
I am using the Anthropic workbench with claude-3-5-sonnet-20241022 (latest)
But it think it has to do more with the freshness of training data.
AWS IPV6 Egress is a new technology from AWS which was introduced only recently. Previously, we had to deploy NAT gateway which supported IPV4. I am assuming claude-3-5-sonnet-20241022 (latest) was not trained on this data.
I like that https://artificialanalysis.ai/leaderboards/models describes both quality and speed (tokens/s and first chunk s). Not sure how accurate it is; anyone know? Speed and variance of it in particular seems difficult to pin down because providers obviously vary it with load to control their costs.
True that - and I think Gemini-Exp-1206 is Gemini 2.0 Pro in testing. I noticed how they only replaced the "experimental" moniker for one of their experimental models, and it turned into 2.0 Flash. And that still experimental model is currently leading across all categories.
Claude MCP does the same thing. It’s the setup that is hard. It will do push pull create branch automatically from a single prompt. 500$ a month for Devin could be worth it if you want it taken care off plus use the models for a team, but a single person can set it up
The Web is dead. It’s pretty clear future web pages, if we call them that, will be assembled on-the-fly by AI based your user profile and declared goals, interests and requests.
I think people just have rose-tinted glasses on. Sure the hardware from Nokia was great, but software was very poor even by the standards of that time.
Windows Phone was actually great though, and would've eventually been a major player in the space if Microsoft were stubborn enough to stick with it long enough, like they did with the Xbox.
By his own admission, Gates was extremely distracted at the time by the antitrust cases in Europe, and he let the initiative die.
511 comments
[ 5.5 ms ] story [ 344 ms ] threadAlso a whole lot of computer vision tasks (via LLMs) could be unlocked with this. Think Inpainting, Style Transfer, Text Editing in the wild, Segmentation, Edge detection etc
They have a demo: https://www.youtube.com/watch?v=7RqFLp0TqV0
"That's an insightful question. My understanding of your speech involves a pipeline first. Your voice is converted to text and then I process the text to understand what you're saying. So I don't understand your voice directly but rather through a text representation of it."
Unsure if this is a hallucination, but is disappointing if true.
Edit: Looking at the video you linked, they say "native audio output", so I assume this means the input isn't native? :(
If you're using Gemini in aistudio(not sure about the real-time API but everything else) then it has native audio input
I wasn't going to bother trying those because I was pretty sure it wouldn't get any of them, but decided to give it an easy one (#4) and was impressed at the CoT.
Meanwhile, Google's newest 2.0 Flash model went 0 for 7.
1: https://metro.co.uk/2024/12/11/gchq-christmas-puzzle-2024-re...
Given the right prompt, though, I'm sure it could handle the 'find the corresponding letter from the landmarks to form an anagram' part. That's easier than most of the other problems.
You're saying the ultimate answer isn't 'PROTECTING THE UNITED KINGDOM'?
What do you mean? Is o1 not a single model?
There will probably be a 2.0 pro (which will be 4o/sonnet class) and maybe an ultra (o1(?)/Opus).
If they want a rematch, they'll need to bring their 'A' game next time, because o1-pro is crazy good.
However their on device TPUs lag behind the competition and Google still seem to struggle to move significant parts of Gemini to run on device as a result.
Of course, Gemini is provided as a subscription service as well so perhaps they’re not incentivized to move things locally.
I am curious if they’ll introduce something like Apple’s private cloud compute.
we need to separate inference and training - the real winners are those who have the training compute. you can always have other companies help with inference
I’m not saying on device will ever truly compete at quality, but I believe it’ll be good enough that most people don’t care to pay for cloud services.
inference basically does not matter, it is a commodity
training doesn’t matter if inference costs are high and people don’t pay for them
Training is amortized over each inference, so the cost of inference also needs to include the cost of training to break even unless made up elsewhere
Stack enough GPUs and any of them can run o1. Building a chip to infer LLMs is much easier than building a training chip.
Just because one cost dwarfs another does not mean that this is where the most marginal value from developing a better chip will be, especially if other people are just doing it for you. Google gets a good model, inference providers will be begging to be able to run it on their platform, or to just sell google their chips - and as I said, inference chips are much easier.
I don't know where did you get that price from but 1x RTX 3090 is $1,900. 16x is ~$30,000.
> The parts are expensive
Now that we invested ~$30k in GPUs, we only need to find a motherboard that can accommodate 16x pcie4 x16 GPUs, right? And we also need a CPU that can drive that many pcie4 x16 lanes?
Well, none of them exist, not even in the server parts sector let alone client commodity hardware. In any case, you'd need two CPUs so even with this imaginary motherboard we are already entering the server rack design space. And that costs 100's of thousands of $$$.
> but there is a competitive market in processors that can do LLM inference
Nothing but the smallest and smallish models. If that existed then why would you set yourself out building a 16x RTX 3090 machine?
Sorry, but you're just spitting out non-sense.
The second Apple comes out with strong on-device AI - and it very much looks like they will - Google will have to respond on Android. They can't just sit and pray that e.g. Samsung makes a competitive chip for this purpose.
Economically this fits the cloud much better.
If anything, I think the upcoming iOS AI update will bring them to a similar level as android/google.
While there is a chance that Apple might come out with a very sophisticate on-device model. The problem here is that they would only be able to compete with other on-device models. The magnitude of compute needed to keep pace with SOA models is not achievable on a single device. It will take many generations of Apple silicon in order to compete with the compute of existing datacenters.
Google also already has competitive silicon in this space with the Tensor series processors, which are being fabbed at Samsung plants today. There is no sitting and praying necessary on their part as they already compete.
Apple is a very distant competitor in the space of AI, and I see no reason to assume this will change, they are uniquely disadvantaged by several of the choices they made on their way to mobile supremacy. The only thing they currently have going for them is the development of their own ARM silicon which may give them the ability to compete with Google's TPU chips, but there is far more needed to be competitive here than the ability to avoid the Nvidia tax.
they’re a little bit less of a nobody than they used to be, but they’re basically a nobody when it comes to frontier research/scaling. and the best model matters way more than on-device which can always just be distilled later and find some random startup/chipco to do inference
Is it really that hard to imagine people have different viewpoints, and decisions than yourself without being painted as vapid, airheads?
The level of optimism for Apple AI capabilities on here is wrong. I can imagine people having wrong viewpoints, but it is wrong.
That may not be as big a disadvantage as you think.
Anthropic claim that they did not use any data from their users when they trained Claude 3.5 Sonnet.
About 7 years ago I trained GAN models to generate synthetic data, and it worked so well. The state of the art has increased a lot in 7 years, so Apple will be fine.
At best Synthetic data is a "slow follow" for training a model due to the need for human review, but a competitive model, it does not make.
Besides, did Anthropic and e.g. Mistral inherently have such troves of data to train on that Apple doesn't? For the last 6 months, Anthropic has had the SOTA model for the average production usecase.
> Google also already has competitive silicon in this space with the Tensor series processors, which are being fabbed at Samsung plants today. There is no sitting and praying necessary on their part as they already compete.
Intel had a much bigger advantage with x86, and look where we are now. I find it hard to believe that creating a good AI chip isn't a much smaller challenge than it was to do Apple Silicon. The upcoming SE uses their in-house 5G modem, another huge hardware achievement that no one else has been able to do.
With that in mind, how can you bet against Apple when it comes to designing chips at this point? It's not like Amazon et al aren't producing their own AI chips too. Let alone all of the startups like Cerebras. That indicates the moat and barriers are likely much lower than Apple Slicion or the 5G modem.
If I'm talking nonsense, do correct me.
I’m in the camp that this is the right call for consumers, instead of trying to compete on the large model side. They’ve yet to deliver on their full promise, but if they can, it’s the place where I think more of the industry will go (for consumers)
And regarding Google’s mobile tensor chips, they are infamously behind all other players in the market space for the same generation of processor. They don’t share the same advantages they do in the server space.
their models aren’t even that good. sorry apple fanboys but the talent isn’t there
Apple just isn’t very capable in this space, not sure what’s so hard to accept
I agree that the in-device inference market is not important yet.
inference hardware is a commodity in a way that training is not
And sure, poor reception will be an issue, but most people would still absolutely take a helpful remote assistant over a dumb local assistant.
And you don't exactly see people complaining that they can't run Google/YouTube/etc locally.
Most people are unlikely to buy the device for the AI features alone. It’s a value add to the device they’d buy anyway.
So you need the paid for option to be significantly better than the free one that comes with the device.
Your second sentence assumes the local one is dumb. What happens when local ones get better? Again how much better is the cloud one to compete on cost?
To your last sentence, it assumes data fetching from the cloud. Which is valid but a lot of data is local too. Are people really going to pay for what Google search is giving them for free?
Plus a lot of the "agentic" stuff is interaction with the outside world, connectivity is a must regardless.
Plus once you start with on device features you start limiting your development speed and flexibility.
That is currently Apple’s path with Apple Intelligence for example.
It has no world model. It doesn't know truth any more than it knows bullshit just a statistical relationship between words.
As the global human population increasingly urbanizes, it’ll become increasingly easy to blanket it with cell towers. Poor(er) regions of the world will increase reception more slowly, but they’re also more likely to have devices that don’t support on-device models.
Also, Gemini Flash is basically positioned as a free model, (nearly) free API, free in GUI, free in Search Results, Free in a variety of Google products, etc. No one will be paying for it.
Flash is free for api use at a low rate limit. Gemini as a whole is not free to Android users (free right now with subscription costs beyond a time period for advanced features) and isn’t free to Google without some monetary incentive. Hence why I also originally ask about private cloud compute alternatives with Google.
I see poor reception in both areas and only one has WiFi.
Pretty sure that's not doing any fancy on-device models!
That said, there was a popup today saying that assistant is now using Gemini, so I just enabled it to try. Could well have changed in the last week.
They've ceded the fast mover advantage, but with a massive installed base of Android devices, a team of experts who basically created the entire field, a huge hardware presence (that THEY own), massive legal expertise, existing content deals, and a suite of vertically integrated services, I feel like the game is theirs to lose at this point.
The only caution is regulation / anti-trust action, but with a Trump administration that seems far less likely.
If I ask natural language yes/no questions, Gemini sometimes tells me outright lies with confidence.
It also presents information as authoritative - locations, science facts, corporate ownership, geography - even when it's pure hallucination.
Right at the top of Google search.
edit:
I can't find the most obnoxious offending queries, but here was one I performed today: "how many islands does georgia have?".
Compare that with "how many islands does georgia have? Skidaway Island".
This is an extremely mild case, but I've seen some wildly wrong results, where Google has claimed companies were founded in the wrong states, that towns were located in the wrong states, etc.
> A depsipeptide is a cyclic peptide where one or more amide groups are replaced by ester groups.
Depsipeptides are not necessarily cyclic, and I'd probably use "bond" instead of "group".
https://imgur.com/a/YslvJO2
These errors are happening all the time.
I also miss that it’s not yet really as context aware as ChatGPTo4. Even just asking a follow-up question, confuses Gemini 1.5.
Hope Gemini 2.0 will improve that!
In my own testing "exp 1206" is significantly better than Gemini 2.
Feels like haiku 3.5 vs sonnet 3.5 kind of thing.
https://lmarena.ai/?leaderboard
EDIT: probably not deep research.. is it Google testing their equivalent of o1? who knows..
But, once they do get moving in the right direction the can achieve things that smaller companies can't. Google has an insane amount of talent in this space, and seems to be getting the right results from that now.
Remains to be seen how well they will be able to productize and market, but hard to deny that their LLM models aren't really, really good though.
The context window of Gemini 1.5 pro is incredibly large and it retains the memory of things in the middle of the window well. It is quite a game changer for RAG applications.
Google's marketing wins again, I guess.
For example; those little AI-generated YouTube summaries that have been rolling out are wonderful. They don't require heavyweight LLMs to generate, and can create pretty effective summaries using nothing but a transcript. It's not only more useful than the other AI "features" I interact with regularly, it doesn't demand AGI or chain-of-thought.
This doesn't match my experience of any Google product.
I don't think Google (or really any of FAANG) makes "good" products anymore. But I do think there are things to appreciate in each org, and compared to the way Apple and Microsoft are flailing helplessly I think Google has proven themselves in software here.
Or how would you describe their handling of Stadia, or their weird obsession about shipping and cancelling about a dozen instant messengers?
The IMs post-Hangouts are less explainable, but I do empathize with Google for wanting to find some form of SMS replacement standard. The RCS we have today is flawed and was rushed out of the door just to have a serious option for the DOJ to endorse. This is an area where I believe the United States government has been negligent in allowing competing OEMs to refuse cooperation in creating an SMS successor. I agree it's silly, and it needs to stop eventually.
Have you actually compared these services first hand? Stadia was miles ahead of the competition. The experience was unbelievably good and ubiquitous (Desktop, phone, TV, Chromecast...), and both mouse and gamepad felt like first class input methods.
Microsoft's Xbox game streaming is a complete joke in comparison. Last time I tried, I had to use my mouse to operate a virtual gamepad to operate a virtual cursor to click instruments in MSFS. Four levels of nesting. Development progress is also extremely slow. Not sure where you're seeing laser focus there.
> I do empathize with Google for wanting to find some form of SMS replacement standard
Why did Google out of all companies have to come up with an SMS replacement? Absolutely nobody asked for this! They started out with XMPP, which was federated and had world-class open source implementations, and after what feels like a double-digit number of failed attempts they arrived at SMS over SIP from hell that nobody other than themselves actually knows how to implement and only telcos can federate with (theoretically; practically, they just outsource to Google).
I find it really hard to believe that this is anything other than a thinly veiled marketing plot to be able to point at an "open standard" that Google is almost exclusively running via Jibe (not sure if they provide that for free or are charging carriers for it).
The contortions they went through to decouple their "Allo" and "Duo" brands from Google accounts (something almost everybody has anyway to send email!) for absolutely no benefit and even more significant customer confusion...
And now look at Gemini. It looks like the exact same story to me from the beginning: Amazing technology backed by a great team (they literally invented transformers), yet completely kneecapped by completely confused product development. It's unreal how much better it is queried through the API, but that's unfortunately not what people see when they go to gemini.google.com.
They haven't wielded this advantage as powerfully as possible, but changes here could signal how committed they are to slaying the search cash cow.
Nadella deservedly earned acclaim for transitioning Microsoft from the Windows era to cloud and mobile.
It will be far more impressive if Google can defy the odds and conquer the innovator's dilemma with search.
Regardless, congratulations to Google on an amazing release and pushing the frontiers of innovation.
Weirdly Google is THE AI play. If AI is not set to change everything and truly is a hype cycle, then Google stock withstands and grows. If AI is the real deal, then Google still withstands due to how much bigger the pie will get.
I'm going back and forth between the different models seeing which works best for me but I'm trying to learn how to read and use other people's feedback in making their decisions.
Video production is just not big enough of a market to make a difference in the AI race. I don’t understand why any AI company would spend significant amount of resources matching Sora when I don’t really think it will be a 10 billion dollar product (yet).
Plus Google is well positioned to match it anyway, since they have YouTube data they can probably license to their AI gen video training.
You mean by shifting away from Windows for mobile and focusing on iOS and Android?
The fact GCP needs to have this page, and these lists are not 100% comprehensive is telling enough. https://cloud.google.com/compute/docs/deprecations https://cloud.google.com/chronicle/docs/deprecations https://developers.google.com/maps/deprecations
Steve Yegge rightfully called this out, and yet no change has been made. https://medium.com/@steve.yegge/dear-google-cloud-your-depre...
Google Cloud grew 35% year over year, when comparing the 3 months ending September 30th 2024 with 2023.
https://abc.xyz/assets/94/93/52071fba4229a93331939f9bc31c/go... page 12
The balance sheet shows (income on days from 2024-07-01 through 09-30) is 1.35 * (income on days from 2023-07-01 through 09-30)
These are different because with heavily handwavey math the first is growing 35% in a single quarter and the second is growing 35% annually (by comparing like-for-like quarters)
I used both azure and AWS to show that GCP has lost significant markshare because of its deprecation policy. Enterprises don’t trust GCP won’t deprecate their services.
Some guy had to do it for Azure, then he went to work for them and it is now deprecated itself
https://blog.tomkerkhove.be/2023/03/29/sunsetting-azure-depr...
The challenge is trust.
Google is one of the leaders in AI and are home to incredibly talented developers. But they also have an incredibly bad track record of supporting their products.
It's hard to justify committing developers and money to a product when there's a good chance you'll just have to pivot again once they get bored. Say what you will about Microsoft, but at least I can rely on their obsession with supporting outdated products.
Incredibly bad track record of supporting products that don't grow. I'm not saying this to defend Google, I'm still (perhaps unreasonably) angry because of Reader, it's just that there is a pattern and AI isn't likely to fit that for a long while.
My main issue with Google is that internal politic affects users all the time. See the debacle of anything built on top of Android and being treated as a second citizen.
You can’t trust a company which can’t shield users from its internal politics. It means nothing is aligned correctly for users to be taken seriously.
No, Reader was killed because it:
- was free
- didn't contribute to revenue growth from ads
It is for a company where the promotion culture rewards new initiatives and products and doesn't reward people maintaining products. Which was most certainly the company culture around the time reader was killed.
Bus factor et al. is literally CS 101.
This isn't some minor consumer play, like a random tablet or Stadia. Anyone who has paying attention would have noticed that AI has been an important, consistent, long term strategic interest of Google's for a very long time. They've been killing off the fail/minor products to invest in this.
I think we all acknowledge this.
The question is seldom "why" they kill it (I'd argue ultimately it doesn't matter), it's about how fast and what they offer as a migration path for those who boarded the train.
That also means the minute Gemini stops looking like a growing product it's gone from this world, where Microsoft backed alternatives have a fighting chance to get some leeway to recover or pivot.
Microsoft bought FoxBase in 1992. FoxPro never took the world by storm but it had a dedicated group of devs and ISVs who used it and it solved their needs. The last version was released in 2004, long after Microsoft had released.Net and C# and SQL Server. Microsoft officially ended support for it in 2015.
Google? If the product doesn't become an instant #1 or #2 in its market and doesn't directly contribute to their bottom line in a way which can be itemised in their earnings call, it's gone in less than 3 years guaranteed.
Like how many different instant messengers did they make at the same time only to abandon them all instead of just making one and supporting it?
That's irrelevant to me as a user if I've already invested my time into the product.
To an extent all companies do it. Google just does it much more, to a degree that I tend to ignore most Google's launches because of this uncertainty.
I'm sorry.
After Reader, it was Currents, Google TV, Picasa, Google Now, Spaces, Chromecast Audio,Inbox, GCM, Nest, Fusion Tables, Google Cloud Print, Google Play Music, Google Bookmarks, Chrome Apps, G Suite....
Reader keeps coming up because after Reader, Google's motto turned into "Do be Evil"
Have you seen Google Bard anywhere recently? Me neither :)
Now I'm not sure if are you arguing that a name change is not supporting a product or that Gemini is a different product with a different feature set?
With that being said, I am extremely bullish on Google AI for a long time. I imagine they land at being the best and cheapest for the foreseeable future.
[1] https://policies.google.com/terms/generative-ai
[2] https://policies.google.com/terms/generative-ai/use-policy
https://www.xda-developers.com/google-developer-account-ban-...
If their fraud AI is wrong there is now human to talk to.
That day I learned an important lesson: pimpin' ain't easy.
Gemini: Yes, there is a potential risk of your Google account being suspended if your SaaS is used to process inappropriate content, even if you use Gemini to reject the request. While Gemini can help you filter and identify harmful content, it's not a foolproof solution.
Here are some additional measures you can take to protect your account:
* Content moderation: Implement a robust content moderation system to filter out inappropriate content before it reaches Gemini. This can include keyword-based filtering, machine learning models, and human review.
...
* Regularly review usage: Monitor your usage of Gemini to identify any suspicious activity.
* Follow Google's terms of service: Make sure that your use of Gemini complies with Google's terms of service.
By taking these steps, you can minimize the risk of your account being suspended and ensure that your SaaS is used responsibly.
---
In a follow up question I asked about how to implement robust content moderation and it suggested humans reviewing each message...
As an attacker, instead of DDoSing a service we could just upload a bunch of NSFW text so Google kills their infra for us.
Other providers, like OpenAI, at least provide a free moderation API. Google has a moderation API, that after the free 50k requests it is more expensive than Gemini 1.5 flash (Moderation API costs $0.0005/100 characters vs Gemini 1.5 flash $0.000001875/100 characters).
Eh... I don't know about that. Their tech graveyard isn't as populous as Google's, but it's hardly empty. A few that come to mind: ATL, MFC, Silverlight, UWP.
I can write something with Microsoft tech and expect it with reasonable likelihood to work in 10 years (even their service-based stuff), but can't say the same about anything from Google.
That alone stops me/my org buying stuff from Google.
And even if C++/CX and C++/WinRT aren't that great, with worse tooling than MFC and in maintenance mode, you can easilly create an application with them today.
Hardly the same can be told of most Google technologies.
We do run some non-customer facing assets in Google Cloud. But the website and API are on AWS.
https://www.reddit.com/r/googlecloud/comments/wpq0eg/what_is...
The top comment in that page is:
So I click that link, and I'm greeted with: This is why AWS and Anthropic are getting our money. We can not trust that Google projects will survive as long as our business needs.I don't know about that: my wife built her first SME on Google Workspace / GSuite / Google Apps for domain (this thing changed names so many times I lost track). She's now running her second company on Google tools, again.
All she needs is a browser. At one point I switched her from Windows to OS X. Then from OS X to Ubuntu.
Now I just installed Debian GNU/Linux on her desktop: she fires up a browser and opens up Google's GMail / GSuite / spreadsheets and does everything from there.
She's a happy paying customer of Google products since a great many years and there's actually phone support for paying customers.
I honestly don't have many bad things to say. It works fine. 2FA is top notch.
It's a much better experience than being stuck in the Windows "Updating... 35%" "here's an ad on your taskbar" "you're computer is now slow for no reason" world.
I don't think they'l pull the plug on GSuite: it's powering millions and millions of paying SMEs around the world.
To be fair, it's not that they're bad at it -- it's that they generally have an explicit philosophy against it. It's a choice.
Google management doesn't want to "pick winners". It prefers to let multiple products (like messaging apps, famously) compete and let the market decide. According to this way of thinking, you come out ahead in the long run because you increase your chances of having the winning product.
Gemini is a great example of when they do choose to focus on a single strategy, however. Cloud was another great example.
As a user I always still wish that there were fewer apps with the best features of both. Google's 2(!) apps for AI podcasts being a recent example : https://notebooklm.google.com/ and https://illuminate.google.com/home
> hard to deny that their LLM models aren't really, really good though
I'm so scarred by how much their first Gemini releases sucked that the thought of trying it again doesn't even cross my mind.
Are you telling us you're buying this press release wholesale, or you've tried the tech they're talking about and love it, or you have some additional knowledge not immediately evident here? Because it's not clear from your comment where you are getting that their LLM models are really good.
Although I do still pay for ChatGPT, I find it dog slow. ChatGPT is simply way too slow to generate answers. It feels like --even though of course it's not doing the same thing-- I'm back to the 80s with my 8-bit computer printing thing line by line.
Gemini OTOH doesn't feel like that: answers are super fast.
To me low latency is going to be the killer feature. People won't keep paying for models that are dog slow to answer.
I'll probably be cancelling my ChatGPT subscription soon.
About a year ago, I was saying that Google was potentially walking toward its own grave due to not having any pivots that rivaled OpenAI. Now, I'm starting to think they've found the first few steps toward an incredible stride.
Most of these things seem to just be a system prompt and a tool that get invoked as part of a pipeline. They’re hardly “agents”.
They’re modules.
Bad
"Good"It's like a meme that can be milked for monetization.
I have an A.I. textbook that has agent terminology that was written preLLm days. agents are just autonomous ish code that loops on itself with some extra functionality. LLMs in their elegance can more easily out the box selfloop just on the basis concatenating language prompts, sensibly. They are almost agent ready out the box by this very elegant quality(the textbook agentic diagram is just a conceptual self perpetuation loop), except…
Except they fail at a lot or get stuck at hiccups. But, here is a novel thought. What if an LLM becomes more agentic (ie more able to sustain autonomous chain prompts that do actions without a terminal failure) and less copilotee not by more complex controlling wrapper self perpetuation code, but by means of training the core llm itself to more fluidly function in agentic scenarios.
a better agentically performing llm that isnt mislabeled with a bad buzzword might not reveal itself in its wrapper control code but through it just performing better in an typical agentic loop or environment conditions with whatever initiating prompt, control wrapper code, or pipeline that initiates its self perpetuation cycle.
It'll create endless consulting opportunities for projects that never go anywhere and add nothing of value unless you value rich consultants.
All the common tricks, like creating a list of steps that are then executed by specialized agents in order, for example, fall flat as soon as one agent returns a result that contradicts the initial steps. It's simply a bandaid for short context sizes and LLMs that can't remain focused past the first few thousand tokens of prompt.
https://gemini.google/advanced/?Btc=web&Atc=owned&ztc=gemini...
then sign in with Google account and you'll see it
I will pay more to not feed Google.
We use new words so often that we take it for granted. You've passively picked up dozens of new words over the last 5 or 10 years without questioning them.
I don't think these are necessary buzzwords if the product really does what they imply.
I'm getting fairly excited about "agentic" solutions to the point that I even went out of my way to build "AgentOfCode" (https://github.com/JasonSteving99/agent-of-code) to automate solving Advent of Code puzzles by iteratively debugging executions of generated unit tests (intentionally not competing on the global leaderboard).
And even for this, there's actually only a SINGLE place in the whole "agent" where the models themselves actually make a "decision" on what step to take next, and that's simply deciding whether to refactor the generated unit tests or the generated solution based on the given error message from a prior failure.
Intelligence is a characteristic.
‘Intelligent’ is exactly as precise as ‘funny’ or ‘interesting’. It’s a label for a cluster of observations of another agent’s behavior. It entails almost nothing about how those behaviors are achieved.
This is of course only an opinion, but it’s my professional opinion after thirty five years in the AI business.
Agentic to me means that it acts somewhat under its own authority rather than a single call to an LLM. It has a small degree of agency.
agentic == not people.
Quite sensible, really.
"GVP stands for Good Pharmacovigilance Practice, which is a set of guidelines for monitoring the safety of drugs. SVP stands for Senior Vice President, which is a role in a company that focuses on a specific area of operations."
Seems lot of pharma regulation in my telecom company.
autocorrect of "significantly improved"?
I need to rewire my brain for the power of these tools
this plus the quantum stuff...Google is on a win streak
Why do you want the hardware vs just using it in the cloud? If you're training huge models you probably don't also keep all your data on prem, but on GCS or S3 right? It'd be more efficient to use training resources close to your data. I guess inference on huge models? Still isn't just using a hosted API simpler / what everyone is doing now?
(you'll probably also need to increase your quotas right away, the default is only 10 requests per minute).
Find something you like, use it, be ready to look again in a month or two.
EDIT: Typo
I find myself just paying a la carte via the API rather than paying the $20/mo so I can switch between the models.
Though gpt-4o could say "David Mayer" on poe.com but not on chat.openai.com which makes me wonder if they sometimes cheat and sneak in different models.
I am currently struggling to diagnose an ipv6 mis-configuration in my enormous aws cloudformation yaml code. I gave the same input to Claude Opus, Gemini and ChatGPT ( o1 and 4o).
4o was the worst. verbose and waste of my time.
Claude completely went off-tangent and began recommending fixes for ipv4 while I specifically asked for ipv6 issues
o1 made a suggestion which I tried out and it fixed it. It literally found a needle in the haystack. The solution is working well now.
Gemini made a suggestion which almost got it right but it was not a full solution.
I must clarify diagnosing network issues on AWS VPC is not my expertise and I use the LLMs to supplement my knowledge.
But it think it has to do more with the freshness of training data.
AWS IPV6 Egress is a new technology from AWS which was introduced only recently. Previously, we had to deploy NAT gateway which supported IPV4. I am assuming claude-3-5-sonnet-20241022 (latest) was not trained on this data.
>General availability will follow in January, along with more model sizes.
>Benchmarks against their own models which always underperformed
>No pricing visible anywhere
Completely inept leadership at play.
Publishers are being squeezed and going under, or replacing humans with hallucinated genai slop.
It’s like we’re taking the private equity model of extracting value and killing something off to the entire web.
I’m not sure where this is headed, but I don’t think Sundar has any strategy here other than playing catch up.
Demis’ goal is pretty transparently positioning himself to take over.
Additionally, Microsoft didn't really have any advantage in the smart phone space.
Google is already a product the majority of people on the planet use regularly to answer questions.
That seems like a competitive advantage to me.
I think people just have rose-tinted glasses on. Sure the hardware from Nokia was great, but software was very poor even by the standards of that time.
By his own admission, Gates was extremely distracted at the time by the antitrust cases in Europe, and he let the initiative die.