Personal opinion, I'm not sure this matters too much. This is using your own session cookie to access Bard, and it will be rate limited to whatever rate limit Google apply to individuals using Bard, which will likely be insufficient to run a business off.
If you want to get a better API, more model access, and higher quotas, you can pay for access via Google Cloud.
Disclaimer, I work for Google, not on this, and this is all just my personal opinion not based on any insider info.
They really need to release an official API. I was trying to find out the pricing for API access, but it's not available yet. The API used by this Python package seems to go through an unofficial route.
Thanks for sharing the link. It is a testament to Google's failure marketing its cloud offerings that, despite wanting to try out this API, I had been unable to find it until you helpfully shared the link.
I went down the cloud console rabbit hole to play with it. Only the bison model was available (of the non-descriptively named gecko, otter, bison, unicorn models). For innocuous prompts I got this error: "The response is blocked because the input or response potentially violates our policies. Try rephrasing the prompt or adjusting the parameter settings. Currently, only English is supported."
Where it did respond, I liked the quality. It was comparable to the GPT-3.5 API. Perhaps a little better even.
From personal experience, Bard's quality seems to be between GPT-3.5 and GPT-4, closer to GPT-4 if you have to bet. Except when fresh or live data matters, where Bard is clearly superior. (Bard's training data is up to Feb 2023 compared to ChatGPT's Sep 2021, and Bard also gets live data from Google search.)
MMLU benchmark score agrees with this estimate: GPT-3.5 (70.0%), PaLM 2 (81.2%), GPT-4 (86.4%).
I wonder if I’m the only one who’s getting tirelessly bored of all this AI stuff in the news. Sure it has some entertainment value and it is interesting and useful in certain ways, but the constant discussion and hype and near obsession with it, or at least that’s how it seems, is at odds with my reactions.
1) the hype machine is sickening. Its the crypto bros but on steroids and because $$$ the various money pundits have bought in. And philosophers. Excluding the people who should know better and use terms-of-art in ways which are misunderstood, the pundits are doing what they do based on their belief they know what the terms of art mean. A word like "hallucinate" implies a consciousness. They don't know its a co-opted term for when a model does crazy shit (even using the word crazy implies something)
2) the concerns Hinton and others express about regulation, and a need for oversight are real. Forget the hype machine, call these "expert systems" and reflect back on the machine assisted admissions model for the UK medical schools which entrenched "men get a +2 boost, women -2" enrollment because of how they modelled "do it like now"
So I am over the hype, and I am also thinking some of the back story "..wait" stuff is really critical.
I don't believe its heading to AGI. I don't believe its even close. I am super concerned about its (mis)application to the real world. Student essays? less concerned. we've dealt with calculators before. Applying gatekeeping to medicine or social services? or writing translations of instructions for machines which have consequences in the real world? Ruh-oh...
Watson was trumpeted as gods gift to diagnosticians. Then scales fell from their eyes. That said, the newer models image analysis and detection of early stage illness eg pancreatic cancer, thats truly exciting. I don't want scales to have to fall from anyone's eyes yet.
If we start firing diagnosticians before its field proven for a decade, we fucked up. If we use it as an adjunct to improve efficiency and reduce mis-diagnosis, I'm there.
What's really interesting to me is how a decent chunk of the voices behind wait and register are involved in the AI systems themselves. I'm guessing it's hoping to kick the ladder out from above
Nah, I think thats cynicism speak. For some of them, they're leaving the big shops because their concerns got buried in a rush to monetize. And because the time for their theoretics may be moving, and they're bored and don't need the hassle. Somebody like Hinton is my age +5, he's late stage career. Nobody gives him shit, he does what he wants.
Kicking the ladder out to stop young pups isn't in his model. Why would he do that? His legacy is secure.
I'm not convinced Scott Aarenson is doing his job right atm. He needs to be more "lets wait and see" and less "we've got a fix for this in planning. His "we have a fix for typing the words as AI sourced" went very quiet.
> reflect back on the machine assisted admissions model for the UK medical schools which entrenched "men get a +2 boost, women -2" enrollment because of how they modelled "do it like now"
Would you happen to have any further reading on this please, as a Brit this somehow passed me by without me ever hearing about it, sounds fascinating.
Bad news, with Microsoft selling this add a way of gaining ground on Google, and a bunch of other "hot startups" dog and pony shows, this will be in the news for awhile.
It's a big deal, and it wouldn't be hyped for no reason. If you think smart language models are just for "entertainment value", then I think your world view is at odds with others in the field. This is what the industry is hawking around, and you see a reflection of that on HN and other places. Eventually the novelty will fade, but this is still very new tech so I think there's still a lot more to come out before we get there. Now, this is just my view, but a Bard API is much more valuable news than a bug fix in a compiler or something.
You are witnessing the birth of a new frontier. A lot of businesses are getting created and disrupted. This is a healthy thing, the problems associated with AI notwithstanding. If you are in the middle of it, as I am, there is no time to get bored; you have to keep up.
> I wonder if I’m the only one who’s getting tirelessly bored of all this steam engine stuff in the news. Sure it has some entertainment value and it is interesting and useful in certain ways, but the constant discussion and hype and near obsession with it, or at least that’s how it seems, is at odds with my reactions.
During I/O they said that they'd be expanding the number of supported countries the day of the conference, but access is still restricted in France. I don't know what they're doing.
I wonder if Google has not yet released an API due to a desire to keep control and also ensure their products (docs, gmail) have a major lead over everybody outside Google? What is their motivation for not providing API access? It can’t be purely due to it being too difficult for them, since they are very skilled engineers.
40 comments
[ 436 ms ] story [ 1450 ms ] threadPopularly of this clearly shows there’s demand for it, so I hope Google officially releases an API soon.
Anyone know what I’m talkin about ??
And this API was inspired from it according to the author
If you want to get a better API, more model access, and higher quotas, you can pay for access via Google Cloud.
Disclaimer, I work for Google, not on this, and this is all just my personal opinion not based on any insider info.
I went down the cloud console rabbit hole to play with it. Only the bison model was available (of the non-descriptively named gecko, otter, bison, unicorn models). For innocuous prompts I got this error: "The response is blocked because the input or response potentially violates our policies. Try rephrasing the prompt or adjusting the parameter settings. Currently, only English is supported."
Where it did respond, I liked the quality. It was comparable to the GPT-3.5 API. Perhaps a little better even.
MMLU benchmark score agrees with this estimate: GPT-3.5 (70.0%), PaLM 2 (81.2%), GPT-4 (86.4%).
But it looks like you have to rent an instance of Vertex AI to use it. It doesn't seem to be pay per use as OpenAIs model are priced with.
I guess it's only fair that we may scrape Bard just like Bard scrapes us? I think it's called "disruption" :-)
Worked for us.
1) the hype machine is sickening. Its the crypto bros but on steroids and because $$$ the various money pundits have bought in. And philosophers. Excluding the people who should know better and use terms-of-art in ways which are misunderstood, the pundits are doing what they do based on their belief they know what the terms of art mean. A word like "hallucinate" implies a consciousness. They don't know its a co-opted term for when a model does crazy shit (even using the word crazy implies something)
2) the concerns Hinton and others express about regulation, and a need for oversight are real. Forget the hype machine, call these "expert systems" and reflect back on the machine assisted admissions model for the UK medical schools which entrenched "men get a +2 boost, women -2" enrollment because of how they modelled "do it like now"
So I am over the hype, and I am also thinking some of the back story "..wait" stuff is really critical.
I don't believe its heading to AGI. I don't believe its even close. I am super concerned about its (mis)application to the real world. Student essays? less concerned. we've dealt with calculators before. Applying gatekeeping to medicine or social services? or writing translations of instructions for machines which have consequences in the real world? Ruh-oh...
Watson was trumpeted as gods gift to diagnosticians. Then scales fell from their eyes. That said, the newer models image analysis and detection of early stage illness eg pancreatic cancer, thats truly exciting. I don't want scales to have to fall from anyone's eyes yet.
If we start firing diagnosticians before its field proven for a decade, we fucked up. If we use it as an adjunct to improve efficiency and reduce mis-diagnosis, I'm there.
Kicking the ladder out to stop young pups isn't in his model. Why would he do that? His legacy is secure.
I'm not convinced Scott Aarenson is doing his job right atm. He needs to be more "lets wait and see" and less "we've got a fix for this in planning. His "we have a fix for typing the words as AI sourced" went very quiet.
Would you happen to have any further reading on this please, as a Brit this somehow passed me by without me ever hearing about it, sounds fascinating.
That’s not what I said.
> Bard API is much more valuable news than a bug fix in a compiler or something
True, but that’s just cherry picking.
So I would say, it is simply not ready yet.
A working internal API is not automatically ready, to be a public facing API.