No; it could easily go either way. The challenge is that an authoritative statement (y'know, most people don't assume that the marketing website is lying to them as a general rule) that a region launch is coming puts AWS themselves in a very uncomfortable position of having to either deny it, or remain silent and start getting inquiries from the region's governing bodies.
It's just a mess. I don't know for a fact whether it was lying or leaking.
LLMs don't lie, they are instead ALWAYS hallucinating. Sometimes those hallucinations will coincidentally match a fact from training data, but a LLM has no concept of truth. They are far more reliable when you restrict them to transforming information than when you allow them the latitude to generate it.
I think your article makes plenty of valid points and was an interesting read. Given the number of times you pre-empt criticism, I suspect you’re aware you are being a little unfair on something that’s openly declared to be beta/preview. But equally you rightly observe that Amazon are pushing this as if it’s production ready!
Amazon Q has been pretty useful for AWS console and I find it using more and more to clarify different ways to architect a solution. Most of the times, it can generate command line, API calls and IAM policies pretty okay.
Rest - all models are actually fakers, hallucinators. It's just that those hallucinations coincidentally mapping on to the reality.
No model even ChatGPT has any understanding of anything whatsoever, the fundamental building blocks and algorithms are all the same. Pretty much.
I'm glad you've had positive experiences with it -- mine have 100% been squarely in the same tone as TFA: it is UI intrusive and absolutely stunningly useless for the reasonable questions I've asked it
The example I have on-hand because we laughed about it in Slack is:
> how much does a db.r7g.4xlarge cost per hour?
Q: Visit the Amazon EC2 pricing page at https://aws.amazon.com/ec2/pricing/on-demand/ for current On-Demand pricing
geez, thanks, asshole; did you perhaps mean https://aws.amazon.com/rds/postgresql/pricing/?pg=pr&loc=3#O... [1] instead of just making up some shit? I voted its dumb answer down, and actually submitted an explanation in their textbox, which I'm sure will go into some training run of something and at next year's re:Invent they'll reinvent Amazon Q as Amazon Q Prime or such, now with more RLHF!11
I’m becoming convinced that using AI from a company that has any ulterior business incentives than providing AI services is a recipe for bias like this.
probably a sane default position, but in this specific case AWS's goals for a "helpful agent" align with my own: to consume more of their resources
Had the thing answered me, it's possible I would have launched a db.r7g.4xlarge cluster at that very moment, burning almost $2/hr and then maybe even forgotten about it, always in AWS's best interest
Instead, they have to hope that I want to run that so bad I'll look up the cost on my own, at my own cognitive expense and time, and then navigate back to that page to try again
I guess, having now typed that out, their incentives are disjoint with my own as far as accurately answering pricing questions goes: I guess there is also an audience for whom getting a nonsensical answer just causes them to YOLO the cluster and only tear it down later when they realize it's been burning $2/hr -- possibly then getting mad at AWS for price gouging
I don't have a ChatGPT account in order to ask it, but based on Corey's post it seems that had I asked ChatGPT for the price, there's a non-trivial chance I would have gotten an accurate answer from it having scraped the Internet, including docs.aws.amazon.com, or via its newfound ability to invoke APIs and look up the price at that moment
14 comments
[ 3.5 ms ] story [ 43.2 ms ] threadIt's just a mess. I don't know for a fact whether it was lying or leaking.
Amazon Q has been pretty useful for AWS console and I find it using more and more to clarify different ways to architect a solution. Most of the times, it can generate command line, API calls and IAM policies pretty okay.
Rest - all models are actually fakers, hallucinators. It's just that those hallucinations coincidentally mapping on to the reality.
No model even ChatGPT has any understanding of anything whatsoever, the fundamental building blocks and algorithms are all the same. Pretty much.
The example I have on-hand because we laughed about it in Slack is:
geez, thanks, asshole; did you perhaps mean https://aws.amazon.com/rds/postgresql/pricing/?pg=pr&loc=3#O... [1] instead of just making up some shit? I voted its dumb answer down, and actually submitted an explanation in their textbox, which I'm sure will go into some training run of something and at next year's re:Invent they'll reinvent Amazon Q as Amazon Q Prime or such, now with more RLHF!111: or, of course, huge props to https://instances.vantage.sh/rds/ which gives the no-bullshit version unlike AWS's dog vomit pages
Had the thing answered me, it's possible I would have launched a db.r7g.4xlarge cluster at that very moment, burning almost $2/hr and then maybe even forgotten about it, always in AWS's best interest
Instead, they have to hope that I want to run that so bad I'll look up the cost on my own, at my own cognitive expense and time, and then navigate back to that page to try again
I guess, having now typed that out, their incentives are disjoint with my own as far as accurately answering pricing questions goes: I guess there is also an audience for whom getting a nonsensical answer just causes them to YOLO the cluster and only tear it down later when they realize it's been burning $2/hr -- possibly then getting mad at AWS for price gouging
I don't have a ChatGPT account in order to ask it, but based on Corey's post it seems that had I asked ChatGPT for the price, there's a non-trivial chance I would have gotten an accurate answer from it having scraped the Internet, including docs.aws.amazon.com, or via its newfound ability to invoke APIs and look up the price at that moment
Hard to ignore the irony of this complaint appearing on a site that itself obscured half the content with an enormous advert featuring a duck.