It works really well for questions that have easy answers and that are asked a lot.
It works less well when you have a question that doesn't have an obvious answer.
For fun, try 'How can I write to a Parquet file in Java without Hadoop?'. It will vacillate between giving answers that pull in hadoop dependencies, and nonsense that doesn't compile at all.
It also works well for questions that have easy/medium answers and aren’t asked a lot.
Complex questions that involve things that aren’t well documented are obviously going to be harder for an LLM to answer, particularly when the LLM does not have a capability to test its own code.
Main.java:27:33
java: org.apache.parquet.io.OutputFile is abstract; cannot be instantiated
Main.java:29:17
java: cannot access org.apache.hadoop.fs.Path
class file for org.apache.hadoop.fs.Path not found
It basically solves the problem by not importing hadoop dependencies (despite needing to), and hallucinating classes.
It works for impressively difficult questions, but even for simple ones (like this article is about) they would surely already be already answered a decade ago on Stack Overflow so there would be no need to ask them anew.
Cooking sites have been this way for a long time. Google for a recipe for something simple and the top results are long articles about the foods, the process, and so on. At least the better cooking sites put a link to the recipe near the top.
I take the point, but a developer with any experience is able to use google as a portal to get to the relevant SO or documentation page very quickly in spite of whatever shenanigans google+seo tries. It's not like every time you google some syntax you're actually distracted by tutorialspoint or whatever garbage. These sites must just feed on people who are in the first day or two of their careers, after that you don't even seen them.
> a developer with any experience is able to use google as a portal to get to the relevant SO
That's mostly because SO search is (or, has historically been) terrible.
> or documentation page
Ironically, documentation pages exist and many of them are terrible. But even the great ones aren't linked to in code snippets. Got a squiggly line? It would be handy to have a link to documentation!
> That's mostly because SO search is (or, has historically been) terrible.
It still is. There is a trick though - start typing a new question and it will suggest you possible duplicates. Works like magic, they really did a great job with this "search".
100% agréé, and not just on programming searches. That's why it's so easy for my to justify gpt plus to myself. Could be programming questions, holiday planning, business advice. Just about anything
> * But to me, the user, it feels like I’m paying for my answers with my time pointlessly scrolling around the page to find the answer I came to the site for. I can’t stand it; I feel like I am being duped, misled, being taken for a ride, just so I know how to make pancakes, or how to do a simple thing in common programming language.*
Absolutely agree with this. I strongly prefer reading the official language API documentation directly, which makes me very sensitive to the quality of the documentation.
I have also replaced my google searches (which usually are more along the lines of “what’s the most idiomatic way to do x?” or “what’s the most correct way to do x in $LANG?”) with ChatGPT. It’s super fast, generally pretty good for the types of style questions you’d usually have learning a new language, and I can directly query it about what tradeoffs different techniques have in various common situations.
ChatGPT has finally gotten me off Google’s Wild Ride.
I like to use the Bing AI for that because ChatGPT can sometimes hallucinate while Bing will use its knowledge + stuff from websites which makes it a more reliable source for things which should match reality.
On the contrary, I find all the LLMs (including ChatGPT browsing beta) which search to find answers repeat the same blogspam as everyone else. They save time by parsing the blogspam for me, but it’s still low quality.
Phind gets an honorable mention though - if that’s the style you prefer I’d strongly recommend it.
ChatGPT does hallucinate all the time, but for “learning a language from scratch” it’s very very very minimal. I can’t recall a single time it gave me a response to “idiomatic way to do x in $LANG” which included hallucinated information. Just can’t back it into a corner trying to get it to customize something that needs rigorous optimization for your use case.
Overall, the large innate knowledge base of ChatGPT-4 seems to contain a great model of the modern features of most programming languages and their packages (popular PyPI packages, Spring, Boost, Qt, etc, etc). Once I let it browse the internet, quality goes down and roughly matches Bing and Bard.
Phind I think gets around this by having a custom ranking of sources or something. Not sure what their secret sauce is but it mostly works pretty well.
I don’t think I’ve ever gotten language syntax wrong but ChatGPT will happily tell you to use functions that don’t exist or use them incorrectly. Also yes Bing sources aren’t necessarily quality sources but they are more aligned to reality than what ChatGPT will give you
By the author's metric is even worse to read a book. If you are looking for the exact answer to your exact code maybe that is the problem.
To get free training so you can solve your problem yourself and understand the overall situation seems a very good deal. The example to complain about seems very badly chosen.
I think that Google is getting worse and that advertisements in webpages should be regulated. But to complain that you are being taught to fish instead of given a fish seems too entitled.
There's a world of difference between a well-written, well-edited book and SEO spam. The latter is not making a genuine attempt to actually teach you, that's not the point.
If you grabbed a 'Java Basics' book, you would not expect the author to explain the entire implementation details of ArrayList before telling them about for-each.
But somehow the author of this article thinks that this kind of unnecessary level of detail is appropriate.
A book has an index, a bunch of sources at the end, and maybe a further reading section. The book tends to be more thorough since it doesn't have word limits or up votes to cater to. I wouldn't use the book example here as they tend to be more of a reference if not a how to guide for a very specific topic
How is it different that the page linked in the article? I do not see why that page is so bad, it looks like an average page from a book with a linked index at the top.
The proliferation of SEO fluff makes it way too convenient to use ChatGPT instead.
I'm well aware of ChatGPT's ability to confidently say wrong things, but the cost of reading SEO fluff is so high I'm more than willing for that tradeoff.
Interestingly I've found that GPT also tends to use SEO fluff, likely because its trained off SEO-fluffed content.
The difference is that I can just tell ChatGPT to be concise and the fluff is gone
Not OP, but yeah if you tell it to be concise it will be. With GPT4 you can even prescribe a maximum number of sentences and probably words too (haven't tried this one specifically). It will do it. It's a great tool!
With code questions I usually say: 'code only no explanation'. It gets a bit tiresome to type every time, but gpt4 is too slow to wait for it to 'type' the explanation
The number one reason blogs do this is because of links, which are very hard to build. Actually, these days nobody gives a flying fuck about linking to anyone else’s site anymore and Google knows this. They’ve openly stated that they want to devalue backlinks as a ranking metric.
So, what is the alternative? You plaster the page with an eBooks-worth of content because that satisfies Googles EAT (something something topical authority) signals. It’s that simple.
In many ways you can steer Googles results in a direction but developers are too lazy to make a /link section on their blog and save a bookmark once in a while. It actually helps the people whose content you like and it costs you absolutely nothing.
I absolutely hate the SEO crap that shows up these days, so much that I've become accustomed to immediately scrolling down past the first few results.
In contrast, I read all of K&R before writing a single C program, and when I had to learn PHP, I went to the official documentation (it was a downloadable .chm) and read through most of it first. It's a larger upfront investment, but I think taking the time to understand the language first instead of "JIT'ing" knowledge is worth it.
While you're right, of course, it does depend on whether you expect to be spending enough time on the language to be worth the investment. Anecdotally, a lot of developers will switch technologies or languages every couple of months or years; there's few left who say "I am a $language developer" and stick with that for, say, a decade. So for them it doesn't feel worth investing the time.
And also, googling for something has a high hit rate; nine times out of ten, googling 'how to sort in python' gives a good result.
And also also, these questions are often one-off; you need to look up how to do X in $lang, then either you can copy / paste, or you never touch it again; why put in the time investment if you don't need to rely on your memory?
a lot of developers will switch technologies or languages every couple of months or year
That says more about the horrible state of the industry than anything else. Big languages like Python won't be disappearing anytime soon, so even if I might not be doing more of it for now, learning the language more thoroughly will pay off again in the future.
Because it's rich in content, it's long, has a table of contents, the whole site is relevant to the search query, etc etc etc; from a search engine's point of view, it's more authoritative than Stack Overflow where the authoritartativity (?) is debatable at best, decided by consensus otherwise.
What should happen is that the community behind Python (in this case) works their ass off to make authoritative sources, like MDN did for web standards. And they had to work and fight for years to overtake w3schools.
There are likely tons of blog posts that answer the specific question "python iterate over dictionary" and nothing else. Search engines long ago gained the ability to detect heuristically that a search query was in the form of a question, and they could easily detect that this is a programming syntax question, and de-prioritize pages with 7,000+ words.
I've noticed the same thing recently: Googling for common programming stuff (in a language I don't normally use) will turn up results that usually answer the question, but they're bloated with preamble and filler.
I wonder how far Google could get with manual curation. Like if they said, these sites (eg, Stack Overflow, official documentation) have been around forever and have high-quality content, let's crank up their rankings for the appropriate classes of queries. That has its own problems, but it's gotta be better than the status quo.
The thing is, even if Google _could_ they aren't incentivized to. They are more incentivized to show the ad-ridden content, because often the websites are using Google itself to display the ads.
In the case described in the linked post, Google potentially benefits twice, financially. Why would they want to reduce their profit by showing the more relevant posts (those that quickly get to the answer)?
I don't know though, maybe StackOverflow also uses Google for their ads, but a quick looks suggests they simply have less ads than the results above them in the article's example search.
I know I'm beating a dead horse at this point, but I think one of the most underrated advantages of ChatGPT is simply that it says everything in exactly the same tone and style. When I google for things, who knows what page I'm going to land on. It could be full of ads. It could have tons of content I have to parse before I get to what I care about, as said in OP. It could be written clunkily, in a way that's hard to parse, or it could be littered with spelling mistakes. It could be overly verbose, or explain things that I don't need to know. All this creates high variance in the actual time it takes to parse out what I need.
When I ask ChatGPT it'll always give me the answer in a few paragraphs, and it's always written in the exact same way. What a boon!
> How is this the number one link on Google?
It's the number one link on Google because one proxy Google has to know if you got a result is that you lingered on the page for a long time. This is also, tangentially, why cooking recipe websites are so long. If you bounce immediately off a website, Google thinks that means that you didn't find it to be useful. ChatGPT does not have this problem.
Especially for these type of simple syntax questions that autocomplete or copilot isn’t getting. It’s so so quick to ask ChatGPT for the answer, rather than finding the documentation for the thing you’re looking for and parsing it.
Can’t wait until they (actually) monetize ChatGPT and maximize the time spent on the website, like Google. Can’t wait until ChatGPT says that, when their grandma went to school, they too needed to mentally add 26 + 13, and the teacher would always frown on pupils when they gave the answer 31.
ChatGPT’s strength is that it’s new on the market, needs to build an audience, and therefore their UI and UX are lean, devoid of gaming.
I'm not convinced this is the case. Unlike Google, ChatGPT has clear monetization channels which are not advertising, like ChatGPT+, which is already doing very well, and it's easy to imagine further channels which allow access to specialized models. Additionally, unlike Google, you can only really get away with making your service crappy when you have such a strong moat that it's still significantly better than the competition. The only moat OpenAI has is its model, which is proving a lot easier to replicate than Google circa 2008's moat.
Ads and shit doesn't exist because there isn't a clear way to extract value. They exist because the folks that make decisions love ads and shit. You will end up paying a subscription for a bad product with few alternatives, just like most folks end up using Google now and pay with all their data. There simply isn't a good search engine, Google murdered all the real competition and what was left is mediocre.
even the fact that you always go to that same window, with the same format and interface lowers cognitive load imo. Of course chatgpt hasnt been a magic bullet for me yet, but it definitely eases the pain of developing things outside your usual perview
I’m travelling in France at the moment and chatgpt is an absolutely incredible tour guide.
Yesterday we asked it what “RF” inscribed in a crest on a government building meant (“République Française”). A few days ago we asked why an area was called Gobelins, and if that had any relationship to the modern goblins in fantasy. I asked how to ask for more bread at the table and it gave me a pretty long, fancy French sentence. I asked it to simplify the sentence and it did beautifully. You can ask Google translate to translate something, but chatgpt can give you multiple translations, and tell you the cultural nuances of all of them and teach you French grammar while it’s at it. “How do I say this? Why does it use suis in that sentence? Is that a weird thing to say to the wait staff at a restaurant? What is the protocol there?” Etc.
I do wonder: does / how does chatGPT differentiate between ad-driven content and regular content?
For example, if I ask what the best vacuum cleaner on the market is, I'd rather it not regurgitate bestvacuumreviews.net which is affiliate (not quality) driven
It used to be why Recipe websites were so long, but that’s no longer the case.
There’s three reasons…(1) ad revenue, (2) there’s a belief that the extended life stories confer copyright to recipes, which otherwise cannot be copyrighted and (3) everyone does it.
Source: I made SimplifyRecipe.com, which requires diving into the minutiae of recipe authors.
Otherwise, I agree that getting consistent results from GPT is super useful.
Or they work for a selection of cases but not others. Or they seem to work in a limited number of tests but aren’t truly doing the right thing. Or they do work but have a security flaw that makes your service vulnerable.
Coding solutions have tradeoffs and subtleties which go beyond “it did the thing right a couple of times when I ran it”. If that’s all it took to verify code is good, there wouldn’t be any bugs; you can test code a person writes too.
Yes, ChatGPT can't do all the work for you. You have to know how to test the code thoroughly and think of edge cases and performance tradeoffs and all those other subtleties.
But contrast this with other types of information. If I ask ChatGPT "how hot is the surface of the sun?" and it answers "15 million degrees Fahrenheit", I have zero ability to verify that information. As a result, I don't find ChatGPT useful for most informational searches, except the ones related to coding.
Yeah, in my experience, Stack overflow answers suffer from subtle errors all the time too. Especially for basic questions -- the people answering and upvoting the answers are inexperienced devs, so they often upvote answers that are either the wrong way to do it (eg. reimplement something that has a very elegant built in way to do), or they just contain bugs (eg. because they mixed up the number of characters vs the number of bytes in a string).
They are still useful, you just have to make sure you understand what you are doing and read the relevant docs before you paste the code.
Things like "How do I delete a value from a dict" probably either works or it doesn't.
Then again, for that level of simple stuff, one can probably search a PDF cheat sheet just with their eyes faster than they can type the question for ChatGPT
It might sound weird, but you can actually ask it to do this too.
On the surface it makes no sense to do this since it already gave you an answer. But that's not how LLMs work - they can simulate a variety of different "humans" depending on the prompt and presenting a different prompt persona (security researcher, code reviewer that focuses on edge cases, etc) yields a different quality of response.
Well, but you still don't know if any of those persona are correct. It's great that you can talk to it, but ultimately you have to make the final judgement call, as it will bullshit you, and it can be much better at bullshitting than most humans.
It’s not really an issue for me because I generally know very quickly if it’s wrong or hallucinating and can easily get it back on track.
And in general I find it’s more accurate and far more concise than anything that’s going to show up in a Google search.
I mean, there’s no way I’m taking what it says at face value or copy / pasting code from its response, but that goes for stack overflow, or any other source too. (maybe sans official docs)
I'm curious. How do you know it is speaking untruths? What kind of questions do you ask where you know what a wrong answer would be? Is this just like "remind me how to reverse a string in go" kind of thing? Where you know how to do it, but cannot quickly recall?
Yep, exactly. I ask things that I know the answer to, or at least know enough to understand what the right answer is going to look like. I just can’t recall the specifics or am busy thinking through something else. It’s basically just an assistant, saving me the time of digging through official docs, searching previous project code, my own notes, etc.
The real problem is ad business. If chatgpt becomes monopoly we will see many ad banners. After several years of monopoly we will see chatgpt anders lacking so that you will have to see more banners in response. After some time quality of answers will depend on the amount of money you put into your chatgpt subscription. You will not be able to go back to google. There will be no google any more.
The only really workable solution imo is that search companies need to make their money on searches, not ads; something like Kagi I guess. Any perverse incentive is inimical to a really good search engine.
I think the internet is going to start to stratify into paid and "free" tiers going forward, and honestly it's about goddamned time.
A huge drawback of Kagi is that it requires an account and can associate your search history with it. Using other search engines while not logged in is more privacy friendly.
That problem seems very solvable if they don't need to monetize your data. You might, for example, turn off search history and the use of search history to guess what you might mean with new searches.
I really doubt that using Google while not logged in is more 'privacy friendly' than anything else, given their ability to track with pixels and shadow profiles. In any case I don't particularly care about that issue, I'm not searching for bomb-making guides or anything.
What I care about is a search engine that's trying its hardest to serve me answers to my searches, rather than feeding me ads.
AI tools today are great for the same reason Google was great in its first few years of existence: they want to attract users and sell this awesome new product. But this will quickly change.
The ad and investor-fueled enshittification will inevitably also come to popular AI tools, but in even more sinister ways than web search results. Forget banners and traditional ad campaigns. Users will start seeing subtle product placements in the responses, and certain brands placed higher than others, all embedded in the friendly conversational tone they're used to, to make them even more susceptible to manipulation. Thanks to the sea of private data users are willing to share with their "personal" AI assistant, the responses will be perfectly fine-tuned so that ads are targeted at them _personally_. Advertisers will no longer need to target broad cohorts who may or may not be interested in their brand. They will be able to reach _exactly_ people who are most likely to make a purchase, especially if the ad is delivered at _exactly_ the right moment when the person is most vulnerable.
The only way to avoid this is to use open source and self-hosted AI tools, but the vast majority of people will use whatever service is popular and "free", same as today.
And, not only that. When it explains something and you don't understand, you can ask it to rephrase or give analogies. I guess the only thing missing is imagery
For basic stuff it is much faster than Google. But if it requires minimum thinking it still faster doing myself.
The other day I was lazy to find a generic scraper I have so I could save a reference manual.
I know it is just like 50 lines of code and if you are smart you could do it async. I tried with ChatGPT and it didn't work at all. Many silly mistakes hidden between comments that looked reasonable
I love the way there are 10 competing folk explanations for "why are recipe pages online so long?"
Spending time on the page should actually be a great proxy for usefulness on recipe websites. If it's a long page of introduction and anecdotes, maybe it took me three minutes instead of one to reject it as unpromising. If it's the recipe I actually used, however, that tab is going to be open for the next thirty minutes or more.
I’ve seen “they’re long so there’s more space for ads” and “they’re long so you spend more time there so it doesn’t get penalised by google”, and to me those are pretty closely related.
I see that one a lot and it doesn't hold up to scrutiny because copyright on grandma isn't how recipe websites make their money. They trade in a generic field flooded with fly-by-nights that exist exclusively to run ads. They don't care nor need to care whether their filler material isn't copied.
But why would staying long on a web site correctly indicate the page provided the answer? Is you spent a short amount it could also indicate you found what you were looking for, and off you went.
It's a situation where Google doesn't benefit much more from it being the actual answer, and the site doesn't either.
In the long run it could have an impact on the user perception, that could be nullified by the page still staying ranked higher than the competiting pages and the user finding nowhere else better.
It happened when Google Penguin was rolled out, because Penguin’s claimed focus was on better content itself to combat backlink spam practices that abused PageRank descendants.
It sort of worked but there were entire valuable websites with great content (but according to Penguin comprised of non-rich content) that were dropped pages deep into the search results. Recipe websites were among these.
There exist reams of articles on how to fix your content to get Penguin to like it, and the buried recipe nonsense started exactly at the same time.
Funnily enough, I see this as one of the biggest disadvantages of ChatGPT. I am used to searching the web and navigating results, and have developed a lot of heuristics for assessing the credibility of a page. ChatGPT flattens the true, not-true-but-maybe-helpful, and straight up time-wasting BS into the same confident patter. So I have to treat everything it says with equal scepticism.
The time wasting SEO BS is all but non existent in my use. ChatGPT has nearly eliminated Google because if I want to find the best 10 films involving aliens in the 90s by obscure directors I get exactly that, instead of three million pages of trash.
1. Build a skill set to navigate millions of pages of fluff SEO garbage and gain intuition of which sites are more likely to be credibility based on something akin to a internet dosing rod.
2. Build skill set to evaluate if a statement or claim is factually true. i.e. ChatGPT just told me something, how likely is it to be true.
I know which skill set I would rather develop, obviously #2, as it's transferable to nearly every intellectual venture a human would consider, while #1 is not.
Also, I feel the people that play the FUD campaign on GPT hallucinations just haven't used it much. The average quality of GPT answers, at least in programming, is far above the average hit on google, and the fact that I can interact with the answer and shape it to go in any direction I want or mix it match it with nearly any other concept on the spot, it's light years more valuable than the average google search. At this point, I would guess I only resort to google 10% of what I used to, and it's normally to attempt to find a specific website that has specific information I remember accessing before that I want to look up again.
There is a YouTube Premium and I don't see a reason why there couldn't exist Google Search Premium but that again doesn't solve the problem of SEO spam and garbage websites.
I've been using Kagi [1] for both technical and non-technical searches recently, and not having to mentally filter ads out of results is a delight. The top result is almost always very good.
This is literally above the fold[1]. The Python website linked is actually pretty good, what a dumb click-baity blog post. I'll probably get downvoted by the "I use ChatGPT daily and it 15x'ed my productivity" crowd, but every time I have this debate, I get zero salient examples.
The index was on page 2 on the authors screen, you probably have a bigger screen.
But even if you click the link in the index, then you still need to scroll down two to three pages to get to the actual answer:
for key in a_dict:
print(key, '->', a_dict[key])
I don't know about you, but I think that an article title "How to Iterate Through a Dictionary in Python" should maybe start with that snippet, instead of first explaining over 9 pages:
- "What dictionaries are, as well as some of their main features and implementation details"
- "How to iterate through a dictionary in Python by using the basic tools the language offers"
- "What kind of real-world tasks you can perform by iterating through a dictionary in Python"
- "How to use some more advanced techniques and strategies to iterate through a dictionary in Python"
Yes, if you spend 30 minutes reading the article, you will know how to iterate over dictionaries afterwards. But if you asked a human being, they would just say "for key in dict:".
Edit: I tried sending the query "How to Iterate Through a Dictionary in Python" to ChatGPT, and it immediately shows relevant code samples that exactly answer the question, without pages of pointless filler: https://imgur.com/a/CenZFl0
There's no arguing that the ChatGPT answer is a million times better.
Yep bad example indeed.
BTW, does imgur allow linking images directly anymore? Your link opened a page with your tiny screenshot at top and some random image for rest of the page and it was very confusing.
I felt it wasn't fair to say the Real Python page about iterating over dicts had "useless" content. It had a table of contents, what's wrong with that? It's going into detail about dictionaries. Sure, you just want a quick fix. That page is not for you but there is no need to caption a screenshot with "…so…much…nonsense". Nonsense is a bit extreme, isn't it?
Especially as realpython is one of the websites when looking for anything Python related. Their explanations are clear, concise and easy to read, not to mention that they usually cover the whole topic, not just some minor aspect of it (hello SO). When realpython is among the hits I know I don't need to look any further.
However, I would argue it is concise. It just doesn't limit the page to a single subtopic / subquestion that someone might ask, but instead covers the whole topic. Feel free to skip most of it and take just solution to the problem at hand (index is usually great for this purpose).
Not really when the "Introduction to Dictionaries", which doesn't belong on an article about how to iterate through them, is 20 paragraphs long.
If you want to write an article on Python dictionaries, sure spend 20 paragraphs on it. But don't title the article "How to iterate through a dictionary"
I'm surprised they didn't spend another twenty describing the origination of term iterate from the original Latin 'iterum'.
Yeah, finding a pretty complete article about dictionaries and having to scroll down after finding your exact doubt in one item in the table of contents is very painful. I feel sorry for whoever goes through such nightmare. /s
25 years in, how is it possible not to have mastered this like muscle memory? You're supposed to develop skills to quickly assess search results, scan a site for trustworthiness, find the info you need on a page through heuristics you've either been building for decades or have developed natively because you were born into a world where it's a primary skill. It's become like breathing, hasn't it?
I can't begin to describe how much it slows me down to have to keep typing out my question repeatedly to an AI, only to be robbed of the ability to judge the information in the context of where it came from, and oh yeah, any confidence that it's accurate.
Younger people haven't had the same time to build these heuristics. Some of these heuristics you're utilizing may also be referencing what information looked like prior to the influx of blog spam.
> Younger people haven't had the same time to build these heuristics.
This is a key point, and I don't think the ability to acquire knowledge online is a skill which only young people specifically have failed to acquire.
Said ability must be trained on, as with anything, a suitable dataset. This objective cannot be meaningfully achieved if all data sources someone has ever consumed and learned from are TikTok, ragebait, clickbait, affiliate link blogs, email chains, WhatsApp message chains, sensationalist news outlets, etc.
This is why people still fall for the Nigerian Prince, Hot MILFs in Your Area, job application with deposit and all the modern variants; not only old people but all types of people.
I think some of us nerds have a slight advantage in that we experience the internet early on in a slightly different way: as a parallel to the real world, with friends and foes, with truth and deceit and as a playground to write a game mod or a bot for shit and giggles. We are, due to that I'd pose, much more keen to learn how and be able to effectively navigate it.
The longer you stay on a webpage, the more favorably Google perceives it. Even if you're frustrated sifting through spam, if you find your answer quickly and spend less time, it doesn't count as a positive quality signal.
To counter this, websites employ a strategy known as the "Bucket brigade" to ensure you spend at least 30 seconds on their site. It's a basic principle of website optimization and it's just the tip of the nasty SEO iceberg.
I don't know if anyone asked and I don't know if it can be figured out but I was wonderin are the bots any better than figuring out them search operators?
166 comments
[ 3.3 ms ] story [ 207 ms ] threadIt works less well when you have a question that doesn't have an obvious answer.
For fun, try 'How can I write to a Parquet file in Java without Hadoop?'. It will vacillate between giving answers that pull in hadoop dependencies, and nonsense that doesn't compile at all.
(here's a useful proper answer: https://stackoverflow.com/questions/59939309/read-local-parq...)
Complex questions that involve things that aren’t well documented are obviously going to be harder for an LLM to answer, particularly when the LLM does not have a capability to test its own code.
I briefly checked if the mentioned dependencies exist (they do). Too late over here to actually try the code.
That's mostly because SO search is (or, has historically been) terrible.
> or documentation page
Ironically, documentation pages exist and many of them are terrible. But even the great ones aren't linked to in code snippets. Got a squiggly line? It would be handy to have a link to documentation!
It still is. There is a trick though - start typing a new question and it will suggest you possible duplicates. Works like magic, they really did a great job with this "search".
Absolutely agree with this. I strongly prefer reading the official language API documentation directly, which makes me very sensitive to the quality of the documentation.
I have also replaced my google searches (which usually are more along the lines of “what’s the most idiomatic way to do x?” or “what’s the most correct way to do x in $LANG?”) with ChatGPT. It’s super fast, generally pretty good for the types of style questions you’d usually have learning a new language, and I can directly query it about what tradeoffs different techniques have in various common situations.
ChatGPT has finally gotten me off Google’s Wild Ride.
Phind gets an honorable mention though - if that’s the style you prefer I’d strongly recommend it.
ChatGPT does hallucinate all the time, but for “learning a language from scratch” it’s very very very minimal. I can’t recall a single time it gave me a response to “idiomatic way to do x in $LANG” which included hallucinated information. Just can’t back it into a corner trying to get it to customize something that needs rigorous optimization for your use case.
Overall, the large innate knowledge base of ChatGPT-4 seems to contain a great model of the modern features of most programming languages and their packages (popular PyPI packages, Spring, Boost, Qt, etc, etc). Once I let it browse the internet, quality goes down and roughly matches Bing and Bard.
Phind I think gets around this by having a custom ranking of sources or something. Not sure what their secret sauce is but it mostly works pretty well.
To get free training so you can solve your problem yourself and understand the overall situation seems a very good deal. The example to complain about seems very badly chosen.
I think that Google is getting worse and that advertisements in webpages should be regulated. But to complain that you are being taught to fish instead of given a fish seems too entitled.
It does not look like SEO spam to me. The site seems actually nice and well produced.
I may be missing something, thou. Why is that page so bad that the author does not want it in the list of results?
But somehow the author of this article thinks that this kind of unnecessary level of detail is appropriate.
I'm well aware of ChatGPT's ability to confidently say wrong things, but the cost of reading SEO fluff is so high I'm more than willing for that tradeoff.
Interestingly I've found that GPT also tends to use SEO fluff, likely because its trained off SEO-fluffed content.
The difference is that I can just tell ChatGPT to be concise and the fluff is gone
I preface all prompts with "Be concise and ask follow-up questions if you need more information" -- Greatly improves the output
So, what is the alternative? You plaster the page with an eBooks-worth of content because that satisfies Googles EAT (something something topical authority) signals. It’s that simple.
In many ways you can steer Googles results in a direction but developers are too lazy to make a /link section on their blog and save a bookmark once in a while. It actually helps the people whose content you like and it costs you absolutely nothing.
In contrast, I read all of K&R before writing a single C program, and when I had to learn PHP, I went to the official documentation (it was a downloadable .chm) and read through most of it first. It's a larger upfront investment, but I think taking the time to understand the language first instead of "JIT'ing" knowledge is worth it.
And also, googling for something has a high hit rate; nine times out of ten, googling 'how to sort in python' gives a good result.
And also also, these questions are often one-off; you need to look up how to do X in $lang, then either you can copy / paste, or you never touch it again; why put in the time investment if you don't need to rely on your memory?
That says more about the horrible state of the industry than anything else. Big languages like Python won't be disappearing anytime soon, so even if I might not be doing more of it for now, learning the language more thoroughly will pay off again in the future.
Because it's rich in content, it's long, has a table of contents, the whole site is relevant to the search query, etc etc etc; from a search engine's point of view, it's more authoritative than Stack Overflow where the authoritartativity (?) is debatable at best, decided by consensus otherwise.
What should happen is that the community behind Python (in this case) works their ass off to make authoritative sources, like MDN did for web standards. And they had to work and fight for years to overtake w3schools.
I wonder how far Google could get with manual curation. Like if they said, these sites (eg, Stack Overflow, official documentation) have been around forever and have high-quality content, let's crank up their rankings for the appropriate classes of queries. That has its own problems, but it's gotta be better than the status quo.
In the case described in the linked post, Google potentially benefits twice, financially. Why would they want to reduce their profit by showing the more relevant posts (those that quickly get to the answer)?
I don't know though, maybe StackOverflow also uses Google for their ads, but a quick looks suggests they simply have less ads than the results above them in the article's example search.
When I ask ChatGPT it'll always give me the answer in a few paragraphs, and it's always written in the exact same way. What a boon!
> How is this the number one link on Google?
It's the number one link on Google because one proxy Google has to know if you got a result is that you lingered on the page for a long time. This is also, tangentially, why cooking recipe websites are so long. If you bounce immediately off a website, Google thinks that means that you didn't find it to be useful. ChatGPT does not have this problem.
ChatGPT’s strength is that it’s new on the market, needs to build an audience, and therefore their UI and UX are lean, devoid of gaming.
Same as Google, up until 2008.
A lot more than I've ever paid to use the Google search engine.
Paid for service over ads is the new way forward.
Yesterday we asked it what “RF” inscribed in a crest on a government building meant (“République Française”). A few days ago we asked why an area was called Gobelins, and if that had any relationship to the modern goblins in fantasy. I asked how to ask for more bread at the table and it gave me a pretty long, fancy French sentence. I asked it to simplify the sentence and it did beautifully. You can ask Google translate to translate something, but chatgpt can give you multiple translations, and tell you the cultural nuances of all of them and teach you French grammar while it’s at it. “How do I say this? Why does it use suis in that sentence? Is that a weird thing to say to the wait staff at a restaurant? What is the protocol there?” Etc.
We’ve been using it constantly.
For example, if I ask what the best vacuum cleaner on the market is, I'd rather it not regurgitate bestvacuumreviews.net which is affiliate (not quality) driven
There’s three reasons…(1) ad revenue, (2) there’s a belief that the extended life stories confer copyright to recipes, which otherwise cannot be copyrighted and (3) everyone does it.
Source: I made SimplifyRecipe.com, which requires diving into the minutiae of recipe authors.
Otherwise, I agree that getting consistent results from GPT is super useful.
Or they work for a selection of cases but not others. Or they seem to work in a limited number of tests but aren’t truly doing the right thing. Or they do work but have a security flaw that makes your service vulnerable.
Coding solutions have tradeoffs and subtleties which go beyond “it did the thing right a couple of times when I ran it”. If that’s all it took to verify code is good, there wouldn’t be any bugs; you can test code a person writes too.
But contrast this with other types of information. If I ask ChatGPT "how hot is the surface of the sun?" and it answers "15 million degrees Fahrenheit", I have zero ability to verify that information. As a result, I don't find ChatGPT useful for most informational searches, except the ones related to coding.
(The example answer is wrong according to https://www.space.com/17137-how-hot-is-the-sun.html, albeit in a way that I could see potentially confusing an LLM.)
But the nice thing about ChatGPT is that you can give it more context than it is possible in a Google query, and you can ask follow-up questions.
They are still useful, you just have to make sure you understand what you are doing and read the relevant docs before you paste the code.
Things like "How do I delete a value from a dict" probably either works or it doesn't.
Then again, for that level of simple stuff, one can probably search a PDF cheat sheet just with their eyes faster than they can type the question for ChatGPT
On the surface it makes no sense to do this since it already gave you an answer. But that's not how LLMs work - they can simulate a variety of different "humans" depending on the prompt and presenting a different prompt persona (security researcher, code reviewer that focuses on edge cases, etc) yields a different quality of response.
And in general I find it’s more accurate and far more concise than anything that’s going to show up in a Google search.
I mean, there’s no way I’m taking what it says at face value or copy / pasting code from its response, but that goes for stack overflow, or any other source too. (maybe sans official docs)
I think the internet is going to start to stratify into paid and "free" tiers going forward, and honestly it's about goddamned time.
I have no way to verify that.
What I care about is a search engine that's trying its hardest to serve me answers to my searches, rather than feeding me ads.
AI tools today are great for the same reason Google was great in its first few years of existence: they want to attract users and sell this awesome new product. But this will quickly change.
The ad and investor-fueled enshittification will inevitably also come to popular AI tools, but in even more sinister ways than web search results. Forget banners and traditional ad campaigns. Users will start seeing subtle product placements in the responses, and certain brands placed higher than others, all embedded in the friendly conversational tone they're used to, to make them even more susceptible to manipulation. Thanks to the sea of private data users are willing to share with their "personal" AI assistant, the responses will be perfectly fine-tuned so that ads are targeted at them _personally_. Advertisers will no longer need to target broad cohorts who may or may not be interested in their brand. They will be able to reach _exactly_ people who are most likely to make a purchase, especially if the ad is delivered at _exactly_ the right moment when the person is most vulnerable.
The only way to avoid this is to use open source and self-hosted AI tools, but the vast majority of people will use whatever service is popular and "free", same as today.
For basic stuff it is much faster than Google. But if it requires minimum thinking it still faster doing myself.
The other day I was lazy to find a generic scraper I have so I could save a reference manual. I know it is just like 50 lines of code and if you are smart you could do it async. I tried with ChatGPT and it didn't work at all. Many silly mistakes hidden between comments that looked reasonable
Spending time on the page should actually be a great proxy for usefulness on recipe websites. If it's a long page of introduction and anecdotes, maybe it took me three minutes instead of one to reject it as unpromising. If it's the recipe I actually used, however, that tab is going to be open for the next thirty minutes or more.
I’ve seen “they’re long so there’s more space for ads” and “they’re long so you spend more time there so it doesn’t get penalised by google”, and to me those are pretty closely related.
It's a situation where Google doesn't benefit much more from it being the actual answer, and the site doesn't either.
In the long run it could have an impact on the user perception, that could be nullified by the page still staying ranked higher than the competiting pages and the user finding nowhere else better.
https://en.m.wikipedia.org/wiki/Google_Penguin
Actually you're probably getting someone's list of 10 best films that got copied to 1000 more sites and ended up training ChatGPT...
1. Build a skill set to navigate millions of pages of fluff SEO garbage and gain intuition of which sites are more likely to be credibility based on something akin to a internet dosing rod.
2. Build skill set to evaluate if a statement or claim is factually true. i.e. ChatGPT just told me something, how likely is it to be true.
I know which skill set I would rather develop, obviously #2, as it's transferable to nearly every intellectual venture a human would consider, while #1 is not.
Also, I feel the people that play the FUD campaign on GPT hallucinations just haven't used it much. The average quality of GPT answers, at least in programming, is far above the average hit on google, and the fact that I can interact with the answer and shape it to go in any direction I want or mix it match it with nearly any other concept on the spot, it's light years more valuable than the average google search. At this point, I would guess I only resort to google 10% of what I used to, and it's normally to attempt to find a specific website that has specific information I remember accessing before that I want to look up again.
Probably because there would be barely anything left in the results once ads and spam are removed.
[1] https://kagi.com
Basically, just go to cheat.sh/git and you get a list of useful examples for how to use the git command.
I have it mapped to a small bash function so that I can just type ‘cheat git’ and I get the content right in my terminal.
https://github.com/tldr-pages/tldr
https://github.com/dbrgn/tealdeer
[1] https://imgur.com/a/KFJ9DrB
I am declining your promise of erotic imagery but thanks for thinking of me.
But even if you click the link in the index, then you still need to scroll down two to three pages to get to the actual answer:
I don't know about you, but I think that an article title "How to Iterate Through a Dictionary in Python" should maybe start with that snippet, instead of first explaining over 9 pages:- "What dictionaries are, as well as some of their main features and implementation details"
- "How to iterate through a dictionary in Python by using the basic tools the language offers"
- "What kind of real-world tasks you can perform by iterating through a dictionary in Python"
- "How to use some more advanced techniques and strategies to iterate through a dictionary in Python"
Yes, if you spend 30 minutes reading the article, you will know how to iterate over dictionaries afterwards. But if you asked a human being, they would just say "for key in dict:".
Edit: I tried sending the query "How to Iterate Through a Dictionary in Python" to ChatGPT, and it immediately shows relevant code samples that exactly answer the question, without pages of pointless filler: https://imgur.com/a/CenZFl0
There's no arguing that the ChatGPT answer is a million times better.
Google should stop this. It's just going to make ChatGPT more attractive.
(not affiliated in any way)
I believe the whole discussion here is about the site being anything but concise.
If you want to write an article on Python dictionaries, sure spend 20 paragraphs on it. But don't title the article "How to iterate through a dictionary"
I'm surprised they didn't spend another twenty describing the origination of term iterate from the original Latin 'iterum'.
Far faster than wading through the verbose and insufferable output of some chatbot and your give the source of all that knowledge some recognition.
I can't begin to describe how much it slows me down to have to keep typing out my question repeatedly to an AI, only to be robbed of the ability to judge the information in the context of where it came from, and oh yeah, any confidence that it's accurate.
This is a key point, and I don't think the ability to acquire knowledge online is a skill which only young people specifically have failed to acquire.
Said ability must be trained on, as with anything, a suitable dataset. This objective cannot be meaningfully achieved if all data sources someone has ever consumed and learned from are TikTok, ragebait, clickbait, affiliate link blogs, email chains, WhatsApp message chains, sensationalist news outlets, etc.
This is why people still fall for the Nigerian Prince, Hot MILFs in Your Area, job application with deposit and all the modern variants; not only old people but all types of people.
I think some of us nerds have a slight advantage in that we experience the internet early on in a slightly different way: as a parallel to the real world, with friends and foes, with truth and deceit and as a playground to write a game mod or a bot for shit and giggles. We are, due to that I'd pose, much more keen to learn how and be able to effectively navigate it.
To counter this, websites employ a strategy known as the "Bucket brigade" to ensure you spend at least 30 seconds on their site. It's a basic principle of website optimization and it's just the tip of the nasty SEO iceberg.
I don't know if anyone asked and I don't know if it can be figured out but I was wonderin are the bots any better than figuring out them search operators?