It gets worse if the content you're trying to index are labeled with the same titles like "me_irl". While they're usually just pointless memes, there are occasionally a few gems that I would like to find again to share with friends but it's basically impossible with such poor signals. I think it's a bit sad that those rare gems are essentially lost and forgotten.
Thanks for posting op, it's interesting to think about.
> Reddit posts are good because the people who create these posts or make comments are doing it to share their knowledge.
I would offer a bit of a different perspective here, I think maybe even most do it for what you could call self-soothing.
For example, a lot of people share knowledge, but it's _their knowledge_, which can also be defined as _sharing their subjective past_ and that's known to be a very soothing thing for people who may even be troubled by the unknown in their personal lives. Hop online, boom, you're a domain expert in your own past. Find a place that's _about your past_, be it /r/linux or /r/formula1 or whatever, and there you go, it's comfy. You generally get an upvote + reply vibes dopamine bonus just for being there, due to your relevant past.
So, Reddit's sooth-sayers (so to speak) are kind of lazy but incentivized-lazy in some ways, which I think can also speak to some issues with the platform surrounding cognitive blind spots and what some here have called the echo chamber effect. That's an opportunity for a new search engine, in some ways, but it also may offer insights to new services that could be even more effective.
> And there is no financial incentive associated with it. [posting on Reddit]
There is though. I mean I've myself been financially incentivized into doing it, as have probably many, many others. Nothing sneaky, even.
You've got people posting their stuff all over the place, and the more community or indie or niche it is, the more welcomed is the post-as-advertisement or comment-as-advertisement. And that's not even going into corporate efforts to operate on the platform, which can be very subtle.
> However, once Reddit creates a search engine, and once people get to know that there is an opportunity to game the system and create a financial opportunity,
Perhaps--but Google was _really_ good for a very long time. That was worth a lot to millions of people.
I like to think that in the future, an ecosystem-mindset toward incentive systems could really help, like planning for different emergent systems that incentivize good results.
> I would offer a bit of a different perspective here, I think maybe even most do it for what you could call self-soothing.
>
> For example, a lot of people share knowledge, but it's _their knowledge_, which can also be defined as _sharing their subjective past_ and that's known to be a very soothing thing for people who may even be troubled by the unknown in their personal lives. Hop online, boom, you're a domain expert in your own past. Find a place that's _about your past_, be it /r/linux or /r/formula1 or whatever, and there you go, it's comfy. You generally get an upvote + reply vibes dopamine bonus just for being there, due to your relevant past.
That's a bit too psycho-analytical. A lot of what gets shared on technical/domain-expertise subs are simply technical knowledge. Most posts on such subs are questions, and upvoted comments are typically answers. If you accuse people of feeling good about helping others with their knowledge, well, I'd say that the joy is well-earned by the generous act.
In fact, if we are talking about the exhilaration of being upvoted, it's much easier to farm karma on reddit by lying. There are many big subs that are purely made up stories.
Then, I'll admit that I don't understand your point. Your tone seems to suggest that there is a better alternative than putting someone's past experience as part of their argument. What might that be?
This is exactly right: if Reddit ever decided to build a search engine they'd wind up destroying their site, because everyone would try to game the results and fill Reddit with crap to do it.
The point it is that search engines are adversarial: people who are skilled and determined are actively trying to replace good results with their results.
yes, however there is an element of real feedback: if an subreddit turns out to be garbage, then people will choose a different subreddit with a different moderation policy. It is possible to prioritize by number of point or by some dynamics over the popularity of an subreddit - that would incorporate this feedback.
one advantage that reddit has is that its content is community moderated. People may have an incentive to game search, but communities have an incentive to keep the content authentic. Not to mention that Reddit as a platform also has some means to restrict spam accounts.
Google if it does this sort of clean-up at all has to do so algorithmically which is kind of shoddy. Reddit basically gets a human workforce for free, so I'm not so sure the comparison holds.
Their site has been thoroughly trashed already. It's an absolute cesspool of bots and groupthink. When I joined back in 2007, it was a completely different site full of actual discussion and not just cringey self deprecating nerd jokes and memes.
It's a shame, but that's how things go. Once a thing gets that big, it gets its soul sucked out.
It's true, early reddit was rather similar to HN actually. I can even pinpoint one of the cataclysmic events, reddit's "Eternal September" so to speak, where things accelerated downhill: When digg 4.0 was rolled out and failed spectacularly, leading to an influx of digg users to reddit.
Prior to that I checked digg out every once in a while and got immediately annoyed of all the "cringey self deprecating nerd jokes and memes". And suddenly they were everywhere on reddit.
Your experience on reddit is 100% the result of the subreddits to which you subscribe. Unsub ALL default reddits and you'll be surprised at how quickly the quality goes up.
Yeah, I unsubbed the main subreddits years ago. I only stuck around for the tech stuff. And as they all grow bigger they turn into meme fests, "hey guys can you be Google for me?", stupid one liner in-jokes, and lowest common denominator shit posting.
Avoid r/Canada just a toxic community where mods delete anything that offends them and goes against their beliefs. I honestly think it is also overrun by bots trying to steer an narrative. Everyone just argues and fights now it seems.
I think a lot of the Canada-based subreddits are so political and group-thinky, for lack of a better term, that they are better off avoided at this point. /r/Ontario is the most glaringly obvious example of an echo chamber that I see no point in ever checking it, even though I live in Ontario. My small town's subreddit is decent though, just for the odd news stories every now and then.
Best advice for reddit I can give is to stick to small, focused subreddits. In my case I only visit gaming/tech/dev/podcast related subreddits. The site is only usable like this now, imo.
The idea of reddit building competent software is hard to take seriously. I am appalled every time I try to buy an ad. Those folks have no idea what they are doing. Rank amateurs.
There is a Mark Zuckerberg quote about twitter, something like "Twitter is a clown car that fell into a gold mine." I often think that about reddit, minus the gold mine.
But the practice of appending 'reddit' to your Google search isn't depending on Reddit building a better search engine. It's using Google's search engine, and trying to direct it to a database that you hope has the answer you're looking for. So there's that. Of course, if that practice became widespread, then people could still try to stuff Reddit posts with junk in the hope that it gets picked up by Google in these searches, and then the Reddit folks would have to try to filter that out.
There was a post, which trended on HN yesterday (2000+ upvotes) that stated Reddit is sitting on a gold mine and they can build a better search engine because people are appending 'reddit' at the end of it.
This is a response to the original post, explaining why the logic behind is it false.
Frankly I don't really see the logic. You're failing to disambiguate "generally better" with "better for Reddit inc" or "better for redditors," which are all separate things. That is, I'm not convinced that a Reddit search engine would necessarily be subject to the same level of gaming, for a variety of reasons:
* Reddit groups are moderated in a way that "THE INTERNET" as a whole isn't.
* Reddit is a destination in itself. It's not like Google, which is a means to arrive at various destinations.
* A Reddit search engine just has to index Reddit. It can do things like entirely filter out clickable external links, which would make SEO much less fruitful.
* Reddit is dependent on being an effective conversational watercooler. For financial reasons, it's not going to allow itself to become filled with the same level of cruft as the entire web is.
This is somewhat evidenced by the difference between Reddit and Usenet. Compared with how Usenet quickly devolved into an unmanageable morass with the advent of spam, Reddit has managed to stay fairly legible.
(BTW off topic but personally I think this whole post could've just been a comment subthread in the original post. Of course we all want our ideas to bubble to the top. But if everybody started a whole new thread every time they thought they had something REALLY IMPORTANT to say that shouldn't be ignored, then every large thread would give birth to dozens of baby threads and make HN less effective overall. On the other hand, I see someone else thanked you for putting this in a separate "breakout room" so to speak, so what do I know? Cheers.)
@rukshn Thanks for your post. Personally I'm not convinced by the idea of adding site:reddit.com to your Google search query either, not just because it will get gamed by SEO practitioners as you suggest, but also because there's a lot more to the internet than reddit and a lot of great content hidden away which isn't on reddit. I also believe that a more general solution to better search is both to reward informational sites and also to penalise advertising sites - that's why I built a search engine with a PageRank-style algorthim which also detects result pages containing adverts and heavily downranks them.
> people could still try to stuff Reddit posts with junk in the hope that it gets picked up by Google in these searches
This is already happening. I search for reviews and help on Reddit via Google often, and it's not that uncommon for the first result to be SEO spam: a Reddit post with no votes or comments, just lots of keywords and a link to a product or another site.
> I was too young to use Google when it first got started, but according to many, in the beginning, Google had better or more accurate search results. One reason for this was the fact that most of the sites back then were indie websites, which did not care about SEO and money and created websites for sharing information.
> But I think all can agree that Google results reduced when people started to game the algorithm and create search engine optimized garbage websites.
Well not quite.
There were piles of SEO crap sites being produced in 2001. Google used to manage to react to any new exploit and improve their results. They actively fought that battle for at least a decade or so. They used to win.
It is only more recently that they seem to have given up.
>It is only more recently that they seem to have given up.
They haven't given up; the OP has a point. The "sites" you are hoping for Google to return _don't exist_. Any website online right now that doesn't exist to drive ad revenue is exceedingly rare. In 2001, there were way more websites that existed just for fun; any tom, dick and harry could open up note pad and get a website online. That doesn't exist anymore.
It's my opinion that those who complain about Google search results are frustrated that Google can no longer find a web that no longer exists
No. There still are forums on lots of topics. But search how to fix X and instead of prioritizing a forum site with answers you get 5 top results trying to sell you the parts to fix X. The answers are always way down the list.
What is Reddit if not the largest forum in the world? You have the same gripe; you are looking for a web where people share things for free (like in forum); and when Google first existed most of the web is like that.
And if google were to prioritize forums; then SEO blackhats would do everything in their power to make every website look like a forum; and then you are back to square one. The problem is the profit motive, not Google's algorithms.
Yes but google crawls the internet it would be pretty trivial for it to scan a site and realize it was full of affiliate links and trying to sell things. They could rank those sites lower then just a site with text and not trying to sell everything.
Explain away how literally verbatim searching with quotes doesn't return verbatim results. Or how searches that match the very title of an article returns mountains of SEO spam before listing the article, if it shows up at all.
If there's an algorithm being gamed that causes that, that algorithm was designed explicitly to be gamed.
It does exist, just the same - probably more so, because the web as a whole is many orders of magnitude large now, so there's just more of everything.
However, 90% of everything is crap, so the good stuff is not buried under thousands, or millions, or mere billions of pages of crap, but trillions.
> any tom, dick and harry could open up note pad and get a website online.
This part hasn't changed one iota - you can still do _exactly_ that, even following a tutorial from 1996 to the letter, if you want. All that stuff still works fine - and I have websites that old, built that way, that are still up and work fine.
Google intentionally finds synonyms for obscure jargon that would have typically turned up these sites. The verbatim setting barely makes a difference.
The reliance on synonyms and semantically related topics has made iterative refining impossible as well, it's essentially non-converging.
My solution has been to search in multiple foreign languages, which looks less conducive to these "affordances".
Yeah, having spent a lot of time working with dense vector based techniques, especially in the context of a domain which is really similar to semantic search, I can say 100% that I don't want BERT and related techniques anywhere near my google search. That's the real crux of the problem, is the reliance on these sorts of models, and their utilization within query rewriting schemes.
> It's my opinion that those who complain about Google search results are frustrated that Google can no longer find a web that no longer exists
You ever try to search for a niche piece of software for your computer? Google surfaces the "Top 7 Best software" content farm articles before you ever get to the official sites. That's what some of us are frustrated about.
Have you ever searched for a recipe in Google? Have you noticed that the top results always go on for a few pages of SEO-optimized storytelling and explanation before you actually see the recipe?
I'm sure someone at Google has. They could fix it if they wanted to, and boost well-known sites that get straight to the recipe (allrecipes.com, foodnetwork.com) instead of making you scroll through the SEO spam first. They haven't.
I used to think this too but if all the recipe websites start going straight to the recipe without the the personalization, then how would you rank them? Remember, there are thousands of these websites while there are only so many ways you can boil an egg, for example.
PageRank-like system, ideally based on real identities? (i.e. depending on how often the recipe is linked from Facebook, etc.)
Also if you are specifically doing it for recipes, just select manually websites that have precise ingredient amounts, very detailed and explicit recipes, that explain the reasoning behind each step, provide photos of how things should and shouldn't look like, why the proposed technique is optimal and/or the tradeoff of alternative techniques, the chemistry involved, comparison with other recipes, etc. (although in my experience most or all recipe sites are horrible and aren't even vaguely close to something like this)
Back in 2001 advertising was far more lucrative than it was today. So a website could be both: useful, and a for-profit, ad-supported enterprise. Hell, the reason why most online versions of newspapers were free was because advertising was so lucrative. Now, ads pay peanuts, and only make sense if you already have ludicrous scale, or are able to game search results with low-effort """content""" to get that scale.[0]
So it's both "less signal" (fewer high-quality websites) and "more noise" (more spammy SEO-to-oblivion sites).
[0] A similar thing happened to video advertising, which is why even really good YouTube videos are filled to the brim with ad integrations, merch tie-ins, and calls to like and subscribe.
Sure they do. It is just in 2001, there was 30M websites, and let's say 5% of them were good. And in 2020, there is 2000M websites... and the number of good ones is about the same (because there are only so many people willing to spend time making personal website). So the fraction is much lower now.
It is easier than ever to open your websites, and there are tons of free providers today. Look at any manual aggregator -- like hacker news or hackaday.com -- to see them. This is a variety I have not had in 2001!
Part of the problem is that Google started heavily favoring the age of a domain about a decade or so ago. If you spin up a new site, with high quality content, you simply won’t show up in Google for a long time.
Also, the amount of good content today is vastly larger than it was in 2001. Though you are right about the ratio changing.
Example. If you searched the word "flight" in 2002, the first two results would be NASA links and the third would be the Museum of Flight. These pages still exist. But now the entire first page of Google search results are cost comparison websites. This is Google's badness, not the web's.
I mean, you do you but when I search flight I for one am looking to purchase a flight. If I want NASA links or the Museum of Flight I would specify that.
Alternatively, I could have searched for "flight comparisons" or "US airlines" back in 2002 if that's the type of result that I actually wanted.
The problem is that there is no way I can tell current Google to give me stuff that's interesting, novel and information dense anymore. It's all well and good if I knew that the Museum of Flight existed before my search, but the point of Google is to tell me about its existence. I can't tell Google "flight ... but no cost comparison or airliner website stuff"; such a query is currently not possible.
> demand for this is very low and implementation cost is extremely high.
Demand is real as evidenced by everyone in this thread wanting such a search engine. We are not representative of genpop, but we aren't insignificant either.
Google had this in 2002 (i.e. a default to novel and info dense results unless otherwise specified), so it can't be that hard or expensive.
> They haven't given up; the OP has a point. The "sites" you are hoping for Google to return _don't exist_.
I was beginning to wonder if this explanation was true but after starting to use Marginalia (mostly for fun) I now know the old web still exists - Google just doesn't let me see it.
Because its simply not possible. There are too many barriers now. You need to get a static ip for one. Domains cost more. Email for those domains costs quite a bit. And your friends are far less likely to visit since theyre all addicted to infinite scrolling websites
They have given up, as in if you Google anything technical these days you'll get tons of spam sites who just copies the content from GitHub and StackOverflow. Google used to punish sites that just copied content from other sites, but that isn't the case anymore.
When it came out, PageRank was a really innovative way to order search results that cut through most of the "SEO" at the time, which was webpages doing lots of unsophisticated things to range well for a given topic (e.g., putting that topic in the <title> many times).
Yeah. PageRank worked because web site owners were linking to each other as a service to the "Surfers", signaling what they think are high quality sites. There is an incentive to keep sites high quality or it'll risk being delinked.
When people stopped "surfing" and starting "searching", the whole mechanism broke down. Plus Google started using other metrics for ranking sites these days.
SEO before Google PageRank was basically submitting a site multiple times in an online form to Altavista and it would rank higher. As a teenager I automated this with the only programming language I knew, mIRC scripts.
Before the rise of smartphones a competitor could have emerged and take users from Google. Yahoo wasn't completely dead yet. Platforms were mostly open, a sea change could have happened (though highly unlikely).
Into the smartphone area, other engines became anecdotal (bing just exists in the background), there is enough lock-in to not have to fear against direct competition, and the main focus becomes having users spend more time on the web, search more, instead of staying in native apps. That's were, if my memory is correct, they ditch language based landing pages and start fazing out modifier support and all the "power user" part of their interface.
Fighting SEO scams becomes a waste of time when it has no real impact on wether user will continue to use the product or not.
The real experts or people who create great content just for the sake of sharing information never care about backlinks and whatever else Google requires for SEO.. OTOH the SEO scammers like Pinterest create little content and give the search engine exactly what it wants (backlinks, social signals, etc) which is why the good content is so hard to find.
But at the end of the day it's the search engine's job to separate the wheat from the chaff and google has been doing a very poor job of it lately. Like the article right said, the best is to prioritise websites that are about sharing knowledge not affiliate marketing, paid ads, marketing, etc.
Reddit doesn't really need to build a search engine, they could just have an algolia instance like HN does: https://hn.algolia.com/.
While not perfect, it does a pretty good job, in my experience.
However, the article makes a valid point -- the moment people start using it heavily, we'll get a reddit comment equivalent of SEO, with keyword stuffing and whatnot.
I absolutely love the HN Algolia search. I am amazed by its sheer speed, and that it does exactly what you'd expect, lets you sort by popularity and filter by time, and does no neural-network-deep-learning-garbage query rewriting.
I'm thankful for its existence, and I hope they get plenty of clients due to this HN demo of their power.
I hope they can afford to keep it alive forever. I use it instead of the " reddit" suffix in Google, usually HN has what I'm interested in, and in any case has much higher signal-to-noise ratio.
Algolia was the first thing I though about when the post was made yesterday.
I commented this elsewhere but I don't think Reddit wants a good search option. Users tend to repost a non-trivial amount of content/questions/etc., and if those users could find the original content with a better search, it's likely reddit would see less user engagement.
> Reddit posts are good because the people who create these posts or make comments
> are doing it to share their knowledge. And there is no financial incentive
> associated with it. They know that do it to share their knowledge.
Ok, that's why people are Googling for "xyz reddit"...
> However, once Reddit creates a search engine, and once people get to know that
> there is an opportunity to game the system and create a financial opportunity,
> people will abuse that system and we will be back to the place where we are
> now. SEO stuffed websites.
There's nothing in this article explaining how the existence of better search on Reddit would provide different incentives than the very popular search engine for Reddit that is being used today (Google). What is this "financial opportunity" that doesn't already exist that will spell doom?
Marketer perspective: you can not build a communication tool you find useful that won't end up being used to make money. It will get figured out.
I could go into a bit more philosophical angle with reterritorialization done by capital, but I think it's much simpler to consider the following: between politics (where once any division in population is found, it becomes valuable instantly), classical business marketing (where once anyone makes purchasing decision informed by something, it becomes valuable), capital markets (where once anything can be used to predict anything about any company or asset class, it becomes valuable) and more personal scams (where once you figure out someone's niche interest, it becomes valuable) there just isn't anything left. Go ahead, try to find something.
Reddit is being constantly targeted. I still use it, because what else?, but if the method isn't obvious, here's what you do: you are hired by/own small company making niche potato chips dip sold via Amazon. You go to google keywords and check 'potato chips dip', you google all you can find there and some of your ideas, you write down all top10 results and check every now and then (well, your SEO monitoring app does it for you). Whatever you find that allows for user input, you generate that input - accounts are cheap - and maybe do some external SEO (thus beating 99,9% of social media results online).
That's it, it's easy. What Reddit (and any mildly aware SM company) does, is they try to offer marketers access to audience for a price that's lower than cost of what I just described. There will be edge cases, especially on international markets where value of time for various business owners differs vastly, which will lead to sites slowly getting more clogged up with ads, but that's the general gist. If you can imagine using similar method to get in front of your eyes when you're looking for something, then it would just be quite weird if nobody ever did it.
What if you explicitly target it with respect to the money?
Most of the crap on Google is going to be monetized sites. If you just block/downrank all the sites with e.g. ads, then the profit motive for spam goes down - the more value you extract, the less value you are able to extract.
It's an improvement, even though it's going to be gamed as well.
That said, I have no idea how you would do it. More specifically, 1) I see almost everything as an ad, 2) you're going to get an extremely biased internet and logistics/technicalities of the project are hellish.
1) you downgrade all blogs with ads. All journalism (if they advertise their own stuff that's still profit motive and ad, right?) and all company blogs. All social media (that have over 100mln users right now). Also all blogs that are meant to promote author as an authority to get them gigs down the road, I'm guessing. You will be left with more overt PR and
2) organizations who can afford to not have profit motive in any form. That's just unsustainable for almost everything. NGOs are still asking for donations. That's going to be mostly politically motivated messaging and tiny % of people who can afford to just post for free cause they like to (god forbid they lose that position and setup patreon).
Audience allows for attaching marketing, and that allows to attach all content production to business (and other cases I mentioned). Again, I like the idea in principal, but 'lets just get the money out of the system' was tried in many different contexts and the same structural issues resurface.
The only thing I noticed on SO is sometimes having question titled in a more general way, with details being actually focused on a specific new framework, then discussion and answer showing how amazing the framework is for this use case. Which is also the only thing I can reasonably imagine being promoted, but incentives here are so closely aligned I can't even be mad honestly.
I don't know what happens on 'front page'/'trending' of SO and other other stackexchanges.
EDIT: I checked stats and SO has <15mln users. Part of it being the way it is comes from being relatively tiny and I honestly can't guess how expensive moderation on SO is.
> However, once Reddit creates a search engine, and once people get to know that there is an opportunity to game the system and create a financial opportunity, people will abuse that system and we will be back to the place where we are now. SEO stuffed websites.
I help run a Reddit website and open source project for managing scheduled posts [1] and we see a lot of garbage there, sadly. Over the years the site has processed nearly half a million posts and has tens of thousands of registered Redditors. In the early days it was pretty cool, we had a lot of individual users with various interesting projects. I remember early on there were some musicians using it and a few book authors that would release their books one chapter at a time - using our service to schedule the posts so they didn't have to do it manually.
But, as time went on, the entire platform has been just swamped with mostly live sex workers and people shilling bullshit products. I'm not judging the sex workers or saying they shouldn't be allowed on the platform and we don't ban them, but I would say now literally 99/100 of new users are just trying to push their Only Fans accounts to porn subreddits. Regarding the weird product shills, we've seen a few really odd campaigns. There was one guy who would post to various left-leaning and "green" oriented subreddits on the terrible nature of plastics while also promoting his own metal straw sales. Again, none of it is illegal and I'm not even sure if any of it is unethical or immoral since it really depends on if you think people are doing these things in bad faith, etc.
Either way, the project has for sure taken a backseat for us. If the server stays up and everything works, cool, if not, that's fine too. I have no interest in trying to monetize the service anymore whereas in the early days we took donations.
My point is the world of gamifying and commercializing Reddit content is already very very well established and there are large players doing it at a very large scale. If you measure it the porn industry is virtually all of it, but if you exclude them and look a bit harder you'll find other industries that have also carved out their niche in exploiting the platform.
Accepting payment for this kind of service is actually quite problematic. Even more so now that the vast majority of the userbase is Only Fans creators. I'm not saying it should be like that, but if you've ever tried to accept payments for anything "high risk" it's a nightmare.
For example, Stripe won't allow any of it at all. It's blacklisted in multiple ways through their restricted services.
This is already happening. Advertisers create normal looking fake accounts, and subtly inject product placements into discussions.
The best way to market a thing is to not make people realize they are being marketed with.
I remember a few years ago, there was a post (AMA) by someone who does this and his whole job was to maintain hundreds of normal looking accounts and post SEO friendly stuff, good words about products, etc on reddit.
I find that most people on this site use it to engage in discussion, to learn and to teach if they think they have something of value to say. For the most part, you're right about Reddit these days. When I used that site I'd check votes every time I was done scrolling. This site I check threads to see if anyone responded to me with anything I'd like to reply to.
Although this article has many valid points, it's important to say that Reddit is mostly used by english speakers. (Maybe 1/3 of the world, i think that probably less). I'm a software developer and i think that most of the time i prefer websites like stack overflow or github issues directly.
As a software developer i see that reddit answers most questions like "Is it better to use A over B?", "Why using C can be bad for my code?". It's for questions that are more open and less exact or technical.
So yeah, i don't see reddit being used as a search engine because english speakers are not even half of the world and it definitely don't answers most daily basis questions.
This thesis is off. It's not as if Reddit just appeared on the internet yesterday. It's been a place people look for this kind of info for a long time, and people have been trying to game it for just as long. And it already has a good search engine - that just happens to be Google. So it still leaves the question: why does Reddit still have such a high signal-to-noise ratio.
A few things I've noticed:
1. A lot of conversations happen before that many people are interested. Subreddits tend to attract mavens, and they often discuss things months or years before people really care (or the marketing team for whatever is being discussed is even looped in). Pay attention to when the posts you're looking at occurred. In a lot of cases, they were there earlier than you'd expect.
2. There is incredible dispersion in where conversations on a topic occur. It's not uncommon to have 10s or 100s of different communities discussing the same thing, and its not clear which is going to end up being the place people trust. Many of the sub-communities are also somewhat mutually exclusive(geography, android vs ios, etc), meaning it's going to look incredibly insincere if the same account is posting in a bunch of them.
3. Reddit posts allow negative feedback in a way few other venues due, especially not pages optimized for SEO and controlled by a single entity.
4. It is one of the few platforms with an appetite for long-form content. It is almost an anti-Twitter. Meaningfully moving a Reddit discussion on a single popular post could take hours if it could be done at all. For communities with more lasting artifacts like a wiki, it could be practically impossible.
5. As others have pointed out, the subreddits aren't controlled by and don't have the same incentives as Reddit Inc. Optimization tools aren't going to generalize well since what it takes to get to the top of each is different.
Closed communities is where it's at... But they have the obvious inherent problem: they don't scale. If a closed community lets in too many people, it becomes a somewhat obscured open community, with all the downsides. And if it doesn't, well, you are unlikely to get in on a topic you are interested in.
That's something I've been grappling with recently, and the solution I landed on as "good enough" (though far from perfect) is to have a community where the ability to post and interact is closed, but the ability to view the content is open.
Some subreddits have tried a variation of this, such as the Economics subreddit granting a "Bureau Member" flair for individuals who have proven their knowledge of economics in long form posts, but without taking it farther than just extra flair. There's no weekly threads for Bureau Members only to post, or anything like that.
The issue then becomes how do you set up a judging body that determines the criteria for who is allowed to interact, and prevent that body from being co-opted in some way. Not an easy task, especially for a social media company where there's absolutely perverse incentives at play.
Yeah initially I thought Reddit was useful for "give an opinion" type queries, especially where you're getting answers to a newly posted query. Eg my brother managed to get a specialist doctor to give an opinion about our mom's treatment.
But for the vast majority of commercially relevant searches, it will get sunk quite soon. It can't be terribly hard to hook up a GPT style bot to harvest karma on various accounts and then use that to gain credibility on "which headphones should I buy" type queries. You might not even need to automate it, a farm of real people could potentially be useful for that, I bet that's offered by someone out there already.
> It can't be terribly hard to hook up a GPT style bot to harvest karma on various accounts
Or more easily, steal and repost images on cute animal subreddits, sometimes adding white borders to the bottom right so duplicate image detection algorithms can't find it.
I can second this. Looking for a Bluetooth speaker and if I believed Reddit I am throwing money away if I’m not buying Anker. I’m sure it’s fine but it’s disproportionately represented in Reddit over other audiophile forums
I really like my Anker Bluetooth speaker, it's an old one that just keeps on working, lasted longer than Android phones. Listening to it now, every morning - the Anker love is real. ;)
If you need something cheap that lasts, Anker is your best bet. There are others that have longer lifetime, but they are from Sony, Bose or JBL and thus more expensive. My sister has the Soundcore for 2 years now.
Have had mine for several, don't use it as much as I used to because I prefer the sound signature of my phone speaker for spoken content but still love it for music. Reccomended it to my extended family and still get thrown for a loop when I see them using it all the time, like a piece of my home in their's.
This problem will only get worse as language models get better, making it cheaper to flood a forum with bot comments that look more and more like real humans. Moderators and users alike will spend far more time and resources weeding that stuff out, while it takes far fewer resources to post that stuff in large volumes.
The only way to counter that is to have some kind of account verification requiring 1 account per person. Such schemes are all either very unpopular ("Why does reddit need my passport photo!") or very ineffective ("So I just need a new IP and I can sign up for a new account? Great, I'll set up 1000 accounts")..
Yes, Reddit is being gamed. The question as with any system is: To what extend and how does it compare to alternative systems?
The capital that is most relevant on Reddit is status. It is very easy to have people participate in Reddit, which makes it inviting enough to start building status but it's somewhat harder to actually do. Users care about their status and about their communities and Reddit combines the two in a nifty way, that already gives provides some slid resistance against gamification.
Sure, you can infiltrate the system with any/or a lot of accounts, careful VPNing, lucky/clever reposting and also keep the guise up (you must, since your portfolio of both content and accounts is under constant review, as long as it remains visible). To get a highish ranking article on google I mostly have to write what feels like convoluted infomercials (or maybe let GPT3 do it for me). There is an industry of people being paid to do so. All it takes is one mediocre writer and some time.
To me, the former system seems very brittle and expensive to game, the later much less so, specially since the prevalence of convoluted writing for SEO purposes make the distinction between SEO bullshit and expansive content increasingly difficult: People are getting used to informercial type content on the web and so it is increasingly what they learn to expect. And we give the people what they apparently want.
I still think if you are internet savvy you detect these things almost instantly. Uncanny valley works on text too and the revulsion is just as strong.
And all it takes to destroy your best cooling for cpu now bot setup is one user saying “No it is not and here is why…” right under the comment.
That might happen if the product being pushed is awful. But if the product meets a baseline – not amazing, but perfectly cromulent – then a positive comment might not get such a rebuke.
I guess it's not the end of the world if you get astroturfed but you still get a decent enough product, but still, it's not ideal.
I don't think that's a particularly new thing. I used to be a very active reddit user, but I haven't been active on the site for 2-3 years now. But even 3+ years ago I recall plenty of (granted, anecdotal) reports of commercial astroturfing even for relatively nice subreddits.
I mean the example given is something about a car review. If it wasn't for reddit, it's likely the search query would be suffixed with a community name (like traditional forums), or a (to the searcher) known car review site. That said, the difference is that reddit has forums for everything, so you can expect to find a community about cars on there; if you're a noob at cars, you probably don't know which communities there are, what they're called, or if they're reliable.
The antidote outside of ruthlessly moderated special-interest "walled gardens" is a system no one really wants I think. It would need to avoid mechanisms that give community or interest groups powers to promote some content over others. It would need to avoid algorithms that automatically determine and recommend certain content. Other than pruning illegal contributions and obvious spam, it would need to avoid giving virtually any moderation powers to select individuals or the community. It couldn't allow paid promotion or require payment to participate as that merely enables those with greater financial means to contribute and have their content seen.
Introduce any one of these things and the whole system becomes gameable. Within these constraints it seems the outcome would be a loose graph of content tied to known individuals. It would grow amorphously over time, be largely undirected, and lacking the capability of being fully gamed, would probably avoid the attention of marketers and influencers. And probably general users too, since it wouldn't be easy to find interesting and relevant stuff and be quite boring for the average attention span.
The constraints bring to mind the early 90s web which basically had all of these properties. Outside of bulletin boards and UseNet that is. People wanted more convenience, hence search engines became the defacto way of finding entry points into these networks, which caused changes in how these networks presented and arranged themselves, point being even if we designed the perfect ungameable system, as you alluded to if any search engine or aggregator crawled and ranked its content, it would just be meta-gamed into uselessness.
Personally I've come to the conclusion we just have to accept that for the majority of online community-driven content, high SNR is here to stay, and we have to take what value we can individually as it comes. Unless we're willing to forgo conveniences like search, or relevant content recommendations, which I think for the majority of the web is a big old no.
For enthusiasts, building their own walled gardens with in-tune moderation is probably the way to go. Hacker News I think though not perfect is a good example of this, the SNR here is fairly decent compared to the broader web.
I think it's just that Reddit threads the needle on openness and moderation.
Subreddits are moderated by the subreddit community. Well moderated subs attract and produce good content. Multiple subs on the same topic can exist, often a poorly moderated sub will be replaced by a better one.
The payoff of doing SEO probably wouldn't be anywhere near what it is for Google search. I suspect many reddit users don't know it has a search feature, or tried it exactly once. The basic usage is scrolling through posts. No matter how good their search becomes, the thing you'd want to optimize is presumably be getting onto people's front page.
I've said it before and I'll say it again... I think building better search goes against everything Reddit wants to be. Reddit doesn't want to be Google or Wikipedia. Reddit wants to be a mix between Discord, TikTok, and Instagram.
The redesign they did years ago pushed a more Instagram like design. Things like chat and rPan are meant to keep you on the site interacting with other users in real time. Their new video player is just a TikTok rip off. They give away awards every day and started pushing award karma in hopes that you'll give them to other users. Reddit wants you on their site, interacting with other users in real time and building a better search goes against that goal.
If I'm looking for a new pair of boots, there's two paths I can take on the reddit site. I can search "best boots" or I can go make a post on /r/boots. If their search is fantastic and I find a post that's 2 years old, there may be some great information there, but I'm probably not going to comment. Even if I do comment, I'm probably not going to get a lot of responses. If their search is shit (intentionally or unintentionally) it pushes me toward making my own post and drives up their engagement metrics.
It would be a massive undertaking to improve their search and there's very little incentive. Reddit wants users who just want to browse and that's not what the users coming in from a specific Google search are doing.
There is massive incentive, since Google earns over $50B per quarter selling ads to people who want to buy stuff. Targeting these queries effectively would be hugely more valuable than my current ad selection of increasingly desperate shitcoin/NFT shilling and pleas to buy Reddit gold.
It would be even more amazing if ads were stuck exclusively on search result pages. That would minimize the interruption while maximizing the reevance of targeted ads, with minimal tracking needs.
I disagree. Reddit already has more than enough information to effectively target users. Just because they aren't doing it well right now, doesn't mean they need a new approach.
The subreddits you spend your time on is basically just you classifying your interests for Reddit. I'd argue that knowing that someone spends 1-2 hours a day scrolling through hiking subreddits is just as much of an indicator of their interests that any combination of Google searches. Reddit basically has access to all the information that Google ties to get through 3rd party cookies, except they get it way easier since you're never leaving their platform. They know where you're going, how long you're there, and even if you like or dislike the content (upvotes and downvotes). I'm sure their targeting could be improved with search, but like I said in my first post, I think that's a lot of effort for a small improvement.
This is too dismissive. Reddit users and communities have history, links, upvotes etc, which is hugely valuable, and it should be possible to implement a PageRank-type system to separate the signal from the noise and surface the most valuable results. Now that noise is still considerable, thanks to karma farming etc (did you know you can buy and sell Reddit accounts?), but humans can usually tell apart the shills from the legit posters and automating this doesn't seem unsurmountable if Reddit cared. Which, oddly, they don't seem to.
This is a weird article to me. I somehow agree with the conclusion but disagree with all of its points.
Unless my memory is failing me (possible, it’s late), it straw mans the article it’s referencing by implying Reddit would want to do something like that anyway. It also fails to offer a compelling narrative for what makes Reddit good to me because frankly, and I feel that a lot of people seem to be overlooking this here too, Reddit has plenty of its own problems.
Reddit is owned by Condé Nast and is largely used as a content mill to prop up its existing properties. Reddit is dominated by the Pareto principle. 80% of its content is generated by a very small subset of users. Reddit doesn’t have the best track record for free expression. Many subreddits have crazy moderators that volunteer for some Machiavellian rush and use their power to stifle or twist the narrative of public sentiment. Reddit doesn’t want to kill the golden goose, so when this kind of controversy arises or when controversial subreddits crop up they kill it with fire to protect their investment. Lastly, Reddit users more than half a decade ago were already lambasting the amount of corporate shilling and guerilla advertising that takes place on Reddit daily. A lot of this is, in fact, orchestrated by those users I mentioned who produce 80% of the content because yes, they have discovered a financial incentive to do so.
I wouldn’t deny that Reddit has any sincere users any more than I would deny that we have any sincere users here (I’m certainly not getting paid for this), but I’ve been a bit perturbed by the amount of whitewashed, positive vibes Reddit seems to be getting around here lately. I stick Reddit on the end of search queries too, but I feel like I still wade into results with a similar level of skepticism to that which I carry around most of the rest of the web.
When I append "reddit" to a search term for something it's because I'm looking for the opinions of mavens. For example, I needed a flashlight recently and rather than just buy the first flashlight on Amazon I instead searched "flashlight reddit", found the subreddit for flashlight enthusiasts, scanned their wiki and posts about what to buy and then just bought what seemed like their top recommendation for my usecase. It took a few minutes more, but now, I don't just have a flashlight, I have a flashlight that people who care about flashlights think is a good flashlight.
I think the reason reddit building a better search engine wouldn't help them is that it's already pretty darn easy to just add "reddit" to a search and, at least for me personally, I don't do it that often. Typing "reddit" at the end of my search seems a lot easier and faster than going to reddit and typing my query into their search.
Also, contrary to the blog post, I think people have always tried to game both search results and make money on reddit.
Google is an advertising company. So it makes sense that their search engine (and youtube) exist as a platform for advertising. It really is that simple.
Reddit don't want people to be able to easily find information. They want to make it just easy enough that you wont leave but hard enough that it will take you longer to find what you need. They are all about retention and finding that balance between aggravation and engagement. Twitter is the same.
288 comments
[ 0.25 ms ] story [ 253 ms ] threadIt gets worse if the content you're trying to index are labeled with the same titles like "me_irl". While they're usually just pointless memes, there are occasionally a few gems that I would like to find again to share with friends but it's basically impossible with such poor signals. I think it's a bit sad that those rare gems are essentially lost and forgotten.
> Reddit posts are good because the people who create these posts or make comments are doing it to share their knowledge.
I would offer a bit of a different perspective here, I think maybe even most do it for what you could call self-soothing.
For example, a lot of people share knowledge, but it's _their knowledge_, which can also be defined as _sharing their subjective past_ and that's known to be a very soothing thing for people who may even be troubled by the unknown in their personal lives. Hop online, boom, you're a domain expert in your own past. Find a place that's _about your past_, be it /r/linux or /r/formula1 or whatever, and there you go, it's comfy. You generally get an upvote + reply vibes dopamine bonus just for being there, due to your relevant past.
So, Reddit's sooth-sayers (so to speak) are kind of lazy but incentivized-lazy in some ways, which I think can also speak to some issues with the platform surrounding cognitive blind spots and what some here have called the echo chamber effect. That's an opportunity for a new search engine, in some ways, but it also may offer insights to new services that could be even more effective.
> And there is no financial incentive associated with it. [posting on Reddit]
There is though. I mean I've myself been financially incentivized into doing it, as have probably many, many others. Nothing sneaky, even.
You've got people posting their stuff all over the place, and the more community or indie or niche it is, the more welcomed is the post-as-advertisement or comment-as-advertisement. And that's not even going into corporate efforts to operate on the platform, which can be very subtle.
> However, once Reddit creates a search engine, and once people get to know that there is an opportunity to game the system and create a financial opportunity,
Perhaps--but Google was _really_ good for a very long time. That was worth a lot to millions of people.
I like to think that in the future, an ecosystem-mindset toward incentive systems could really help, like planning for different emergent systems that incentivize good results.
Anyway, fun to think about, thanks again.
That's a bit too psycho-analytical. A lot of what gets shared on technical/domain-expertise subs are simply technical knowledge. Most posts on such subs are questions, and upvoted comments are typically answers. If you accuse people of feeling good about helping others with their knowledge, well, I'd say that the joy is well-earned by the generous act.
In fact, if we are talking about the exhilaration of being upvoted, it's much easier to farm karma on reddit by lying. There are many big subs that are purely made up stories.
The point it is that search engines are adversarial: people who are skilled and determined are actively trying to replace good results with their results.
Google if it does this sort of clean-up at all has to do so algorithmically which is kind of shoddy. Reddit basically gets a human workforce for free, so I'm not so sure the comparison holds.
It's a shame, but that's how things go. Once a thing gets that big, it gets its soul sucked out.
Prior to that I checked digg out every once in a while and got immediately annoyed of all the "cringey self deprecating nerd jokes and memes". And suddenly they were everywhere on reddit.
Best advice for reddit I can give is to stick to small, focused subreddits. In my case I only visit gaming/tech/dev/podcast related subreddits. The site is only usable like this now, imo.
This is a response to the original post, explaining why the logic behind is it false.
* Reddit groups are moderated in a way that "THE INTERNET" as a whole isn't.
* Reddit is a destination in itself. It's not like Google, which is a means to arrive at various destinations.
* A Reddit search engine just has to index Reddit. It can do things like entirely filter out clickable external links, which would make SEO much less fruitful.
* Reddit is dependent on being an effective conversational watercooler. For financial reasons, it's not going to allow itself to become filled with the same level of cruft as the entire web is.
This is somewhat evidenced by the difference between Reddit and Usenet. Compared with how Usenet quickly devolved into an unmanageable morass with the advent of spam, Reddit has managed to stay fairly legible.
(BTW off topic but personally I think this whole post could've just been a comment subthread in the original post. Of course we all want our ideas to bubble to the top. But if everybody started a whole new thread every time they thought they had something REALLY IMPORTANT to say that shouldn't be ignored, then every large thread would give birth to dozens of baby threads and make HN less effective overall. On the other hand, I see someone else thanked you for putting this in a separate "breakout room" so to speak, so what do I know? Cheers.)
It's like a distributed reddit and individual site owners could agree to have their content pulled into a main site, eg via rss or meta tags.
Human moderation has value and there was a time when Google let users curate their results.
This is already happening. I search for reviews and help on Reddit via Google often, and it's not that uncommon for the first result to be SEO spam: a Reddit post with no votes or comments, just lots of keywords and a link to a product or another site.
> But I think all can agree that Google results reduced when people started to game the algorithm and create search engine optimized garbage websites.
Well not quite.
There were piles of SEO crap sites being produced in 2001. Google used to manage to react to any new exploit and improve their results. They actively fought that battle for at least a decade or so. They used to win.
It is only more recently that they seem to have given up.
They haven't given up; the OP has a point. The "sites" you are hoping for Google to return _don't exist_. Any website online right now that doesn't exist to drive ad revenue is exceedingly rare. In 2001, there were way more websites that existed just for fun; any tom, dick and harry could open up note pad and get a website online. That doesn't exist anymore.
It's my opinion that those who complain about Google search results are frustrated that Google can no longer find a web that no longer exists
What is Reddit if not the largest forum in the world? You have the same gripe; you are looking for a web where people share things for free (like in forum); and when Google first existed most of the web is like that.
And if google were to prioritize forums; then SEO blackhats would do everything in their power to make every website look like a forum; and then you are back to square one. The problem is the profit motive, not Google's algorithms.
If there's an algorithm being gamed that causes that, that algorithm was designed explicitly to be gamed.
That, and the Google search results are bad for the parts that do exist.
Still lots of great content still out there, often times the target of my original Google query.
However, 90% of everything is crap, so the good stuff is not buried under thousands, or millions, or mere billions of pages of crap, but trillions.
> any tom, dick and harry could open up note pad and get a website online.
This part hasn't changed one iota - you can still do _exactly_ that, even following a tutorial from 1996 to the letter, if you want. All that stuff still works fine - and I have websites that old, built that way, that are still up and work fine.
The reliance on synonyms and semantically related topics has made iterative refining impossible as well, it's essentially non-converging.
My solution has been to search in multiple foreign languages, which looks less conducive to these "affordances".
When the "No Search results" page was deleted, the internet died.
You ever try to search for a niche piece of software for your computer? Google surfaces the "Top 7 Best software" content farm articles before you ever get to the official sites. That's what some of us are frustrated about.
<Topic> sucks
RIP "sucks".
I'm sure someone at Google has. They could fix it if they wanted to, and boost well-known sites that get straight to the recipe (allrecipes.com, foodnetwork.com) instead of making you scroll through the SEO spam first. They haven't.
Also if you are specifically doing it for recipes, just select manually websites that have precise ingredient amounts, very detailed and explicit recipes, that explain the reasoning behind each step, provide photos of how things should and shouldn't look like, why the proposed technique is optimal and/or the tradeoff of alternative techniques, the chemistry involved, comparison with other recipes, etc. (although in my experience most or all recipe sites are horrible and aren't even vaguely close to something like this)
So it's both "less signal" (fewer high-quality websites) and "more noise" (more spammy SEO-to-oblivion sites).
[0] A similar thing happened to video advertising, which is why even really good YouTube videos are filled to the brim with ad integrations, merch tie-ins, and calls to like and subscribe.
It is easier than ever to open your websites, and there are tons of free providers today. Look at any manual aggregator -- like hacker news or hackaday.com -- to see them. This is a variety I have not had in 2001!
Also, the amount of good content today is vastly larger than it was in 2001. Though you are right about the ratio changing.
Example. If you searched the word "flight" in 2002, the first two results would be NASA links and the third would be the Museum of Flight. These pages still exist. But now the entire first page of Google search results are cost comparison websites. This is Google's badness, not the web's.
The problem is that there is no way I can tell current Google to give me stuff that's interesting, novel and information dense anymore. It's all well and good if I knew that the Museum of Flight existed before my search, but the point of Google is to tell me about its existence. I can't tell Google "flight ... but no cost comparison or airliner website stuff"; such a query is currently not possible.
Demand is real as evidenced by everyone in this thread wanting such a search engine. We are not representative of genpop, but we aren't insignificant either.
Google had this in 2002 (i.e. a default to novel and info dense results unless otherwise specified), so it can't be that hard or expensive.
> No one will pay for that.
So we have ads.
I was beginning to wonder if this explanation was true but after starting to use Marginalia (mostly for fun) I now know the old web still exists - Google just doesn't let me see it.
When people stopped "surfing" and starting "searching", the whole mechanism broke down. Plus Google started using other metrics for ranking sites these days.
Before the rise of smartphones a competitor could have emerged and take users from Google. Yahoo wasn't completely dead yet. Platforms were mostly open, a sea change could have happened (though highly unlikely).
Into the smartphone area, other engines became anecdotal (bing just exists in the background), there is enough lock-in to not have to fear against direct competition, and the main focus becomes having users spend more time on the web, search more, instead of staying in native apps. That's were, if my memory is correct, they ditch language based landing pages and start fazing out modifier support and all the "power user" part of their interface.
Fighting SEO scams becomes a waste of time when it has no real impact on wether user will continue to use the product or not.
But at the end of the day it's the search engine's job to separate the wheat from the chaff and google has been doing a very poor job of it lately. Like the article right said, the best is to prioritise websites that are about sharing knowledge not affiliate marketing, paid ads, marketing, etc.
This statement aged me 10 years
Reddit cant build any search functionality whatsoever. Current search is so broken It cant even find my own posts.
While not perfect, it does a pretty good job, in my experience.
However, the article makes a valid point -- the moment people start using it heavily, we'll get a reddit comment equivalent of SEO, with keyword stuffing and whatnot.
I'm thankful for its existence, and I hope they get plenty of clients due to this HN demo of their power.
I hope they can afford to keep it alive forever. I use it instead of the " reddit" suffix in Google, usually HN has what I'm interested in, and in any case has much higher signal-to-noise ratio.
I commented this elsewhere but I don't think Reddit wants a good search option. Users tend to repost a non-trivial amount of content/questions/etc., and if those users could find the original content with a better search, it's likely reddit would see less user engagement.
…
So how can we fix the system? The best way to fix the system is to prioritize websites that are there to share knowledge
OK, so append Reddit to the google search, got it.
> Reddit posts are good because the people who create these posts or make comments > are doing it to share their knowledge. And there is no financial incentive > associated with it. They know that do it to share their knowledge.
Ok, that's why people are Googling for "xyz reddit"...
> However, once Reddit creates a search engine, and once people get to know that > there is an opportunity to game the system and create a financial opportunity, > people will abuse that system and we will be back to the place where we are > now. SEO stuffed websites.
There's nothing in this article explaining how the existence of better search on Reddit would provide different incentives than the very popular search engine for Reddit that is being used today (Google). What is this "financial opportunity" that doesn't already exist that will spell doom?
I could go into a bit more philosophical angle with reterritorialization done by capital, but I think it's much simpler to consider the following: between politics (where once any division in population is found, it becomes valuable instantly), classical business marketing (where once anyone makes purchasing decision informed by something, it becomes valuable), capital markets (where once anything can be used to predict anything about any company or asset class, it becomes valuable) and more personal scams (where once you figure out someone's niche interest, it becomes valuable) there just isn't anything left. Go ahead, try to find something.
Reddit is being constantly targeted. I still use it, because what else?, but if the method isn't obvious, here's what you do: you are hired by/own small company making niche potato chips dip sold via Amazon. You go to google keywords and check 'potato chips dip', you google all you can find there and some of your ideas, you write down all top10 results and check every now and then (well, your SEO monitoring app does it for you). Whatever you find that allows for user input, you generate that input - accounts are cheap - and maybe do some external SEO (thus beating 99,9% of social media results online).
That's it, it's easy. What Reddit (and any mildly aware SM company) does, is they try to offer marketers access to audience for a price that's lower than cost of what I just described. There will be edge cases, especially on international markets where value of time for various business owners differs vastly, which will lead to sites slowly getting more clogged up with ads, but that's the general gist. If you can imagine using similar method to get in front of your eyes when you're looking for something, then it would just be quite weird if nobody ever did it.
Most of the crap on Google is going to be monetized sites. If you just block/downrank all the sites with e.g. ads, then the profit motive for spam goes down - the more value you extract, the less value you are able to extract.
That said, I have no idea how you would do it. More specifically, 1) I see almost everything as an ad, 2) you're going to get an extremely biased internet and logistics/technicalities of the project are hellish.
1) you downgrade all blogs with ads. All journalism (if they advertise their own stuff that's still profit motive and ad, right?) and all company blogs. All social media (that have over 100mln users right now). Also all blogs that are meant to promote author as an authority to get them gigs down the road, I'm guessing. You will be left with more overt PR and
2) organizations who can afford to not have profit motive in any form. That's just unsustainable for almost everything. NGOs are still asking for donations. That's going to be mostly politically motivated messaging and tiny % of people who can afford to just post for free cause they like to (god forbid they lose that position and setup patreon).
Audience allows for attaching marketing, and that allows to attach all content production to business (and other cases I mentioned). Again, I like the idea in principal, but 'lets just get the money out of the system' was tried in many different contexts and the same structural issues resurface.
I don't know what happens on 'front page'/'trending' of SO and other other stackexchanges.
EDIT: I checked stats and SO has <15mln users. Part of it being the way it is comes from being relatively tiny and I honestly can't guess how expensive moderation on SO is.
I help run a Reddit website and open source project for managing scheduled posts [1] and we see a lot of garbage there, sadly. Over the years the site has processed nearly half a million posts and has tens of thousands of registered Redditors. In the early days it was pretty cool, we had a lot of individual users with various interesting projects. I remember early on there were some musicians using it and a few book authors that would release their books one chapter at a time - using our service to schedule the posts so they didn't have to do it manually.
But, as time went on, the entire platform has been just swamped with mostly live sex workers and people shilling bullshit products. I'm not judging the sex workers or saying they shouldn't be allowed on the platform and we don't ban them, but I would say now literally 99/100 of new users are just trying to push their Only Fans accounts to porn subreddits. Regarding the weird product shills, we've seen a few really odd campaigns. There was one guy who would post to various left-leaning and "green" oriented subreddits on the terrible nature of plastics while also promoting his own metal straw sales. Again, none of it is illegal and I'm not even sure if any of it is unethical or immoral since it really depends on if you think people are doing these things in bad faith, etc.
Either way, the project has for sure taken a backseat for us. If the server stays up and everything works, cool, if not, that's fine too. I have no interest in trying to monetize the service anymore whereas in the early days we took donations.
My point is the world of gamifying and commercializing Reddit content is already very very well established and there are large players doing it at a very large scale. If you measure it the porn industry is virtually all of it, but if you exclude them and look a bit harder you'll find other industries that have also carved out their niche in exploiting the platform.
[1]: https://cronnit.com
For example, Stripe won't allow any of it at all. It's blacklisted in multiple ways through their restricted services.
The best way to market a thing is to not make people realize they are being marketed with.
I remember a few years ago, there was a post (AMA) by someone who does this and his whole job was to maintain hundreds of normal looking accounts and post SEO friendly stuff, good words about products, etc on reddit.
people post to Reddit (and here) not to share their knowledge but because they get a little squirt of dopamine from upvotes and downvotes and replies.
As a software developer i see that reddit answers most questions like "Is it better to use A over B?", "Why using C can be bad for my code?". It's for questions that are more open and less exact or technical.
So yeah, i don't see reddit being used as a search engine because english speakers are not even half of the world and it definitely don't answers most daily basis questions.
A few things I've noticed:
1. A lot of conversations happen before that many people are interested. Subreddits tend to attract mavens, and they often discuss things months or years before people really care (or the marketing team for whatever is being discussed is even looped in). Pay attention to when the posts you're looking at occurred. In a lot of cases, they were there earlier than you'd expect.
2. There is incredible dispersion in where conversations on a topic occur. It's not uncommon to have 10s or 100s of different communities discussing the same thing, and its not clear which is going to end up being the place people trust. Many of the sub-communities are also somewhat mutually exclusive(geography, android vs ios, etc), meaning it's going to look incredibly insincere if the same account is posting in a bunch of them.
3. Reddit posts allow negative feedback in a way few other venues due, especially not pages optimized for SEO and controlled by a single entity.
4. It is one of the few platforms with an appetite for long-form content. It is almost an anti-Twitter. Meaningfully moving a Reddit discussion on a single popular post could take hours if it could be done at all. For communities with more lasting artifacts like a wiki, it could be practically impossible.
5. As others have pointed out, the subreddits aren't controlled by and don't have the same incentives as Reddit Inc. Optimization tools aren't going to generalize well since what it takes to get to the top of each is different.
Example:
„What is the cheapest best cooling for cpu now?”
All discussions point to product that more experienced users know its crap.
But still the wrong product is at the top with most (botted) upvotes.
Companies realized that ppl look for opinions on reddit -> that created demand for those services -> bots companies implemented new strategy.
Its getting very very hard to get honest expert opinion. Seems like the only way now is to belong to specific closed communities.
Some subreddits have tried a variation of this, such as the Economics subreddit granting a "Bureau Member" flair for individuals who have proven their knowledge of economics in long form posts, but without taking it farther than just extra flair. There's no weekly threads for Bureau Members only to post, or anything like that.
The issue then becomes how do you set up a judging body that determines the criteria for who is allowed to interact, and prevent that body from being co-opted in some way. Not an easy task, especially for a social media company where there's absolutely perverse incentives at play.
But for the vast majority of commercially relevant searches, it will get sunk quite soon. It can't be terribly hard to hook up a GPT style bot to harvest karma on various accounts and then use that to gain credibility on "which headphones should I buy" type queries. You might not even need to automate it, a farm of real people could potentially be useful for that, I bet that's offered by someone out there already.
Or more easily, steal and repost images on cute animal subreddits, sometimes adding white borders to the bottom right so duplicate image detection algorithms can't find it.
I used to be able to gauge real world opinions on reddit pretty accurately, i.e. be aware what was happening before the curve. That is gone now.
The only way to counter that is to have some kind of account verification requiring 1 account per person. Such schemes are all either very unpopular ("Why does reddit need my passport photo!") or very ineffective ("So I just need a new IP and I can sign up for a new account? Great, I'll set up 1000 accounts")..
The capital that is most relevant on Reddit is status. It is very easy to have people participate in Reddit, which makes it inviting enough to start building status but it's somewhat harder to actually do. Users care about their status and about their communities and Reddit combines the two in a nifty way, that already gives provides some slid resistance against gamification.
Sure, you can infiltrate the system with any/or a lot of accounts, careful VPNing, lucky/clever reposting and also keep the guise up (you must, since your portfolio of both content and accounts is under constant review, as long as it remains visible). To get a highish ranking article on google I mostly have to write what feels like convoluted infomercials (or maybe let GPT3 do it for me). There is an industry of people being paid to do so. All it takes is one mediocre writer and some time.
To me, the former system seems very brittle and expensive to game, the later much less so, specially since the prevalence of convoluted writing for SEO purposes make the distinction between SEO bullshit and expansive content increasingly difficult: People are getting used to informercial type content on the web and so it is increasingly what they learn to expect. And we give the people what they apparently want.
And all it takes to destroy your best cooling for cpu now bot setup is one user saying “No it is not and here is why…” right under the comment.
Now, some of it is so good, it takes longer to confirm than to find.
If there is enough value in a community people will exploit it.
I guess it's not the end of the world if you get astroturfed but you still get a decent enough product, but still, it's not ideal.
Introduce any one of these things and the whole system becomes gameable. Within these constraints it seems the outcome would be a loose graph of content tied to known individuals. It would grow amorphously over time, be largely undirected, and lacking the capability of being fully gamed, would probably avoid the attention of marketers and influencers. And probably general users too, since it wouldn't be easy to find interesting and relevant stuff and be quite boring for the average attention span.
The constraints bring to mind the early 90s web which basically had all of these properties. Outside of bulletin boards and UseNet that is. People wanted more convenience, hence search engines became the defacto way of finding entry points into these networks, which caused changes in how these networks presented and arranged themselves, point being even if we designed the perfect ungameable system, as you alluded to if any search engine or aggregator crawled and ranked its content, it would just be meta-gamed into uselessness.
Personally I've come to the conclusion we just have to accept that for the majority of online community-driven content, high SNR is here to stay, and we have to take what value we can individually as it comes. Unless we're willing to forgo conveniences like search, or relevant content recommendations, which I think for the majority of the web is a big old no.
For enthusiasts, building their own walled gardens with in-tune moderation is probably the way to go. Hacker News I think though not perfect is a good example of this, the SNR here is fairly decent compared to the broader web.
0. Unlike the majority of the web, Reddit is moderated by actual humans.
Subreddits are moderated by the subreddit community. Well moderated subs attract and produce good content. Multiple subs on the same topic can exist, often a poorly moderated sub will be replaced by a better one.
The redesign they did years ago pushed a more Instagram like design. Things like chat and rPan are meant to keep you on the site interacting with other users in real time. Their new video player is just a TikTok rip off. They give away awards every day and started pushing award karma in hopes that you'll give them to other users. Reddit wants you on their site, interacting with other users in real time and building a better search goes against that goal.
If I'm looking for a new pair of boots, there's two paths I can take on the reddit site. I can search "best boots" or I can go make a post on /r/boots. If their search is fantastic and I find a post that's 2 years old, there may be some great information there, but I'm probably not going to comment. Even if I do comment, I'm probably not going to get a lot of responses. If their search is shit (intentionally or unintentionally) it pushes me toward making my own post and drives up their engagement metrics.
It would be a massive undertaking to improve their search and there's very little incentive. Reddit wants users who just want to browse and that's not what the users coming in from a specific Google search are doing.
The subreddits you spend your time on is basically just you classifying your interests for Reddit. I'd argue that knowing that someone spends 1-2 hours a day scrolling through hiking subreddits is just as much of an indicator of their interests that any combination of Google searches. Reddit basically has access to all the information that Google ties to get through 3rd party cookies, except they get it way easier since you're never leaving their platform. They know where you're going, how long you're there, and even if you like or dislike the content (upvotes and downvotes). I'm sure their targeting could be improved with search, but like I said in my first post, I think that's a lot of effort for a small improvement.
It is vividly burned in my memory because I used to work for one of those trash website that polluted Google search results.
When Panda came out, I was like… what took you so long (also, time to get a new job).
For whatever reason, Google is too shy to pull that move again.
Just imagine, Pinterest could easily be obliterated in one random day.
Unless my memory is failing me (possible, it’s late), it straw mans the article it’s referencing by implying Reddit would want to do something like that anyway. It also fails to offer a compelling narrative for what makes Reddit good to me because frankly, and I feel that a lot of people seem to be overlooking this here too, Reddit has plenty of its own problems.
Reddit is owned by Condé Nast and is largely used as a content mill to prop up its existing properties. Reddit is dominated by the Pareto principle. 80% of its content is generated by a very small subset of users. Reddit doesn’t have the best track record for free expression. Many subreddits have crazy moderators that volunteer for some Machiavellian rush and use their power to stifle or twist the narrative of public sentiment. Reddit doesn’t want to kill the golden goose, so when this kind of controversy arises or when controversial subreddits crop up they kill it with fire to protect their investment. Lastly, Reddit users more than half a decade ago were already lambasting the amount of corporate shilling and guerilla advertising that takes place on Reddit daily. A lot of this is, in fact, orchestrated by those users I mentioned who produce 80% of the content because yes, they have discovered a financial incentive to do so.
I wouldn’t deny that Reddit has any sincere users any more than I would deny that we have any sincere users here (I’m certainly not getting paid for this), but I’ve been a bit perturbed by the amount of whitewashed, positive vibes Reddit seems to be getting around here lately. I stick Reddit on the end of search queries too, but I feel like I still wade into results with a similar level of skepticism to that which I carry around most of the rest of the web.
I think the reason reddit building a better search engine wouldn't help them is that it's already pretty darn easy to just add "reddit" to a search and, at least for me personally, I don't do it that often. Typing "reddit" at the end of my search seems a lot easier and faster than going to reddit and typing my query into their search.
Also, contrary to the blog post, I think people have always tried to game both search results and make money on reddit.
Reddit don't want people to be able to easily find information. They want to make it just easy enough that you wont leave but hard enough that it will take you longer to find what you need. They are all about retention and finding that balance between aggravation and engagement. Twitter is the same.