Out of curiosity I have no idea how to approach this. Are there indexes of websites available to the public that you are suggesting that can be searched via grep or some simple scripting?
Or are you suggesting writing a web crawler to build our own index?
Or are you suggesting finding curated links to sites?
There was a post a month or two ago that talked about the design of the Yahoo homepage back then, and looking at the link again, it's striking how much more pleasurable an experience it is.
I still have a paperback book that claimed to be a comprehensive, not curated list of internet sites. It didn't even come with electronic media as became customary with computer books later on.
i might be wrong but i didn't think OP was suggesting to re-index the web. that's a totally different beast from finding an answer to a question/problem or learning a new topic.
depending on what i am looking for there are a few avenues i would explore if google wasn't available. github, gitlab, & stackoverflow/stackexchange for code related stuff.
wikipedia for general topics. youtube for how-tos. twitter for the news. newsletters and podcasts for links to new articles.
I would say that using a website's own internal search is fair game, but good luck trying to find, for example, the link to a blog you forgot the name to. You can try Wikipedia, Youtube, StackOverflow, WaPo's article, but I doubt you'll ever make your way to that blog without the URL to it.
The funny thing is I never book mark. Most of the sites I visit I just remember the URL. On rare occasions when I can’t remember the URL, I’ll search for in on Google. Like most people, I use Google a lot when I’m looking for things I don’t already know, but not when going to my commonly used sites. In the early 2000s I developed a website for a local public radio station. One thing it thought me was the usefulness of “speakable URLs”. Since most pages I built would require someone on the air speaking the url, they had to be short and distinctive, easy to say, and easy to remember (since the listener wouldn’t be taking notes, they had to remember it long enough to get to a computer and type it in). That got me in the habit of just remembering the URL, which I still do to this day.
Web rings were common. If you visited a webpage on a particular topic, they would usually contain links to other related sites on a side bar. Dmoz was an old registry and I think Yahoo had a registry at one time as well.
Altavista was early. If you were on in 1996, that was only 5 years after the “birth” of the web. Before spiders, things were organized like Gopher. There were directories you could look up. Some pages kept archives of links.
Web crawlers were introduced in 1993. At that point, gopher probably had more content. Remember WAIS? All the libraries liked that protocol. Encarta was still selling like hot cakes.
> The web was supposed to forcefully challenge our opinions and push back, like a personal trainer who doesn't care how tired you say you are.
What does this even mean?
The web wasn't "supposed to" be anything. Though I'm not sure what magic search engine OP actually has in mind and how it's supposed to work.
Besides, one of the modern mysteries is that we're in the age of instant information yet you'll notice how many people will write up an entire comment online or bicker IRL instead of doing a cursory search. I don't think it's the internet creating human stupidity / laziness. Unfortunately we had that long before, and search engines simply try to show the best results with minimal context.
Also, I think discussion around tech would be much improved if we tried to come up with a better idea whenever we go through the trouble of complaining about something. Anyone can enumerate why things are suboptimal, and usually when you try to come up with alternatives, you find out it's just trade-offs with no ideal solution.
Trying to pitch an alternative solution (like how a search engine should work) helps drill down into real conversational bedrock that's much more interesting.
I support the author here. There was a promise of a marketplace of ideas, making the world a better place, that being interconnected would make the world smarter and democratize knowledge. All that may be true, it may be out there, but it's damn hard to find.
Google, Youtube, Reddit, and Facebook all prioritize freshness. Instead of being exposed to things outside our comfort zones, we take solace protected inside filter bubbles.
Instead of the best answer, what usually floats to the top is the most repeated, the most seod, the newest, the most politically correct.
Google's results are considerably worse than they were and part of that is google trying too hard to think what we want instead of guiding us to ask better questions.
> google trying too hard to think what we want instead of guiding us to ask better questions.
Can you explain what this means?
I see people saying this wrt to Duck Duck Go: "No, the results aren't worse, you just have to re-learn how to search with it" and all they really mean is that you need to stack more context into the search input which is strictly worse since you can already do that in Google if you need to put a finer point on your search.
And they mean that since DDG doesn't know "Elm" is a programming language most of the time, by "re-learning search" they seem to mean adding "lang" to the input. Where "re-learn" seems like a romantic way to phrase this obvious problem of the search engine needing more context. In the same way I had to "re-learn how to use a keyboard" when my crappy 2017 Macbook Pro keys started coming off.
> you need to stack more context into the search input which is strictly worse since you can already do that in Google if you need to put a finer point on your search
Hum, not exactly. Google will ignore extra context that doesn't match your profile. People coped very well when it didn't.
Said commenter, as they type a comment in a marketplace of ideas where people discuss fairly complex ideas.
The internet is a tool, not a solution. I absolutely disagree that it's hard to find for anyone looking for it. The issue is that most people aren't looking for it, and you can't force them.
A lot of people are just looking for entertainment, and that's perfectly fine. They spent their whole day working hard, come home, and now you want to force them to spend their night studying and discovering new ideas? That may be the internet you want but it's not what people want. The internet can be for more than one thing.
> Instead of the best answer, what usually floats to the top is the most repeated, the most seod, the newest, the most politically correct.
This happens quite often on Hacker News. One of the fastest and easiest ways to accumulate karma is to be the first to post something like "$hated_company has always been doing $horrible_thing, they need to change", which most people agree with, every time a popular story about them shows up. (Thankfully, usually only a couple of these appear in most discussions, and usually people don't spam this.)
> Said commenter, as they type a comment in a marketplace of ideas where people discuss fairly complex ideas.
Hacker News can be great for complex discussions, but it's not free of filter bubbles and echo chambers.
Right. I think Hacker News does pretty well for itself, and it shows: the users that accumulate karma the fastest usually have insightful comments, which is rarely true elsewhere.
Well, of course, that's tautologically true. The difference is that the most popular comments are more often to be insightful, and the people with the most karma have it because they post insightful things, rather than the type of comment I mentioned above. (Believe me, I would know if people were spamming those; there are only a handful of people that accumulate karma faster than I do, and they all do it "fairly".)
I consider both kinds of comments to be low-quality, and yes Hacker News has both. But the really nice comments aren’t just straightforward praise or criticism.
Eh. If I post something taking shots at Europeans before America wakes up (3-4am CST) I usually get down voted for a few hours before American commenters come on and exchange fire with the Euros.
Filter bubbles aren't inherently bad. For example, Google bubbling up news from my city makes sense, but if it were to show me small news from every single city on the planet, that would be insane. Similarly, I enjoy it when it gives me some article about Animal Crossing that I've been playing lately, but doesn't show me any sports news that I have zero interest in. There are just so many different interests out there that it's simply impossible to follow every single one, so having news bubbles absolutely makes sense.
When people talk negatively about bubbles, they generally focus on the couple cases where being in a bubble is harmful, such as when it comes to science, politics and so on, but ignore all the other times where a bubble is good.
I haven't found individual blogs to typically be the problem but commercial websites that focus on things like how-tos. This is especially true if I'm just looking for an answer to a question. I frequently just end up searching Reddit or better a specific subreddit.
Even reviews. Searching best TV brings CNET, techradar, digitaltrends, Tom's hardware rank above rtings. Almost every one of those sites has the same paragraph of mumbo jumbo next to a TV name. It's become an art. There's nothing wrong with the CNET or digitaltrends roundups, but they also don't really add to the conversation or offer any unique insight, do they cut clutter better than their competitors? Hell 3 of my top 10 are futureplc results (Tom's hardware, techradar, whathifi.) The same company has seod its way to 3/10 results for that search.
There's obviously a level of notoriety beyond just quality. I don't need the same article 3 times under 3 logos.
>Said commenter, as they type a comment in a marketplace of ideas where people discuss fairly complex ideas.
Hacker News is, in the most charitable interpretation, an act of charity by pg and YC to attempt to provide a forum for a specific subculture. It's obvious from the goal of the project: "things that hackers would find interesting".
But what most people get from the Internet is not a product of someone with effectively unlimited funds (and legendary software engineering ability) building a website to support their community. They get to choose between eyeball-extraction megasites or clunky off-the-shelf vBulletin or Wordpress clones run by underqualified volunteers. It certainly is nice to be among the target audience of Hacker News -- but what if I weren't?
I feel for you in empathizing for people's ability to just have what they want, but... I worry about it too. (I say all this with compassion and a non-confrontational tone.)
> The issue is that most people aren't looking for [a marketplace of ideas], and you can't force them
No? We can't? Or we shouldn't? Might that be our hyper-individualist culture constraining our imagination of what the right path (survival) might be?
Our evolution within terrestrial physical reality "forced" us to participate in well-calibrated local marketplaces of ideas, and our psychology evolved specifically so that we had a fine-tuned balance of what we subjectively "wanted" and what we found ourselves coming to believe, despite that initial-condition "want" -- dissenting views in a room have both a repulsion but also a very specific gravity -- a closeness that emerges amongst holders of opposing ideas, when these ideas have manifested and are walking around in human bodies within shared meatspace -- we start to empathize with holders of countering views that we're forced to share physical space with. We talk about empathy like it's feelings for the other pieces of meat, but it's perhaps better conceived as a kinship of one tight bundle of ideas for another. It's evolved and it's ancient and it's a very specific foraging strategy for 2D terrestrial creatures finding information/food under those constraints.
And now, we've designed systems that aren't nearly as clever and well-calibrated as our meatspace selves evolved to be. In the purely physical space we evolved for, we had to share space with people we probably disagreed with, and we developed unique tendencies based on the nature of living on a 2D terrestrial plane. Heck, we'd have different psychology favoured if we made it to this level of the great filter, but happened to evolve in the air (3D grid) or within a more one-dimensional environment or some hyperdimensional space.
Speaking of high-dimensional space: enter the internet. Might our prior foraging strategies and adaptations stacked onto our prior foraging strategies... might they fail now? Foraging strategies are informed by the math of the landscape. (search terms: optimal foraging strategies, radius of perception, levy walk, agent-based modeling, in the vein of [1]) Our psychology is tailored to adapting to physical reality on a plane, and the internet might totally fuck that up. (What is an internet bubble? Maybe it's just my stepping out of the 2D terrestrial grid and engaging through a hidden, non-spatial dimension with some foraging target I can sense near me?) It's like all places are piped into one another, outside physical reality. This isn't Kansas. It's the formation of a hyperdimensional object. It's no longer a 2D grid, and our predispositions and adaptations for navigating such a grid might drive us to extinction.
imho, we DO need to consider "forcing" (as a collective) changes in that infrastructure, or else our hitherto evolved psychology might just as well see us destroyed. So in the end, what people "want" is maybe not the highest principle to hold, because what we individually "want" might be tuned for a dying reality, and a prior foraging landscape.
Heh, anyhow... perhaps this is just a mad-cap rant from a technologist and failed biochemist <3
Yup. one thing I hate about google search is that it gives me what it thinks I want rather than what I'm actually asking for. Also, the "politically correct" bubble mask is in full effect.
Furthermore, it's so heavily weighted to only show results from sites with lots of SEO juice. I can't find what I'm looking for anymore.
There are some areas where google really excels and that's in technical searches. Those are really well done.
I don’t know about the “politically correct” comment, but the one thing I would ask of google if I could ask for something that they would deliver is to ALWAYS return results where ALL of my search terms are included. It doesn’t have to be verbatim, I’m good with them doing conjugations and maybe synonym matching, but if I add a qualifier to my search query, I didn’t add it because I don’t need it. This is my biggest frustration on the web. Lots of different search engines think it is better to return lots of unrelated garbage rather than returning no hits. No hits has actual meaning to me about my search, changing my search to return stuff I don’t want is just noise.
Just add '+' sign before each keyword- seems to be working. I tried a sequence of 10 or so random and made-up words to see if it will find anything.while it returns results, it does it on a shorter search and it also says at the top that no results were found for this query.Then I started removing one keyword at the time and return the search. Eventually it found a matching result and it contains all four words from my query.
Are you sure this is on Google? I just tried with a three word query, and it failed for me.[1]
If true, this would be big news, as the "+" qualifier on search terms stopped being obeyed about a decade ago with the launch of... whatever that now-forgotten Facebook competitor Google used to have. The workaround was to use quotes around individual words, which mostly worked, unless it didn't. But the combination of using quotes and "verbatim" (under Tools/All Results") almost always worked.
[1] The way it failed is sort of interesting. I randomly tried "+rhino +cereal +gm", and the first result didn't include the word "rhino", and considered "gm" to be a synonym for General Mills. Quotes around the individual terms seems to work for this query, even without verbatim.
Based on Google’s advanced operator page at one point, the addition of a plus sign to a search termed either prompted the lookup of Google+ pages or a blood type.
'+' was replaced by adding quotes around each search term.
>There was a promise of a marketplace of ideas, making the world a better place, that being interconnected would make the world smarter and democratize knowledge
Did anyone really believe this other than middle aged libertarian hackers from the 90s who kept repeating this in prayer like fashion?
The internet is, as the name suggests, a network. Google also is a sort of network, one that is semantically searchable. But Google Search is not a fact checker. It doesn't have an editorial room for every piece of content it exposes, it doesn't have deep knowledge about everything it throws back at us, and it could not have those things at the scale it operates.
That's not Google's fault either. It's not a search engine's job to correct wrong reporting. It's the job of journalists and news outlets to fact-check information, it's the job of citizens to be critical of the information they consume, and it's the job of governments to facilitate that this happens.
It's our fault that our civic institutions have degraded, and not the fault of Google that it spits our own stupidity back at us. It's not Google's fault that we don't like what we see when we look in the mirror and techno utopianism isn't going to save us either.
I think it’s mostly the average quality of the information available has regressed.
I’m fairly certain they’re doing the best they can with what’s available to them, but right now they probably get more accurate results by focusing on what is popular than by blindly accepting the contents of the websites.
That goes back to them upranking fresher results. If the old results were better, they should keep them at the top and ignore fresher but worse quality content. Google exists to rank content, and it should be by quality not age.
> There was a promise of a marketplace of ideas, making the world a better place, that being interconnected would make the world smarter and democratize knowledge.
I vividly remember the promise, and believing in that promise, but there was never a time that the Internet was this. In retrospect I don't see why we should have believed in it in the first place.
It's like sites are modifying their templates, and it refreshes the published date, and google goes off that page change date and not the date on the page. It's terrible and googles search by date is now nearly useless. Sometimes it helps filter out old results, but it is impossible to keep new results from bleeding into the past.
Politically correct is what people like, IMO, as measured by Disney products. I don’t believe Disney is politically correct as a matter of whim, as opposed to well measured brand strategy.
> being interconnected would make the world smarter and democratize knowledge. All that may be true, it may be out there, but it's damn hard to find.
Wikipedia does a damn fine job in my opinion. You can find decent, democratized knowledge on almost any topic complete with translation into your preferred language (if not on wiki, then through google translate). Or efforts like MIT's open course ware, which hosts lectures and syllabus materials on a variety of topics. There's also arxiv derivatives and a thriving black market (scihub, libgen, etc) of open access to research papers on any topic you'd like. Social media =/ the internet.
The funny part about democratizing knowledge is that it happens on the supply side too. In a knowledge democracy, knowledge comes not just from the knowledge elite, but from the knowledge poor as well! ;)
No but in all seriousness, the mistake was in relying so heavily on private corporations motivated by profit. An unpopular idea is an unprofitable one. I'm not sure what a non-profit Google would look like... maybe the same... but I feel pretty confident that a non-profit YouTube for example would likely have a totally different recommendation algorithm.
The tech has changed,not humans. Back in the day, Romans were happily watching gladiators slaughtering each other and all they cared was 'bread and entertainment'. Obviously not everyone, some tried to invent things,do commerce,medicine and etc. Those were the curious ones. It was a small percentage then and it is a small percentage now. The majority are much simpler people. The internet is bursting at its seams with amounts of knowledge available one click away from potential users( yes,you may need to look deeper to find it) but majority don't need it,they want entertainment and they get it.
You're spot on. But really... all people are basically the same. And like Google, they're incentivized by the system to be horrible to each other and/or horrible to the environment. I think people all have a choice. But I think the good choices are really hard to make when you're only surrounded by bad choices.
Human life is usually too short to care too much and we also get bogged down with stuff like work and family,so all those 'great ideas' quickly turn into distant dreams for many.
What people don't like to admit is that Google, YouTube, Reddit and Facebook all prioritize market share above some odd gatekeeping ideal that people have decided an idealized internet should be. They've turned it into a MacDonald's instead of a Tavern on the Green, because that's what most people find accessable even though it's worse in almost every way. But, Tavern on the Green could not serve the same number of people or make the same amount of money as MacDonald's. You can't always scale up a BBS to serve 2.5 billion people and expect it to have the same charm and not expect companies to charge for services to getting results. Dial up services and bulletin boards had their charms, but this internet is way better =)
Personally I just think google is losing to seo because google is playing it too neutral and not opinionated enough. Like facebook, they want decisions to look like math chose the answer, and not people, because they want to deflect accusations of bias. Google would be better if it were more opinionated. I go there because I WANT them to rank results and tell me what they think are the best quality answers.
Google may also be lost in an A/B testing catch 22 feedback loop, where what gets clicked gets elevated, and whats elevated gets clicked. The less straightforward, more clickbait cryptic results get clicked to see what they are.
The result that spells out all the answers might not get
a click at all. Ive noticed in the last year or two wikipedia not even being on the first page of searches it should be the top two or three results for.
They can put in optimization and skew the results with opinion, but not doing that might have more to do with lawyers than desire. There is a keen eye on their search results as far as political and business results are concerned.
Except that Google is quite happy to manually adjust things for their own ends, they just lie about it. They arrange search to make them money, not to help us find what we want.
> I support the author here. There was a promise of a marketplace of ideas, making the world a better place, that being interconnected would make the world smarter and democratize knowledge. All that may be true, it may be out there, but it's damn hard to find.
There was never any such promise, no contract signed in blood. It was just the wishful thinking of a group of people sharing a particular political persuasion. Just because a monocultural group of pioneers who are all radical thinkers (But not too radical) build something, doesn't mean that the people moving into what they build are interested in sharing, or even tolerating their idiosyncratic culture or politics.
The web is supposed to be a way to route a bit of information from one computer to another. This is so low-level, and so broad, that it proscribes nothing about the outcome.
I strongly disagree that the democratization of knowledge is hard to find / that what floats at the top is the most popular, and I strongly disagree that Google is doing anything to make it so.
Imagine in 1995 asking someone whether MSG is harmful or whether razor blades were ever found in Halloween candies. You would hear fairly confident answers of yes to both. Perhaps someone would feel like checking things, and they'd pull out a book of popular facts or an encyclopedia that happily answered yes (or perhaps no) without much discussion, and that would be the end of it. (The Encarta College Dictionary, which awkwardly straddled the transition from books to the internet, had an entirely uncritical definition of "Chinese restaurant syndrome" as a real thing, for instance.)
Go to Google and type in "is msg harmful" and the featured snippet says that while some people have a sensitivity to it, it's definitely not harmful in anywhere near the amounts used in food.
Go to Google and type in "razors in halloween candy" and the top result is a Wikipedia page debunking the urban legends of the '80s. After it is a YouTube video reporting on an actual case of razor blades being found in candy this past Halloween, which Wikipedia hasn't been updated for. In a single Google SERP above the fold, you already have a more accurate synthesis of multiple sources than any single publication has. (And no, I didn't plan this, I didn't even know about the 2019 cases until just now!)
What about political correctness? There's no shortage of content criticizing either the current president or his political opponents past and present. Or, say, search for "puberty blockers" and you get YouTube videos both in favor of them and against them, an handout from Seattle Children's Hospital on how to get them, an article from The Federalist on why they're clearly dangerous, a paywalled article from The Economist discussing it, an article from NBC News saying they've lowered the suicide rate among trans kids... basically any viewpoint you want or don't want, you can find it.
I agree. That might be what the author and other people hoped the internet would become but money drives the development of most things. If there is such an opportunity, then strike out on their own. People talk about privacy but most people just want free and easy, not to be challenged. Each of us is more than welcome to stay in the corners of the internet that we value.
The internet is arguably* terrible today. There are about four websites that most people visit each and every day. And of those four websites, they're most likely owned by one or all of the five largest tech companies.
Before social media, people built their own websites just to put their art out there. Now everything runs through a walled garden, and to your point, it's easier to tweet than to go build a website no one will visit.
>Before social media, people built their own websites just to put their art out there. Now everything runs through a walled garden, and to your point, it's easier to tweet than to go build a website no one will visit.
And yet there is more content on the web, of greater variety and higher quality, than there ever was on that old web. Youtube is a walled garden, but some of what's on there is brilliant, and there's certainly plenty of non-commercial "for the love" content as well. Twitter is, well, Twitter, but accounts like TheSunVanished are publishing ARGs on the platform. Streaming music, video, gaming, all sorts of platforms are full of creative expression despite also being centralized.
Is all of that really unarguably terrible merely because everyone putting their art out there didn't first go through the trouble of manually writing HTML pages on a shared host first?
The internet is definitely, and I still stand by this comment, unarguably terrible today, compared to what it used to be. Is that anyone's fault? Probably not.
People used to download MP3s from AOL chat rooms. There were these people that just served up their music libraries using a chat bot for other people to request downloads.
Once you had enough tracks, you could serve up your own bot to let others share in on the fun.
I can't blame progress for killing culture, but I'm glad that back then, something like serving up an MP3 library in an AOL chatroom was a deeply awesome experience.
>People used to download MP3s from AOL chat rooms. There were these people that just served up their music libraries using a chat bot for other people to request downloads.
>Once you had enough tracks, you could serve up your own bot to let others share in on the fun.
And nowadays people torrent. It's not as if piracy and file sharing are dead or anything.
But I think you're really saying the web is terrible for you, because it no longer resembles what you used to enjoy, and because other people use it in ways you don't find interesting. And that's valid for you, but it's a subjective opinion, not objective fact.
Personally, I'm happy giving up downloading random mp3s from the web to have the depth of content and interactivity the modern web offers. But I care more about content than culture.
Wikipedia is what the web was supposed to be. Prior to that you had Usenet (for conversation) and services like Gopher, Archie, and Veronica (various sorts of databases). The web was meant to be an accessible front end for that that wasn't as headache-inducing as 80x25 terminal.
Teams and editorial boards are great at working together to build things, wikipedia, reddit wikis, fandom/wikia. But the tools they have to calculate consensus (not just the most popular answer but the most correct one, given a cultures beliefs) are stone age. Mahalo.com was an attempt at collaborative information sorting, and https://inside.com/ isnt a bad successor, in theory.
I also think the premise that search/feed are the best options is flawed. The directory was, and stills is a great way to drill down into topics. Search should be more encyclopedia/directory like, and less feed like.
And even in the realm of search, some flags or operators to specify if I'm looking for a historical result, a timely news result, a review, a fact, or an opinion could come in handy.
Google would benefit from humans sitting down, searching things and then going "what should the top results actually be, why are we not there" from a much more exerting their bias standpoint. How are they ranking recipes without a kitchen to taste them? I would totally support a Google recipe division taste testing and ranking technique,to actually know what the best pot pie recipe is. It would be less a waste of money than another messenger product every other year.
Have to be honest, I'm surprised the article wasn't about SEO. That gets a lot of blame for ruining the internet, especially on tech sites.
But Google's propensity to reward sites/pages that are popular or new rather than those which are actually more accurate/better in terms of quality is definitely an issue.
> If you have a good suggestion on how to rapidly measure site accuracy and quality I know some VCs who would very much like to chat.
Bring back some variety of DMOZ, perhaps in a federated (easy to fork) version. That was quite successful at surfacing the best-quality online resources by topic, and even the early Google index seemed to rely on it quite a bit. But it wasn't a VC-funded project, of course.
DMOZ, really? Yes, at the beginning, yes. But 5-6 years later I know plenty of companies and even bloggers who would locate a "volunteer" and ply him with hundreds or thousands of dollars to get them in, get free traffic, and that beast of a PageRank 7 link.
Yes, but this only ever impacted categories where for-profit links are common (and over time, people learn to disregard these links). And Google Search still does a pretty good job of searching for relevant businesses, since it's one of the main things that people use it for.
The issues with the DMOZ approach were basically speed and corruption:
1. It was slow to add categories/sites, which especially hurt categories where things change pretty quickly (gaming, tech and media are good examples, since new systems and frameworks need to have categories added ASAP).
2. Editors were often drawn into corruption, and either judged submissions based on how much they were paid elsewhere or prioritised their own/friends/family's websites.
Both of these issues could potentially be fixed with some more resources and better oversight, but it may mean any future DMOZ equivalent would need a lot more funding than the previous one.
> The issues with the DMOZ approach were basically speed and corruption:
Federation would help with both factors, though. A workable "right to fork" is a powerful incentive against corruption. Notably DMOZ was not federated or "forkable" in any real sense, even though it did have a reasonable amount of sites mirroring it.
2) check references quality - is reference real? does the quoted text match the text from the reference? is it an academic paper published in a journal?
3) authorship quality - what is the academic "impact factor" score for the author?
4) confirmed viewer reviews - subjective review by confirmed users
5) accessibility score - automated user interface usability analysis
It's obviously a harder problem than I want to believe it is, but considering the terrible quality of results as of late, I don't think this would be worse.
First, start with a whitelist. Hand pick high quality publications, and rank them towards the top. This may tilt results back towards institutions, and away from blogs.
Second, punish similarity. If everybody is reposting AP or Reuters without any additional information, consider them a dupe and don't list them. They can run their portals, but they don't need to show up in search.
It's come up multiple times in this thread, car manuals is a good example. They would be better off throwing away every result they have and hand indexing the good information, than what gets returned right now.
Recipes in particular have turned into a giant story about the way grandma used to do it with a picture followed by the same couple variants with different proportions. Pick winners by hand.
Someone has a finance question, just put boggleheads at the top, instead of whichever 59 affiliate credit card sites sprung up.
Need health advice? Put examine.com at the top above WebMD and healthline. Why? Because a human exper compared them and decided examine is a better first result. You could comb through tens of thousands of sites with a team of hundreds of people, something Google easily has at its disposal. What PageRank had, that seems to be missing now, is a seed of "we trust these most" and let the network grow from there. It tried to find expertise, instead of clickability. It was about getting you the best information first.
How would you pick the options by hand? This way you would introduce the thought and point of view bias of the people working at Google to the results. Additionally how would you pick who is the winner? For example how would you compare a technology article is more correct coming from various sources? They also have their own bias introduced to their content.
>This way you would introduce the thought and point of view bias of the people working at Google to the results.
Correct. I would do that.
.
>They also have their own bias introduced to their content.
Yes. Good.
I'm not necessarily saying they hand pick the best article for every single story. Although techmeme.com and hn do that to an extent, when they notice a better version of an article, they replace the top link with the better version.
> If you have a good suggestion on how to rapidly measure site accuracy and quality I know some VCs who would very much like to chat.
I spent a few days thinking about it not so long ago and I have thought of something rarely mentioned. Don't get me wrong, I don't think I have completely solved the problem, just noticed it changes the perspective.
If I remember well, from my user perspective, the biggest change Google introduced was the ranking by page. Yahoo used to rank by site not by page.
Maybe going back to a ranking by site would help creating a good index.
A site would be associated to a number of keywords, say 20 and that's it. That would give incentive to pick the keywords you want to rank for carefully and really be an expert about them instead of having SEO experts deciding which keywords they want to rank for this week and write empty TF-IDF optimized blog posts.
This sort of search engine would not give you the answer to everything but it would give back power to the websites. The information retrieval process would then be 2 steps :
SEO is behind many of these dynamics, though. The "more accurate"/"better quality" signal is getting so noisy that rewarding freshness and hoping the user meant to search a very current topic is perhaps the best you can do. Quite disappointing of course (since we'd rather have good-quality content be easily reachable) but not entirely unexpected.
I see it that way: people tried to get exposure with Google finding ways to do it. For example by exchanging links. Was it bad people Exchanged links? No. Was it used by spammers? Yes, also. Did Google decide to punish it? Yes. Who is hurt? Normal website users. Spammers found new ways. But the truth is with each Google update normal website owners was punished. And for spammers it just got a little bit more complicated to find a new hack.
Now we are at a point where normal website users have very little ways to be high in search results. And people with money can buy it either with very expensive SEO or with expensive Ads.
Sometimes i wonder why the heck should normal website user even try to please Google. As a normal website owner i dont feel Google gives me as much as it expects me to do.
Dont link to this, use AMP, God forbid to Exchange links. There are books about how to please Google. But what is the point? It all comes to who has more money. I dont, so i will never win a good position.
I think we should just forget all these Google rules because they destroyed the Internet how it used to be. Autodiscoverable.
When I first starting making websites in 1996 I would market them by going to other similar websites and we would link to each other. Also, there were webrings and directories. Now if you link to each other Google thinks you are gaming their search engine and will demote your link and possibly your entire site.
Some years ago when i wanted to search for something, my only concern was to guess how it may be written dont on a page, so Google can find it for me.
I could click next as long as i needed. I could refine query to get better results.
But now result list is extremely limited. Refining query gives the same result.
Google was once a search engine that allowed to discover content. Now, it is not.
You could write an article and Google indexed it and showed it if people searched for it. Now it does not work that way. If your audience visits other pages than yours, it will show irrelevant info from these pages rather than perfect match from yours.
And also Patelisms. Once, a short post was enought for Google to index it. Now it has to be essencially a book. It does not need to answer any question, as long as it has a length of a book and thousands of illustrations.
I wished there was a search engine that finds pages matching query, not guessing answers. Giving the freedom to explore rather than giving cheap crappy answers.
> If your audience visits other pages than yours, it will show irrelevant info from these pages rather than perfect match from yours.
I empirically disagree. For me Google often shows small sites with perfect matches before big sites with vague matches and a few of my small sites also rank very well next to giants.
> I wished there was a search engine that finds pages matching query, not guessing answers.
It only proves Google is unpredictable. And this is also a sad reality. I often see "my" Google is better adjusted to find some answers but extremely lame at 90% of other queries.
Ps. Quotation marks help in some degree. But the response pool is often very small. Also sometimes quotation marks return broader results than expected to
A lesson in optimizing the wrong cost function. The internet set information free. And like the millennia before, the masses congregated to gossip, laugh, fight, and whatever else helps pass the time.
which are some interesting search engines, do ya folks recommend. I recently ran into dogpile.com. really relevant results. please don't recommend ddg | bing as ddg simply mirrors bing.
Google didn't ruined the internet. In fact, the internet isn't in ruin. Perhaps the author should reconsider treating Google, or any search engine, as sources of truth.
And if Google ruined the internet, then why give them more power by using Gmail?
Every complaint in the article is about the behavior of the website named google dot com. Google dot com is not the internet. If the complaints were about how Google is changing the behavior of other websites I could maybe somewhat get behind that. But in light of the content of the article the title makes no sense whatsoever.
"Google sometimes gets things wrong" would be a more accurate title. It wouldn't get any upvotes, nor deserve any (surely it's obvious that a website trying to be as many things as Google does will sometimes be inaccurate). But it would be more truthful.
That's probably because Google is still good enough for most people, that there hasn't yet been a need to change the search engine. If Google becomes useless people will organically move to something else, of course it will first start with the tech savvy people, but as with all things it will trickle down to others. If the alternative isn't better then Google then, of course, nothing will change. I don't really see a reason why people feel the need to be so negative about pretty much everything, same goes with every aspect of life.
I am aware that this comment is more or less useless to many, but I still wanted to write it, because I have the power and the freedom to express my _opinion_, and because I still hope that there are people that will understand what I'm trying to say.
Let Google be Google.
If Google's vision sings the same song as the song of the people, then the people will use it.
If it doesn't then people will find another search engine that will sing a song that people will like.
Or in other words, I don't see a reason for us to try to change Google, if Google will want to be the primary search engine, then Google will have to change on it's own.
Am I completly wrong? I Google strong enough to be able to shape the song of creation (strong enough to be able to prevent any other search engine from rising up even though the other search engine could be better)?
> If Google becomes useless people will organically move to something else,
This is not what happens easily. In a near monopoly, the monopoly product can be significantly worse than other options, or what is reasonably possible, but people will continue using it for a long time because of various economic, network, psychological lock-in effects.
Agreed, but even "Google sometimes gets things wrong" isn't entirely accurate.
So it favors popularity over accuracy. So what? Google is not in the business of providing the most accurate search results. It's in the business of generating ad revenue. If popular links make Google more valuable, then favoring them is "getting things right" from the perspective of a private company.
I don't think it's correct to say that it favors popularity over accuracy. Actually I think that is a seriously editorialized take as well, not in keeping with the spirit of HN. I think it would be more correct to say that much of the time popularity is accuracy, in that it's what the searcher is looking for. When there are a variety of interpretations of a query, the most popular interpretation is the rational prior expectation. When not -- for example, when I search for "go" with a history of searching for programming related topics -- Google tries to give me what I'm looking for.
But it's also just a hard problem. That featured snippet about the dentist for example: Google's computers aren't investigative journalists. The purpose of the featured snippet is actually to favor accuracy over popularity by deferring to journalists when Google senses that the searcher is looking for information about an event that was covered in the news. However, if the journalists get it wrong, how is Google going to know? Dollars to donuts, if Google actually knew the right answer, that's what they would surface.
As the defacto front page to the internet, it is aeguably in society's best interest for search results to return more than SEO spam and ads.
I think the modern state of Google is a huge disservice to civilization, compared to what it was and could be. By prioritizing popularity it reduces the majority of search queries to the lowest common denominator and encourages shallow, non-technical culture.
I think what we're seeing in the refinement algorithm is a regression to the layman's mean, so to speak, as they tune (train?) The algorithm to work better for the majority of their users, who happen to be non-technical.
But when you excessively dumb down technology you reduce incentive for people to learn anything and, more importantly here, the dumbing down means showing entertainment and SEO results over possibly more technical content.
Personally I find it disheartening when I search for technical words and the only results are celebrities or media.
They provide value in a specific form, which is aligned to their goals, and it may or may not overlap with what's optimal for society. But they're not in the business of providing what's best for society. That's a byproduct of competition, not a goal from the outset.
But that's the thing. Any single actor can only provide sub-optimal results, but competition and regulation create a competitive game which provides society with optimal results.
Google isn't wrong – it's actually doing its part really well. It's either competition or regulation that need tweaking. Probably both.
In other words
> an economic system that by it's very nature can only provide sub-optimal results is broken.
The economic system can provide optimal results. Individual actors can but don't need to in order for the prior statement to be true.
Normally you would be right. Service X is not the Internet.
But this doesn't really apply anymore once service X dominates the way a large proportion of the western population are trying to locate something - as opposed to clicking on links or bookmarks.
Reality is what actually matters. Not what people think or what they wish. It is what people do and what actually happens.
The reality is that Google not only holds a vastly dominant market position in search - but its search is the default way a majority of people interact with the Internet.
The address bar in your browser is not just an address bar. It is both a search bar and an address bar. And that makes a real difference when the search engine behind it is most often one search engine.
Actually, it isn't. It makes it immeasurably stronger. You see, you just can't get around google. They've poisoned the well in so many ways that you just have to use their products or you will end up being cut out. Email is the best example of that. Google determines who can and who can't email and the best defense against that is to make the problem worse: use a gmail address.
Google didn't stop there. It ruined the internet even at protocols level. Google engineers didn't understand the beauty and simplicity of the original internet protocols, and were in a position to trace the evolution path of such algorithms without any of the elegance and equilibrium the original designers had. Now if you want to make an HTTP query you have to understand an incredible amount of details. Before that with adsense Google forced the web to evolve into a clickbait arena. Google is the worst thing that ever happened to the internet.
In my opinion, "the Feed" ruined the internet. The news feed, the Twitter feed, the Reddit feed, etc. The addictive nature of the Feed, and the tendency to reward dramatic or outrageous or ridiculous content leads to the herd mentality and mindless dogpiling that occurs on these platforms. And then, because expression is compressed into short form soundbites, pics and videos, the platforms actively inhibit constructive, complex discussion. This is one reason I like podcasts and why, for example, Joe Rogan has become popular. The demand for long form, complex discussion is higher than the supply the internet currently provides.
A very good thought, but I don't think all feeds are equal. A strictly chronological feed under user control is a wonderful thing. I have that, for example, in my RSS reader. What sucks is the popularity-contest nature of most feeds. At the top of my Facebook or Twitter or Google News feed is a bunch of stuff that I think is crap, any semi-sensible algorithm could tell that it's crap, it shouldn't be shown to me anywhere. Meanwhile, content from my actual friends or people who I have demonstrated interest in is pushed way down or all the way out. All because of popularity and some very misguided notions of engagement. Bad feeds ruined the internet.
I actually do think likes, stars, upvotes, whatever you want to call them are valuable forms of feedback. I like specific reactions - love, laughter, anger - even better. But I do wonder what it would be like if those only went to the poster. If they weren't shown to anyone else, and didn't affect what was shown to anyone else. I suspect that it would make "the feed" a usable model again, instead of the abomination it has become.
On your last point, that's more or less how HN works for comments. You can't tell if a comment has been upvoted 20 times or -5 times (until it reaches the negative threshold for fading out). But I agree, that would help.
Agreed. It's not the feed that broke the internet – it's the forced filter.
The notion that forced filtration, reprioritization, and chronological rearrangement of content is implicit the idea of information feeds (search, social media, etc) shows how deep the brokenness is baked in.
Sometimes the filter is a feature, but sometimes it's not. In the name of convenience, it feels like we've given up our right to choose to see everything and decide for ourselves how to filter.
Or even youtube. The site gives you zero ability to manage your subscriptions in a reasonable fashion so you're best off just watching whatever it thinks you should watch. I've resorted to bookmarks over subscriptions because it's a better user interface than what the site gives me.
You can't prioritize some content without deprioritizing other content, which effectively amounts to censorship. Algorithmic "timelines" (which aren't timelines at all) are one of the biggest mass scams ever perpetrated. It allows giant companies to sit and mediate personal friendships, and extract rent for anyone who wants more reach (even within their existing, opt-in audience).
I think daily about ways to get people to stop donating content to these censorship and surveillance platforms. Most people don't run businesses, so they never realize the rent-seeking nature of these jerks.
You can't prioritize some content without deprioritizing other content, which effectively amounts to censorship.
This would imply HN "censors" the front page. Of course that's nonsense. Sometimes sorting by an algorithm is just sorting.
In the case of HN it's up votes, and in the case of Google it's a basket of hundreds of criteria (including up votes if you think of clicks on results as voting). Google probably does have some "censorship" rules like filtering out illegal content, but I'd be surprised if they're not impartial about everything else.
That’s precisely what they do. Everyone’s allowed to censor what they wish on their own webpage. I, for example, censor from my own webpage (which otherwise contains a lot of information about me) anything someone could use to physically harm me.
The issue comes up when the censorship is used, for example, in DMs or timeline posts between friends (as it is on Facebook and Instagram and Discord), versus one’s own content on a webpage.
There’s a difference between moderation and rent-seeking.
What I would like to see is a way to build charters and algorithmicly enforced governments, and then fork communities when divide exceeds concensus. What I mean by this is a subreddit where you can choose how votes are tallied, and instead of a fiefdom, becomes an open source template for other communities who can inherit the historical record.
When you create the community, it gives you a mad libs style page, where you can choose if you are a democracy, republic, autocracy, a mix of them. Vote weighting, whos votes count for what things, vote prediction and extrapolation. Conditions and Voteing for changing the charter rules, votes for how content gets posted, votes for leadership boards and mastheads. The ability for editors to strengthen or amplify certain voices or voters in a community.
If you could seed a community with "role model voters" and then use the other voters who vote like them to extrapolate how they would vote on stories they havent seen. There is this common misunderstanding in the word that votes are only used as votes, and not as a signal to indirectly form a decision.
Every AI feed I've ever used (prismatic being one that comes to mind) buckles under popular vapid content rising above. Extremely strictly moderated feeds do exist tho: https://aldaily.comhttps://longform.org/https://longreads.com/ I just wish there was a way for a group of strangers, who arent math experts, could spin up a collaborative feed reader (something that autoingests rss/twitter), and through their collective upvotes posts it to a more static page. Counterparties did something like that with Percolate.com and was able to still use the product once it pivoted markets. https://www.techmeme.com, https://www.mediagazer.com and https://www.memeorandum.com/ do the same thing. But those were all close knit teams that knew each other, their underlying tools arent accessible to spin up your own. I'm sure these tools exist for newsrooms, with the abundance of modern CRMs coming to maturity, a place to chat and edit before things get published. But they arent built for strangers to create community together.
It would make sense for these communities to more resemble Wikis with more static content, instead of the endless feed first. /r/personalfinance wiki being the front page, and the feed being something on the side powering new content to add to it.
Id really like to see something that combines reddit, git, fandom/wikia, techmeme, rtings.com, slant.co, kit.co into a collaborative consumer reports, wirecutter, or metacritic, rotten tomatoes. A place for people to gather to build consensus around something in a more structured way than wikipedia, and then publish it. Places like https://letterboxd.com succeed in some ways, but only through the existence of a shared culture and keeping to a specific topic.
There could also be an element of customizability for the end user. A metacritic or rotten tomatoes where you can weight certain critics votes, or a wirecutter where you can express your preferences for certain traits and then have it output a ranking list. Not completely dissimilar to rtings.com's magic tv ranker.
with the current confinement situation in the world, my podcast episodes list has been growing. i caught up with some of it over last weekend but i feel like i don't have a dedicated time for podcast anymore.
The feed competes against all the learning resources on the internet. Perhaps people in general are just tired and looking for banter, because the non-banter, however a minority, is completely discoverable.
Ever since the first time I heard that word, I've always imagined animals on a farm feeding happily at a trough, unaware of their impending doom. It turns out, that's closer to reality than I thought.
I don't know, chronological formats have been there even before the WWW, namely Usenet, IRC and even E-Mail in a way. So this stuff also rarely made its way to web searches.
But probably there is a higher percentage hosted on proprietary platforms (Reddit, FB, ...) that replaces services mentioned above but personal websites as well.
> Search for "GM" and the Gmail homepage ranks prominently. Is Google pushing its own products on people? Well, yes, but not here. This is an example of a query refinement algorithm at work. Google is altering its results in recognition of the fact that many people who search "gm" subsequently search for "gmail."
I tried this and there are three news articles about General Motors, an infobox for General Motors, 4 search results for General Motors (their web page, another of their pages, a New York Times article about them, and the Wikipedia article about them), a box containing tweets from General Motors, two more news articles about General motors, and finally a link to GMail. Then the other "GMs" start, including GraphicsMagick for node.js. I think they did a pretty good job interpreting "GM" here, and I don't think the Internet is exactly ruined.
Even as a Gmail user, my results align with yours.
If I type it into the browser search bar however it is the first result. This is not Google's fault, as I'm on Firefox, which is smartly trying to look for sites I frequently visit.
Yeah, same here, gmail is the last result in my try.
Oh, as a gmail user, why not it ranked as number 1? I don't care the GM news : )
I think people have different standard for a good/bad search result.
Just tried this. GMail was the first link, validating "the Gmail homepage ranks prominently." Also, all the suggestions were gmail (before the search.)
The rest of the results were GM related news/Twitter/links.
Are you sure you're not confusing Google and DuckDuckGo? (I know I do from time to time)
For me, a Google search returned only General Motors results in the top spots, while a DuckDuckGo search had Gmail as the first result after the news stories.
To add another data point, my Google results for "gm" look like those on your screenshot, with a market summary card and then GMail being the first result, and General Motors second - with one extra difference that "Top Stories" card in my case is above, not below the GMail result.
tldr; (don't shoot me i'm just the messenger): if google says it's true it must be. This is bad because people are dumb so they will just look at what one search engine says without checking sources and believe that to be the truth.
The author had a few bad queries and chalks the whole thing up to "Google Ruined the Internet". This is akin to having a bad experience with airline food and declaring that flying has ruined travel. It is absurd, it reeks of the stallman-esque style of negative, borderline luddite spew. Instead of offering a solution, this person just rants.
Google has been a pivotal center of the internet - information has never been easier to find! Ease of access to information does not, however, alleviate the need placed upon the reader to sift out fact from fiction.
As a thought experiment, would our luddite-esque author friend prefer that Google was the arbiter of truth, rather than trends? I certainly would not.
I think it depends what you're looking for. I realize this is obscure, but I can't find any reference online to a big Facebook Platform developers' conference that happened in 2007. It drives me nuts because I was there at Chelsea Piers with 1,000 people, but it's like it never existed.
Useful to whom is the question of course. I stopped using google a while ago because it gave me a useful response less and less frequently and I had to scroll over a lot of cruft to find a good answer if any. When google started it had the best results and no cruft.
Like people who complain that the latest macs are “terrible”, perhaps I’m simply not google’s audience any more. That doesn’t make google, or me, wrong.
By becoming so dominant they shifted from organization to inspiration, and not the good kind of inspiration, the kind where you have to spend lots of time shaping yourself to please the algorithm so that what you do and how you look is dominated by Google, whatever you're trying to do/share/sell is secondary to that.
> Google has become a card catalog that is constantly being reordered by an angry, misinformed mob.
Ever heard of PageRank? Google was literally founded on an algorithm that uses the endorsements of "an angry, misinformed mob" to determine importance/relevancy. Obviously this is only one factor in search results (and may not even be used anymore), but this approach is what has made Google successful.
I don't disagree with the general point that the amount of content in today's Web makes the job of a search engine much harder. Perhaps some of Google's techniques lower the result quality for some users, for some queries. That's a much more boring title for a blog post, I guess.
The web wasn't mob-like at all in the beginning. Most of the content was created by reasonably educated people who were publishing because of passion, not profit or other motives.
> founded on ... the endorsements of "an angry, misinformed mob"
I started grad school at Brown in 1997 and I remember a talk there, by someone from Google or connected to it, about PageRank. PageRank was still new, Google was still in beta. Free swag from dotcoms was literally growing on trees. But I digress.
I remember that the narrative about PageRank at the time was not about popularity, but about expertise. I really remember that the presenter brought up a possible threat to validity—what about gaming the system?—and pointed out that if you wanted to persuade the system that you were an expert on a thing, you could get lots of people who talk about the thing a lot to point a link at your site. BUT, he says: that shows the system works! If you can get that many people who are at least mini-experts on the thing to point to you, the only way to do that would be if you, too, were persuasively an expert on the thing.
(These were... naïve times in many ways.)
Twenty years ago this was highly persuasive. It was an un-game-able system, because "gaming the system" meant the system changed you. I can be pretty cynical about a lot of things, including and especially Google, but I really do think that even Google itself believed this. The company was founded on an algorithm that rewarded expertise.
It turned out that the algorithm also rewarded angry, misinformed mobs, so cmckn isn't exactly wrong here. But I think it's important to be clear that to the extent that Google's algorithms have always done this, they a) weren't meant to, and b) actually didn't in the very earliest days, because the WWW link structure really did, in the 90s, work like they thought it did. (Then the measure became the target, etc etc)
I wonder if in the fight against SEOs Google didn't accidentally punish actual experts. SEO organizations will relentlessly hound changes to Google's algorithm in ways that regular users do not.
In the old days it was cost prohibitive to setup a massive network of interconnected fake hosts to boost your rank, but these days with cheap VPSes this is entirely possible and probably even profitable. I'm certainly not the only person who has searched for something and found a ton of differently styled blogs all hosting exactly the same content stolen from some legitimate website and clogging up pages of Google results.
Links are still the super-major-almost-100% factor in determining ranking though, whether you call it PageRank or some other algorithm.
It's why buying expired domains and throwing your totally unrelated content on them works great.
It's why subdomain/folder leasing is a thing where affiliate sites will pay "high PR" sites to reverse proxy a subdomain or folder to them and (that's the important part) link to it from their main site. And boy, does that work. The same content that would be > page 100 suddenly is in the top 3.
There are other factors, but they don't matter nearly as much as Google's "we have 200 factors that contribute to the ranking" stuff makes it seem. You can throw the most atrocious low quality content on a site with lots of incoming links and it will rank at the top.
We have low-quality content generation not because of Google but because of the low cost of publishing (which the author even mentions). It's exactly why we have email spam: it costs nothing to send. To repurpose a Chris Rock bit, "if sending an email cost $5,000 you'd have no more spam email".
What's the alternative being touted here? No Google? Making things harder to find? Seriously?
I'd say a far bigger issue is people sharing content from and to people who think the same, creating these myopic echo chambers of self-reinforcing beliefs.
Google may do a questionable job at filtering out provably false content but people are way worse at that.
Within such a system it's too easy to foster fear, anger and hate and to propagate provably false information. Anti-vaxxers are just one such group that seem to thrive in this informationless world.
Over last decade or so google is becoming less and less helpful. Results are not user-oriented and they can be far removed from original search queries. This is google's fault, they are for-profit organization with own agenda. Google's usage of your personal data to custom-tailor results to your bubble also contributes to echo chamber.
There are also other factors. There is a vast decrease of internet's signal-to-noise ratio due to low-cost content you mentioned. For a while we could rely on google to get us through the jungle to useful information. Now every SEO is targeting google search. Most other search engines use google's results, so no help there.
Blaming google doesn't help, their interests are not aligned with user interests. Author is actually criticizing us and our dependence on this one-size-fits-all source of information. The alternative is building and adopting better alternatives to google search.
I think I am much more likely to find the truth about a question I have with Google than asking on a BBS forum or browsing through a curated list of links on Yahoo.
That's like saying Gutenberg ruined books because now I can find books full of BS.
279 comments
[ 497 ms ] story [ 8864 ms ] threadIt's not as impossible as you might think. But it's certainly not easy.
[1] http://web.archive.org/web/19990208021747/http://yahoo.com/
https://www.dmoz-odp.org/
My family actually had a book like and I learned about a bunch of the net culture sites through it.
depending on what i am looking for there are a few avenues i would explore if google wasn't available. github, gitlab, & stackoverflow/stackexchange for code related stuff.
wikipedia for general topics. youtube for how-tos. twitter for the news. newsletters and podcasts for links to new articles.
&c.
Still nowadays the best use of Google for me is to find those tematic forums
And I must say search result quality has declined a lot over the years with the past 3 being worse of the sum of the past 20.
must be weird to buy a domain name and still be mostly anonymous and invisible :)
Web crawlers were introduced in 1993. At that point, gopher probably had more content. Remember WAIS? All the libraries liked that protocol. Encarta was still selling like hot cakes.
https://www.superhighway98.com/
What does this even mean?
The web wasn't "supposed to" be anything. Though I'm not sure what magic search engine OP actually has in mind and how it's supposed to work.
Besides, one of the modern mysteries is that we're in the age of instant information yet you'll notice how many people will write up an entire comment online or bicker IRL instead of doing a cursory search. I don't think it's the internet creating human stupidity / laziness. Unfortunately we had that long before, and search engines simply try to show the best results with minimal context.
Also, I think discussion around tech would be much improved if we tried to come up with a better idea whenever we go through the trouble of complaining about something. Anyone can enumerate why things are suboptimal, and usually when you try to come up with alternatives, you find out it's just trade-offs with no ideal solution.
Trying to pitch an alternative solution (like how a search engine should work) helps drill down into real conversational bedrock that's much more interesting.
Google, Youtube, Reddit, and Facebook all prioritize freshness. Instead of being exposed to things outside our comfort zones, we take solace protected inside filter bubbles.
Instead of the best answer, what usually floats to the top is the most repeated, the most seod, the newest, the most politically correct.
Google's results are considerably worse than they were and part of that is google trying too hard to think what we want instead of guiding us to ask better questions.
Can you explain what this means?
I see people saying this wrt to Duck Duck Go: "No, the results aren't worse, you just have to re-learn how to search with it" and all they really mean is that you need to stack more context into the search input which is strictly worse since you can already do that in Google if you need to put a finer point on your search.
And they mean that since DDG doesn't know "Elm" is a programming language most of the time, by "re-learning search" they seem to mean adding "lang" to the input. Where "re-learn" seems like a romantic way to phrase this obvious problem of the search engine needing more context. In the same way I had to "re-learn how to use a keyboard" when my crappy 2017 Macbook Pro keys started coming off.
How does your complaint differ from this?
Hum, not exactly. Google will ignore extra context that doesn't match your profile. People coped very well when it didn't.
Said commenter, as they type a comment in a marketplace of ideas where people discuss fairly complex ideas.
The internet is a tool, not a solution. I absolutely disagree that it's hard to find for anyone looking for it. The issue is that most people aren't looking for it, and you can't force them.
A lot of people are just looking for entertainment, and that's perfectly fine. They spent their whole day working hard, come home, and now you want to force them to spend their night studying and discovering new ideas? That may be the internet you want but it's not what people want. The internet can be for more than one thing.
This happens quite often on Hacker News. One of the fastest and easiest ways to accumulate karma is to be the first to post something like "$hated_company has always been doing $horrible_thing, they need to change", which most people agree with, every time a popular story about them shows up. (Thankfully, usually only a couple of these appear in most discussions, and usually people don't spam this.)
> Said commenter, as they type a comment in a marketplace of ideas where people discuss fairly complex ideas.
Hacker News can be great for complex discussions, but it's not free of filter bubbles and echo chambers.
Nothing on or off the internet is. That is why it is important to keep an open mind and read widely and voraciously.
Some people are more insightful than most, but its the exact same system as reddit
Your upboats are time sensitive.
When people talk negatively about bubbles, they generally focus on the couple cases where being in a bubble is harmful, such as when it comes to science, politics and so on, but ignore all the other times where a bubble is good.
Do you read politics you agree with or disagree with? I make an effort to click more on things I think I'll disagree with.
There's obviously a level of notoriety beyond just quality. I don't need the same article 3 times under 3 logos.
Hacker News is, in the most charitable interpretation, an act of charity by pg and YC to attempt to provide a forum for a specific subculture. It's obvious from the goal of the project: "things that hackers would find interesting".
But what most people get from the Internet is not a product of someone with effectively unlimited funds (and legendary software engineering ability) building a website to support their community. They get to choose between eyeball-extraction megasites or clunky off-the-shelf vBulletin or Wordpress clones run by underqualified volunteers. It certainly is nice to be among the target audience of Hacker News -- but what if I weren't?
> The issue is that most people aren't looking for [a marketplace of ideas], and you can't force them
No? We can't? Or we shouldn't? Might that be our hyper-individualist culture constraining our imagination of what the right path (survival) might be?
Our evolution within terrestrial physical reality "forced" us to participate in well-calibrated local marketplaces of ideas, and our psychology evolved specifically so that we had a fine-tuned balance of what we subjectively "wanted" and what we found ourselves coming to believe, despite that initial-condition "want" -- dissenting views in a room have both a repulsion but also a very specific gravity -- a closeness that emerges amongst holders of opposing ideas, when these ideas have manifested and are walking around in human bodies within shared meatspace -- we start to empathize with holders of countering views that we're forced to share physical space with. We talk about empathy like it's feelings for the other pieces of meat, but it's perhaps better conceived as a kinship of one tight bundle of ideas for another. It's evolved and it's ancient and it's a very specific foraging strategy for 2D terrestrial creatures finding information/food under those constraints.
And now, we've designed systems that aren't nearly as clever and well-calibrated as our meatspace selves evolved to be. In the purely physical space we evolved for, we had to share space with people we probably disagreed with, and we developed unique tendencies based on the nature of living on a 2D terrestrial plane. Heck, we'd have different psychology favoured if we made it to this level of the great filter, but happened to evolve in the air (3D grid) or within a more one-dimensional environment or some hyperdimensional space.
Speaking of high-dimensional space: enter the internet. Might our prior foraging strategies and adaptations stacked onto our prior foraging strategies... might they fail now? Foraging strategies are informed by the math of the landscape. (search terms: optimal foraging strategies, radius of perception, levy walk, agent-based modeling, in the vein of [1]) Our psychology is tailored to adapting to physical reality on a plane, and the internet might totally fuck that up. (What is an internet bubble? Maybe it's just my stepping out of the 2D terrestrial grid and engaging through a hidden, non-spatial dimension with some foraging target I can sense near me?) It's like all places are piped into one another, outside physical reality. This isn't Kansas. It's the formation of a hyperdimensional object. It's no longer a 2D grid, and our predispositions and adaptations for navigating such a grid might drive us to extinction.
imho, we DO need to consider "forcing" (as a collective) changes in that infrastructure, or else our hitherto evolved psychology might just as well see us destroyed. So in the end, what people "want" is maybe not the highest principle to hold, because what we individually "want" might be tuned for a dying reality, and a prior foraging landscape.
Heh, anyhow... perhaps this is just a mad-cap rant from a technologist and failed biochemist <3
[1]: https://par.nsf.gov/servlets/purl/10135146
Furthermore, it's so heavily weighted to only show results from sites with lots of SEO juice. I can't find what I'm looking for anymore.
There are some areas where google really excels and that's in technical searches. Those are really well done.
If true, this would be big news, as the "+" qualifier on search terms stopped being obeyed about a decade ago with the launch of... whatever that now-forgotten Facebook competitor Google used to have. The workaround was to use quotes around individual words, which mostly worked, unless it didn't. But the combination of using quotes and "verbatim" (under Tools/All Results") almost always worked.
[1] The way it failed is sort of interesting. I randomly tried "+rhino +cereal +gm", and the first result didn't include the word "rhino", and considered "gm" to be a synonym for General Mills. Quotes around the individual terms seems to work for this query, even without verbatim.
Based on Google’s advanced operator page at one point, the addition of a plus sign to a search termed either prompted the lookup of Google+ pages or a blood type.
'+' was replaced by adding quotes around each search term.
[0] https://googlesystem.blogspot.com/2011/10/googles-plus-opera...
I have been there since the old times of IRC and I have never seen anything promised.
Did anyone really believe this other than middle aged libertarian hackers from the 90s who kept repeating this in prayer like fashion?
The internet is, as the name suggests, a network. Google also is a sort of network, one that is semantically searchable. But Google Search is not a fact checker. It doesn't have an editorial room for every piece of content it exposes, it doesn't have deep knowledge about everything it throws back at us, and it could not have those things at the scale it operates.
That's not Google's fault either. It's not a search engine's job to correct wrong reporting. It's the job of journalists and news outlets to fact-check information, it's the job of citizens to be critical of the information they consume, and it's the job of governments to facilitate that this happens.
It's our fault that our civic institutions have degraded, and not the fault of Google that it spits our own stupidity back at us. It's not Google's fault that we don't like what we see when we look in the mirror and techno utopianism isn't going to save us either.
It does not succeed as well as it did in the past at its current mission. It has regressed.
I’m fairly certain they’re doing the best they can with what’s available to them, but right now they probably get more accurate results by focusing on what is popular than by blindly accepting the contents of the websites.
I vividly remember the promise, and believing in that promise, but there was never a time that the Internet was this. In retrospect I don't see why we should have believed in it in the first place.
It doesn't work.
https://news.ycombinator.com/item?id=22617635
Wikipedia does a damn fine job in my opinion. You can find decent, democratized knowledge on almost any topic complete with translation into your preferred language (if not on wiki, then through google translate). Or efforts like MIT's open course ware, which hosts lectures and syllabus materials on a variety of topics. There's also arxiv derivatives and a thriving black market (scihub, libgen, etc) of open access to research papers on any topic you'd like. Social media =/ the internet.
No but in all seriousness, the mistake was in relying so heavily on private corporations motivated by profit. An unpopular idea is an unprofitable one. I'm not sure what a non-profit Google would look like... maybe the same... but I feel pretty confident that a non-profit YouTube for example would likely have a totally different recommendation algorithm.
Google may also be lost in an A/B testing catch 22 feedback loop, where what gets clicked gets elevated, and whats elevated gets clicked. The less straightforward, more clickbait cryptic results get clicked to see what they are. The result that spells out all the answers might not get a click at all. Ive noticed in the last year or two wikipedia not even being on the first page of searches it should be the top two or three results for.
There was never any such promise, no contract signed in blood. It was just the wishful thinking of a group of people sharing a particular political persuasion. Just because a monocultural group of pioneers who are all radical thinkers (But not too radical) build something, doesn't mean that the people moving into what they build are interested in sharing, or even tolerating their idiosyncratic culture or politics.
The web is supposed to be a way to route a bit of information from one computer to another. This is so low-level, and so broad, that it proscribes nothing about the outcome.
Imagine in 1995 asking someone whether MSG is harmful or whether razor blades were ever found in Halloween candies. You would hear fairly confident answers of yes to both. Perhaps someone would feel like checking things, and they'd pull out a book of popular facts or an encyclopedia that happily answered yes (or perhaps no) without much discussion, and that would be the end of it. (The Encarta College Dictionary, which awkwardly straddled the transition from books to the internet, had an entirely uncritical definition of "Chinese restaurant syndrome" as a real thing, for instance.)
Go to Google and type in "is msg harmful" and the featured snippet says that while some people have a sensitivity to it, it's definitely not harmful in anywhere near the amounts used in food.
Go to Google and type in "razors in halloween candy" and the top result is a Wikipedia page debunking the urban legends of the '80s. After it is a YouTube video reporting on an actual case of razor blades being found in candy this past Halloween, which Wikipedia hasn't been updated for. In a single Google SERP above the fold, you already have a more accurate synthesis of multiple sources than any single publication has. (And no, I didn't plan this, I didn't even know about the 2019 cases until just now!)
What about political correctness? There's no shortage of content criticizing either the current president or his political opponents past and present. Or, say, search for "puberty blockers" and you get YouTube videos both in favor of them and against them, an handout from Seattle Children's Hospital on how to get them, an article from The Federalist on why they're clearly dangerous, a paywalled article from The Economist discussing it, an article from NBC News saying they've lowered the suicide rate among trans kids... basically any viewpoint you want or don't want, you can find it.
It's all there.
Before social media, people built their own websites just to put their art out there. Now everything runs through a walled garden, and to your point, it's easier to tweet than to go build a website no one will visit.
And yet there is more content on the web, of greater variety and higher quality, than there ever was on that old web. Youtube is a walled garden, but some of what's on there is brilliant, and there's certainly plenty of non-commercial "for the love" content as well. Twitter is, well, Twitter, but accounts like TheSunVanished are publishing ARGs on the platform. Streaming music, video, gaming, all sorts of platforms are full of creative expression despite also being centralized.
Is all of that really unarguably terrible merely because everyone putting their art out there didn't first go through the trouble of manually writing HTML pages on a shared host first?
People used to download MP3s from AOL chat rooms. There were these people that just served up their music libraries using a chat bot for other people to request downloads.
Once you had enough tracks, you could serve up your own bot to let others share in on the fun.
I can't blame progress for killing culture, but I'm glad that back then, something like serving up an MP3 library in an AOL chatroom was a deeply awesome experience.
>Once you had enough tracks, you could serve up your own bot to let others share in on the fun.
And nowadays people torrent. It's not as if piracy and file sharing are dead or anything.
But I think you're really saying the web is terrible for you, because it no longer resembles what you used to enjoy, and because other people use it in ways you don't find interesting. And that's valid for you, but it's a subjective opinion, not objective fact.
Personally, I'm happy giving up downloading random mp3s from the web to have the depth of content and interactivity the modern web offers. But I care more about content than culture.
It might be easier, but it depends on your target audience how effective it is.
Wikipedia is what the web was supposed to be. Prior to that you had Usenet (for conversation) and services like Gopher, Archie, and Veronica (various sorts of databases). The web was meant to be an accessible front end for that that wasn't as headache-inducing as 80x25 terminal.
https://news.ycombinator.com/item?id=22744171
Teams and editorial boards are great at working together to build things, wikipedia, reddit wikis, fandom/wikia. But the tools they have to calculate consensus (not just the most popular answer but the most correct one, given a cultures beliefs) are stone age. Mahalo.com was an attempt at collaborative information sorting, and https://inside.com/ isnt a bad successor, in theory.
I also think the premise that search/feed are the best options is flawed. The directory was, and stills is a great way to drill down into topics. Search should be more encyclopedia/directory like, and less feed like.
And even in the realm of search, some flags or operators to specify if I'm looking for a historical result, a timely news result, a review, a fact, or an opinion could come in handy.
Google would benefit from humans sitting down, searching things and then going "what should the top results actually be, why are we not there" from a much more exerting their bias standpoint. How are they ranking recipes without a kitchen to taste them? I would totally support a Google recipe division taste testing and ranking technique,to actually know what the best pot pie recipe is. It would be less a waste of money than another messenger product every other year.
2. especially for businesses, it comes off as unprofessional if you are using gmail, yahoo, etc as your contact address
But Google's propensity to reward sites/pages that are popular or new rather than those which are actually more accurate/better in terms of quality is definitely an issue.
Well, no, I don't. But I highly suspect that they exist and would want to chat.
Bring back some variety of DMOZ, perhaps in a federated (easy to fork) version. That was quite successful at surfacing the best-quality online resources by topic, and even the early Google index seemed to rely on it quite a bit. But it wasn't a VC-funded project, of course.
1. It was slow to add categories/sites, which especially hurt categories where things change pretty quickly (gaming, tech and media are good examples, since new systems and frameworks need to have categories added ASAP).
2. Editors were often drawn into corruption, and either judged submissions based on how much they were paid elsewhere or prioritised their own/friends/family's websites.
Both of these issues could potentially be fixed with some more resources and better oversight, but it may mean any future DMOZ equivalent would need a lot more funding than the previous one.
Federation would help with both factors, though. A workable "right to fork" is a powerful incentive against corruption. Notably DMOZ was not federated or "forkable" in any real sense, even though it did have a reasonable amount of sites mirroring it.
1) look for references to source materials
2) check references quality - is reference real? does the quoted text match the text from the reference? is it an academic paper published in a journal?
3) authorship quality - what is the academic "impact factor" score for the author?
4) confirmed viewer reviews - subjective review by confirmed users
5) accessibility score - automated user interface usability analysis
High quality data exists, but it's not much of the ad supported web and not much of what users what to read.
First, start with a whitelist. Hand pick high quality publications, and rank them towards the top. This may tilt results back towards institutions, and away from blogs.
Second, punish similarity. If everybody is reposting AP or Reuters without any additional information, consider them a dupe and don't list them. They can run their portals, but they don't need to show up in search.
It's come up multiple times in this thread, car manuals is a good example. They would be better off throwing away every result they have and hand indexing the good information, than what gets returned right now.
Recipes in particular have turned into a giant story about the way grandma used to do it with a picture followed by the same couple variants with different proportions. Pick winners by hand.
Someone has a finance question, just put boggleheads at the top, instead of whichever 59 affiliate credit card sites sprung up.
Need health advice? Put examine.com at the top above WebMD and healthline. Why? Because a human exper compared them and decided examine is a better first result. You could comb through tens of thousands of sites with a team of hundreds of people, something Google easily has at its disposal. What PageRank had, that seems to be missing now, is a seed of "we trust these most" and let the network grow from there. It tried to find expertise, instead of clickability. It was about getting you the best information first.
Correct. I would do that.
.
>They also have their own bias introduced to their content.
Yes. Good.
I'm not necessarily saying they hand pick the best article for every single story. Although techmeme.com and hn do that to an extent, when they notice a better version of an article, they replace the top link with the better version.
I spent a few days thinking about it not so long ago and I have thought of something rarely mentioned. Don't get me wrong, I don't think I have completely solved the problem, just noticed it changes the perspective.
If I remember well, from my user perspective, the biggest change Google introduced was the ranking by page. Yahoo used to rank by site not by page. Maybe going back to a ranking by site would help creating a good index.
A site would be associated to a number of keywords, say 20 and that's it. That would give incentive to pick the keywords you want to rank for carefully and really be an expert about them instead of having SEO experts deciding which keywords they want to rank for this week and write empty TF-IDF optimized blog posts.
This sort of search engine would not give you the answer to everything but it would give back power to the websites. The information retrieval process would then be 2 steps :
- find a good website
- find the information within the website
Now we are at a point where normal website users have very little ways to be high in search results. And people with money can buy it either with very expensive SEO or with expensive Ads.
Sometimes i wonder why the heck should normal website user even try to please Google. As a normal website owner i dont feel Google gives me as much as it expects me to do.
Dont link to this, use AMP, God forbid to Exchange links. There are books about how to please Google. But what is the point? It all comes to who has more money. I dont, so i will never win a good position.
I think we should just forget all these Google rules because they destroyed the Internet how it used to be. Autodiscoverable.
I could click next as long as i needed. I could refine query to get better results.
But now result list is extremely limited. Refining query gives the same result.
Google was once a search engine that allowed to discover content. Now, it is not.
You could write an article and Google indexed it and showed it if people searched for it. Now it does not work that way. If your audience visits other pages than yours, it will show irrelevant info from these pages rather than perfect match from yours.
And also Patelisms. Once, a short post was enought for Google to index it. Now it has to be essencially a book. It does not need to answer any question, as long as it has a length of a book and thousands of illustrations.
I wished there was a search engine that finds pages matching query, not guessing answers. Giving the freedom to explore rather than giving cheap crappy answers.
I empirically disagree. For me Google often shows small sites with perfect matches before big sites with vague matches and a few of my small sites also rank very well next to giants.
> I wished there was a search engine that finds pages matching query, not guessing answers.
Why not use quotation marks?
Ps. Quotation marks help in some degree. But the response pool is often very small. Also sometimes quotation marks return broader results than expected to
Google didn't ruined the internet. In fact, the internet isn't in ruin. Perhaps the author should reconsider treating Google, or any search engine, as sources of truth.
And if Google ruined the internet, then why give them more power by using Gmail?
"Google sometimes gets things wrong" would be a more accurate title. It wouldn't get any upvotes, nor deserve any (surely it's obvious that a website trying to be as many things as Google does will sometimes be inaccurate). But it would be more truthful.
Example: if(Donald_Trump.as(President) == Bad){ Election.vote = !Donald_Trump; }else{ Election.vote = Donald_Trump; }
I am aware that this comment is more or less useless to many, but I still wanted to write it, because I have the power and the freedom to express my _opinion_, and because I still hope that there are people that will understand what I'm trying to say.
Let Google be Google.
If Google's vision sings the same song as the song of the people, then the people will use it. If it doesn't then people will find another search engine that will sing a song that people will like.
Or in other words, I don't see a reason for us to try to change Google, if Google will want to be the primary search engine, then Google will have to change on it's own.
Am I completly wrong? I Google strong enough to be able to shape the song of creation (strong enough to be able to prevent any other search engine from rising up even though the other search engine could be better)?
This is not what happens easily. In a near monopoly, the monopoly product can be significantly worse than other options, or what is reasonably possible, but people will continue using it for a long time because of various economic, network, psychological lock-in effects.
So it favors popularity over accuracy. So what? Google is not in the business of providing the most accurate search results. It's in the business of generating ad revenue. If popular links make Google more valuable, then favoring them is "getting things right" from the perspective of a private company.
But it's also just a hard problem. That featured snippet about the dentist for example: Google's computers aren't investigative journalists. The purpose of the featured snippet is actually to favor accuracy over popularity by deferring to journalists when Google senses that the searcher is looking for information about an event that was covered in the news. However, if the journalists get it wrong, how is Google going to know? Dollars to donuts, if Google actually knew the right answer, that's what they would surface.
I'm not the one making that claim, I'm just starting from that assumption since the parents are claiming as much
As the defacto front page to the internet, it is aeguably in society's best interest for search results to return more than SEO spam and ads.
I think the modern state of Google is a huge disservice to civilization, compared to what it was and could be. By prioritizing popularity it reduces the majority of search queries to the lowest common denominator and encourages shallow, non-technical culture.
I think what we're seeing in the refinement algorithm is a regression to the layman's mean, so to speak, as they tune (train?) The algorithm to work better for the majority of their users, who happen to be non-technical.
But when you excessively dumb down technology you reduce incentive for people to learn anything and, more importantly here, the dumbing down means showing entertainment and SEO results over possibly more technical content.
Personally I find it disheartening when I search for technical words and the only results are celebrities or media.
Google isn't wrong – it's actually doing its part really well. It's either competition or regulation that need tweaking. Probably both.
In other words
> an economic system that by it's very nature can only provide sub-optimal results is broken.
The economic system can provide optimal results. Individual actors can but don't need to in order for the prior statement to be true.
But this doesn't really apply anymore once service X dominates the way a large proportion of the western population are trying to locate something - as opposed to clicking on links or bookmarks.
Reality is what actually matters. Not what people think or what they wish. It is what people do and what actually happens.
The reality is that Google not only holds a vastly dominant market position in search - but its search is the default way a majority of people interact with the Internet.
The address bar in your browser is not just an address bar. It is both a search bar and an address bar. And that makes a real difference when the search engine behind it is most often one search engine.
I actually do think likes, stars, upvotes, whatever you want to call them are valuable forms of feedback. I like specific reactions - love, laughter, anger - even better. But I do wonder what it would be like if those only went to the poster. If they weren't shown to anyone else, and didn't affect what was shown to anyone else. I suspect that it would make "the feed" a usable model again, instead of the abomination it has become.
The notion that forced filtration, reprioritization, and chronological rearrangement of content is implicit the idea of information feeds (search, social media, etc) shows how deep the brokenness is baked in.
Sometimes the filter is a feature, but sometimes it's not. In the name of convenience, it feels like we've given up our right to choose to see everything and decide for ourselves how to filter.
I think daily about ways to get people to stop donating content to these censorship and surveillance platforms. Most people don't run businesses, so they never realize the rent-seeking nature of these jerks.
This would imply HN "censors" the front page. Of course that's nonsense. Sometimes sorting by an algorithm is just sorting.
In the case of HN it's up votes, and in the case of Google it's a basket of hundreds of criteria (including up votes if you think of clicks on results as voting). Google probably does have some "censorship" rules like filtering out illegal content, but I'd be surprised if they're not impartial about everything else.
That’s precisely what they do. Everyone’s allowed to censor what they wish on their own webpage. I, for example, censor from my own webpage (which otherwise contains a lot of information about me) anything someone could use to physically harm me.
The issue comes up when the censorship is used, for example, in DMs or timeline posts between friends (as it is on Facebook and Instagram and Discord), versus one’s own content on a webpage.
There’s a difference between moderation and rent-seeking.
Assuming impartiality seems awfully optimistic, given both human nature and the fact that Google admits that it is committed to maximizing profit.
When you create the community, it gives you a mad libs style page, where you can choose if you are a democracy, republic, autocracy, a mix of them. Vote weighting, whos votes count for what things, vote prediction and extrapolation. Conditions and Voteing for changing the charter rules, votes for how content gets posted, votes for leadership boards and mastheads. The ability for editors to strengthen or amplify certain voices or voters in a community. If you could seed a community with "role model voters" and then use the other voters who vote like them to extrapolate how they would vote on stories they havent seen. There is this common misunderstanding in the word that votes are only used as votes, and not as a signal to indirectly form a decision.
Every AI feed I've ever used (prismatic being one that comes to mind) buckles under popular vapid content rising above. Extremely strictly moderated feeds do exist tho: https://aldaily.com https://longform.org/ https://longreads.com/ I just wish there was a way for a group of strangers, who arent math experts, could spin up a collaborative feed reader (something that autoingests rss/twitter), and through their collective upvotes posts it to a more static page. Counterparties did something like that with Percolate.com and was able to still use the product once it pivoted markets. https://www.techmeme.com, https://www.mediagazer.com and https://www.memeorandum.com/ do the same thing. But those were all close knit teams that knew each other, their underlying tools arent accessible to spin up your own. I'm sure these tools exist for newsrooms, with the abundance of modern CRMs coming to maturity, a place to chat and edit before things get published. But they arent built for strangers to create community together.
It would make sense for these communities to more resemble Wikis with more static content, instead of the endless feed first. /r/personalfinance wiki being the front page, and the feed being something on the side powering new content to add to it.
Id really like to see something that combines reddit, git, fandom/wikia, techmeme, rtings.com, slant.co, kit.co into a collaborative consumer reports, wirecutter, or metacritic, rotten tomatoes. A place for people to gather to build consensus around something in a more structured way than wikipedia, and then publish it. Places like https://letterboxd.com succeed in some ways, but only through the existence of a shared culture and keeping to a specific topic.
There could also be an element of customizability for the end user. A metacritic or rotten tomatoes where you can weight certain critics votes, or a wirecutter where you can express your preferences for certain traits and then have it output a ranking list. Not completely dissimilar to rtings.com's magic tv ranker.
with the current confinement situation in the world, my podcast episodes list has been growing. i caught up with some of it over last weekend but i feel like i don't have a dedicated time for podcast anymore.
twitter & the web have replaced that time.
It is well worth a read.
Ever since the first time I heard that word, I've always imagined animals on a farm feeding happily at a trough, unaware of their impending doom. It turns out, that's closer to reality than I thought.
Edit: I mean the word "feed".
But probably there is a higher percentage hosted on proprietary platforms (Reddit, FB, ...) that replaces services mentioned above but personal websites as well.
https://en.wikipedia.org/wiki/Eternal_September
I tried this and there are three news articles about General Motors, an infobox for General Motors, 4 search results for General Motors (their web page, another of their pages, a New York Times article about them, and the Wikipedia article about them), a box containing tweets from General Motors, two more news articles about General motors, and finally a link to GMail. Then the other "GMs" start, including GraphicsMagick for node.js. I think they did a pretty good job interpreting "GM" here, and I don't think the Internet is exactly ruined.
If I type it into the browser search bar however it is the first result. This is not Google's fault, as I'm on Firefox, which is smartly trying to look for sites I frequently visit.
The rest of the results were GM related news/Twitter/links.
For me, a Google search returned only General Motors results in the top spots, while a DuckDuckGo search had Gmail as the first result after the news stories.
https://i.imgur.com/m2iXSN9.jpg
Google has been a pivotal center of the internet - information has never been easier to find! Ease of access to information does not, however, alleviate the need placed upon the reader to sift out fact from fiction.
As a thought experiment, would our luddite-esque author friend prefer that Google was the arbiter of truth, rather than trends? I certainly would not.
I think it depends what you're looking for. I realize this is obscure, but I can't find any reference online to a big Facebook Platform developers' conference that happened in 2007. It drives me nuts because I was there at Chelsea Piers with 1,000 people, but it's like it never existed.
https://web.archive.org/web/20080808121324/http://www.facebo...
“Worse is better” wins again.
According to a Wikipedia citation (The Guardian) this was the original mission statement.[2]
[1] https://about.google [2] https://www.theguardian.com/technology/2014/nov/03/larry-pag...
Disclaimer: I work at Google.
Like people who complain that the latest macs are “terrible”, perhaps I’m simply not google’s audience any more. That doesn’t make google, or me, wrong.
Ever heard of PageRank? Google was literally founded on an algorithm that uses the endorsements of "an angry, misinformed mob" to determine importance/relevancy. Obviously this is only one factor in search results (and may not even be used anymore), but this approach is what has made Google successful.
I don't disagree with the general point that the amount of content in today's Web makes the job of a search engine much harder. Perhaps some of Google's techniques lower the result quality for some users, for some queries. That's a much more boring title for a blog post, I guess.
I started grad school at Brown in 1997 and I remember a talk there, by someone from Google or connected to it, about PageRank. PageRank was still new, Google was still in beta. Free swag from dotcoms was literally growing on trees. But I digress.
I remember that the narrative about PageRank at the time was not about popularity, but about expertise. I really remember that the presenter brought up a possible threat to validity—what about gaming the system?—and pointed out that if you wanted to persuade the system that you were an expert on a thing, you could get lots of people who talk about the thing a lot to point a link at your site. BUT, he says: that shows the system works! If you can get that many people who are at least mini-experts on the thing to point to you, the only way to do that would be if you, too, were persuasively an expert on the thing.
(These were... naïve times in many ways.)
Twenty years ago this was highly persuasive. It was an un-game-able system, because "gaming the system" meant the system changed you. I can be pretty cynical about a lot of things, including and especially Google, but I really do think that even Google itself believed this. The company was founded on an algorithm that rewarded expertise.
It turned out that the algorithm also rewarded angry, misinformed mobs, so cmckn isn't exactly wrong here. But I think it's important to be clear that to the extent that Google's algorithms have always done this, they a) weren't meant to, and b) actually didn't in the very earliest days, because the WWW link structure really did, in the 90s, work like they thought it did. (Then the measure became the target, etc etc)
In the old days it was cost prohibitive to setup a massive network of interconnected fake hosts to boost your rank, but these days with cheap VPSes this is entirely possible and probably even profitable. I'm certainly not the only person who has searched for something and found a ton of differently styled blogs all hosting exactly the same content stolen from some legitimate website and clogging up pages of Google results.
It's why buying expired domains and throwing your totally unrelated content on them works great.
It's why subdomain/folder leasing is a thing where affiliate sites will pay "high PR" sites to reverse proxy a subdomain or folder to them and (that's the important part) link to it from their main site. And boy, does that work. The same content that would be > page 100 suddenly is in the top 3.
There are other factors, but they don't matter nearly as much as Google's "we have 200 factors that contribute to the ranking" stuff makes it seem. You can throw the most atrocious low quality content on a site with lots of incoming links and it will rank at the top.
https://en.wikipedia.org/wiki/Henry_Beard
(I have several of his humorous Latin phrasebooks.)
We have low-quality content generation not because of Google but because of the low cost of publishing (which the author even mentions). It's exactly why we have email spam: it costs nothing to send. To repurpose a Chris Rock bit, "if sending an email cost $5,000 you'd have no more spam email".
What's the alternative being touted here? No Google? Making things harder to find? Seriously?
I'd say a far bigger issue is people sharing content from and to people who think the same, creating these myopic echo chambers of self-reinforcing beliefs.
Google may do a questionable job at filtering out provably false content but people are way worse at that.
Within such a system it's too easy to foster fear, anger and hate and to propagate provably false information. Anti-vaxxers are just one such group that seem to thrive in this informationless world.
There are also other factors. There is a vast decrease of internet's signal-to-noise ratio due to low-cost content you mentioned. For a while we could rely on google to get us through the jungle to useful information. Now every SEO is targeting google search. Most other search engines use google's results, so no help there.
Blaming google doesn't help, their interests are not aligned with user interests. Author is actually criticizing us and our dependence on this one-size-fits-all source of information. The alternative is building and adopting better alternatives to google search.
That's like saying Gutenberg ruined books because now I can find books full of BS.