... or use Firefox (Tor Browser after disabling tor) + uBlock Origin, and avoid all the unwanted crap most websites throw at you today. And train yourself not to automatically search for everything on Google. Instead use IMDB, Wikipedia, Stackoverflow / Stackexchange, Reddit etc. etc. by adding their search to your browser ( https://superuser.com/questions/7327/how-to-add-a-custom-sea... ).
Okay, but iris does have two meanings and sometimes you need to be specific in your queries. Your original query would not have been perfect if you were in fact wondering about the flower. "what is an iris anatomy site:*.edu" is enough to bring up this as the first result:
Sidetrack, but I can recommend the Firefox/Chrome extension "Don't Track Me Google" which cleans up Google result links to be the actual URL, without Google's tracking redirect.
That's google's personalisation. I like freaking ppl out when I just ask them to google a famous politician's name and compare the results. Some times they get similar results on WiFi and very different on mobile data afterwards.
I had to look up some govt pdfs and tables recently and the government sites hosting the files hardly ever show up on the first page on Google anymore. It's a sad story.
At this point I actually think it is. I believe that DDG does tweak the result they get from Bing, and mix in a few other sources. Still, even if it is just Bing, then yes, Bing is now better than Google.
Google is so unbearable because they are no longer interested in you finding the answer to your query; they're interested in selling your query and its context to the highest bidder. That conflict of interest manifests in what Google has become.
Got me thinking - Google has became what it is - this colorful commercial ooze - by parasiting on the pre-Google web. Is there a search engine that searches sites without any Google code in them?
Not TV. It's almost exactly like Altavista, Lycos, Yahoo, Excite and other "search portals" of yore. The thing they swore they would never become, yet here we are now.
I agree. What the search space needs is a different business model. I have no idea what it could be. I'm quite sure a subscription model would fail. Micropayments have not taken off. How can we fund meaningful search divorced from an ad business?
Every month I see my user experience decline with Google, in so much a paid search solution would be a blessing.
Imagine a search engine that truly is private, containerized search and does not need to sell you anything. It only wants to serve you true relevance and accuracy. That engine also skips over all monetized sites that serve more than 5-10% ads. Prefers cookieless sites. That search engine would be bliss.
Working on much of this at Neeva. We have a free tier, no ads cluttering the results. Just focused on the user experience. We also offer a premium tier for those who are keen to support us.
We have been working on exactly that at Neeva and the response has been very positive in terms of growth and adoption. Neeva offers a freemium model (free basic and premium for $5/Month which includes paid versions of VPN and a password manager), you can connect third party apps like dropbox and email putting all your personal docs in one search along with websearch, both versions are ad-free and private, it's available to anyone in U.S. and we are building our own independent stack to make the search experience unique and better.
Just 1% of Google's market might be enough to sustain a non-bloated, VC-free company and make a good product.
The Bloomberg Terminal is extremely expensive and yet companies pay for it because it makes their traders' jobs so much more efficient. I don't see why companies wouldn't do the same for a search engine that filters out all the bullshit so their employees don't have to waste their time wading through it.
Categorized search results are also data. Bloomberg is no different to Google in this regard, it doesn't generate the data, it just fetches and aggregates market data from various sources which anyone could technically do with enough effort and yet people pay for it.
Google has 4.3B users, so 1% of that is 43M. With a $5 subscription, Kagi would have $215M in monthly revenue, enough to pay for 14,000 developer salaries.
A more reasonable target might be 500k users at, let's say, $10/mo.
You are right. And the actual number is lot smaller. We just need 0.001% of Google users to be sustainable with the current team size. (or about 50,000 users out of 4.3B Google users). And if you find 50k users, you will find 100k. First 5k is tricky!
I'm a Kagi private beta tester and love it. Will become a paying customer.
One question: I do remember a page that told me how much I searched and what that would mean once Kagi becomes a paid service. I can't find that info anymore. In light of the mail us Kagi users got today, does that mean that it's going to be a flat rate for starters?
Yes that is what we intend to try ($10/mo flat fee for unlimited). Most users told us they are more comfortable with this model vs pay-per-use which is why we removed the "Consumption" page as is not relevant in this model.
We do not yet know if the flat fee will be viable for the company and we hope to learn that very soon.
Another user here who see's huge value in just renaming the consumption page to a history page. Would be VERY handy! Especially if I could restrict a search to my own search history.
I've been using Kagi for a month or two now. Love it and very excited about your work. Unlike alternatives such as DuckDuckGo, it has easily and painlessly replaced Google Search for me. Can't wait to see more from yall
At this point I’d pay for it. Every search I do these days is a fight with Google to find pearls buried in the overwhelming muck. I’d sincerely like to see how search curation would change if the intent was to keep me as a paying customer, not to show me more ads.
I am using their beta search service. It does seem nice enough. I am not sure yet how it compares with DDG, but I am probably going to sign up as a paying customer.
Give Neeva a try (disclosure I work with them). We offer a freemium model (free basic and premium version for $5/month that includes paid versions of VPN and password manager), it's ad-free and private, it's available to anyone in U.S. and we are building our own independent stack to make the search experience unique and better.
Just a user of Neeva here but a happy one. Also very excited to hear about the work toward an independent stack! Enjoy the spaces and immersive search UIs for various types of searches. Neeva make search fun again. ;)
Why would a .edu help you learn to fix a tire? That seems like a YouTube question, not something a school would teach, and if they did, not something that would go online.
The article we're discussing is about limiting your search results to ".edu" domains to remove bloat, and gives an example of an https://www2.latech.edu/~bmagee/303win97/Group3/2245.html which does have this information on changing a tire.
Too bad .edu seems to be restricted to US-based schools. There's plenty of great stuff hosted by universities in other countries but none of them are on the .edu TLD.
There are other national domain schemes with educational subdomains. 'ac.uk' and 'edu.au' being two of which I'm aware.
A listing of top global educational institutions would only have a few thousand entries.
It's not without some irony that I note that the early, and much belated, Internet was largely edu domains, along with a handful of tech firms, government agencies, and military entities.
I've tried the same and don't find the results super similar. Both search engines seem to think that you're trying to repair the old tire. However the second Google result is goodyear.com which, while trying to sell you tires, does explain how to use the spare:
On Kagi you have to go until the 10th result to get something from Bridgestone, which explains the same thing while being less aggressive about selling tires:
There are many ways laws could be written to would effectively ban the current online advertising business model. One could be to require written permission each and every time a company wants to transfer user data of any kind to anyone else. Google could try keeping all Google site collected data internally but all analytics data collected from non-Google sites would be forbidden without written permission for each and every instance.
Every site would have to run their own data analysis and sell their own ads. Advertisers would be unlikely to take Google or anyone else's word, so audits would be necessary but difficult to conduct. It wouldn't completely end online advertising but would turn it back into something closer to the old traditional model where those selling ad space had a handful of semi-generic personality models for their readers/users and advertisers would select one that best matches the profile of their target customer. It destroys the economics of centralized data collection.
There are many other laws that could accomplish a similar end but through different means. The biggest barrier would be getting politicians to not care about the demands of huge tech companies that dump endless amounts of money into lobbying and campaign funding.
It doesn't matter what new model you come up with. As long as capitalism is around and pushing for ever increasing profits this kind of behavior will always be on the menu.
> How can we fund meaningful search divorced from an ad business?
The only way you have not listed is gov't funding. And i dont think i would trust a gov't run search engine either. May be a well meaning librarian institution could make it work.
> I'm quite sure a subscription model would fail.
this hasn't really been tried yet - it could work, who knows?
Hi there, you should give Neeva a try. Disclosure, I work for them, but we were the first to introduce a subscription, interest/growth has been really strong so far. Neeva offers a freemium model (free basic and premium for $5/Month which includes paid versions of VPN and a password manager), you can connect third party apps like dropbox and email putting all your personal docs in one search along with websearch, both versions are ad-free and private, it's available to anyone in U.S. and we are building our own independent stack to make the search experience unique and better.
There are two versions 1) Free basic which is free -- full search, ad free, private, and personal connectors. 2) Premium for $5/Month includes all the free basic + additional connectors and paid versions of VPN and password manager.
> We plan to fund Andi through a freemium business model, with free anonymous search for everyone for ever, and paid plans for professionals and businesses, including APIs and using Andi for corporate information, with supplemental revenue from anonymous referral link attribution. We're focused on building the product right now, and will figure out the details once we're further along.
> We are committed to staying 100% advertising free. Commercial considerations or partnerships will have zero impact on our search results or recommendations. Our recommendations are always made based on the best results we can find for our customers, and are never subject to commercial influence.
> By sharing search revenue with content producers and media companies that join our network, our mission is to provide funding to great content and reduce media's reliance on invasive ad-tech. Contact us at info@andisearch.com to get involved!
There are only 2 employees[1] of the company so there is a low amount of revenue they need to be self-sufficient
Why stop there? Social media suffers from the same conflict of interest as search. Heck, even mainstream media online has this problem: the public's confidence in journalism has plummeted due to the rise of clickbait, fake news, and sensationalism.
What I really want is a decentralized, non-commercial, open source web. As part of this I would like to see search and communications moved to the client, rather than the cloud. This seems like it would take a huge effort to build, however, and even more monumental effort to bring people into the network. So it feels like a pipe dream at this point. I do think there are a lot of other people out there craving for something similar, based on nostalgia for the old days (90's and earlier).
> based on nostalgia for the old days (90's and earlier).
Indeed. The internet in general and the beginnings of the web were a very different place back then. I remember about 1996 or so seeing a URL on a Pepsi can and thinking, "oh crap, it's over."
I've been sort of thinking there really ought to be a sort of open source manhattan project to disrupt the search ecosystem and offer real competition to big tech.
There's a lot more to search than just search in the traditional sense, and I think multiple cooperating services offering small complimentary functionality slices could probably offer reasonably similar capabilities (save for like image search, maps and translation) with a significantly smaller hardware footprint.
My own search engine is fairly janky, but a lot of its problems are understood and could probably if not be fixed, at least mitigated. That's one functionality slice, searching documents. You could have a slice for forums, one for reviews, one for high value sites like stackoverflow and wikipedia, one for facts, one for popular links in social media and news, one for source code, one for mailing lists, and so on.
As a whole, I think you might actually be able to put together an actually useful information gateway this way. Would be a lot of work, but I think the amount of work probably exceeds its difficulty.
Do we really need it to be a business? Does it need to make money? Torrent communities and DHT don't make any money and yet they're pretty good. Would it be possible to build a decentralized solution around finding things on the internet? Something with trustworthiness ratings and user controlled blacklists perhaps?
Why does web search need to have a business model at all? Why can't we just provide the Library of Congress funding to run something like it? Google got its start as a library project, after all, and it's the natural place for such a service.
Corollary: Web page creators are no longer interested in answering your query; they're interested in getting you to 1) view their page for ads 2) click on their affiliate links 3) buy the product they're selling.
Google has an impossible task in trying to answer a query like "Portable air mattress reviews", because 99.99999% of the results are SEO-optimized crap
We don't know if the task is impossible because Google has no incentive to do it at all, so we can't assume they are really doing their best. They created this situation and they profit from it just the way it is.
Build a better mouse trap etc, if the task was possible, DDG / Bing / Whoever could absolutely dominate the search market by providing a ”no crap results” option
They would only bother to try if they could monetize it, and the only model for that which won't create the same incentives is paid user subscriptions. The market for that may just be way to small for the big guys to consider.
I paid for Which? which is the UK equivalent. Would've subscribed again when I needed it (and there's been a few times since) but their dark pattern upon unsubscription put me off from going anywhere near them again.
I also spend 50£/month on Patreon to support the creators I watch regularly.
The top two organic results for that precise query are pretty informative articles from Wirecutter and Good Housekeeping. Unless I'm misunderstanding and you are expecting something better than this: https://www.goodhousekeeping.com/home-products/a25473658/bes...
I still use Google every day. I was really hoping this advice was for real. Like maybe how to filter your Web results by size since they started off with example of a 9 MB website maybe you could filter your queries down to websites that are 2 MB or less? Since I use pi.hole and the first results are ads my top links are always dead. It is time to switch - Is DDG the best alternative?
DDG has been my default search engine for probably ~7-8 years. In the early days it wasn't that unusual for me to still do some searches on Google for better results, but nowadays that's extremely rare (probably partially because DDG has gotten better and partially because Google has gotten worse). But I still hear people complain about DDG's search results quality, so I think it really depends & YMMV, but it's a pretty low-effort thing to try for a while, so maybe give it a shot?
My loosely held theory on the wide variance in people's judgement of DDG is that some people really like/have come to rely on how much Google personalizes results based on all the other data they have on you, even if they are in the "Google has gotten bad" camp. DDG doesn't do any of that: it's much more like going back 20 years where you have to put all the context you care about into a query, you're not searching "in a bubble". For me, that's a very positive feature, but others may find it frustrating and not even be able to articulate why they're not happy with the results. DDG does do a lot of the nice "enhancement" stuff Google pioneered long ago, like embedding the intro of the Wikipedia page in the sidebar for a query with a strong match, e.g. a person's name. So for me DDG is just the right level of "smart": it understands common queries well enough to immediately highlight useful details, but it doesn't try to be so smart that you as a user stop being able to understand what it's doing under the hood.
I think a subscription model could succeed for a niche group of people if the price was right, and the quality was good enough.
Also, I think Google could make a ton of money if they sold one ad per language at a time. As in, for the next 5 minutes the ad for all English speaking users will be for coca-cola, and then it will be for boscovs. Just like TV ads.
That is to say, a search engine could sell advertising without betraying it's users or compromising its own quality. It is possible, but would need constant vigilance against the temptations of greed.
$5 / mo for a premium Neeva subscription or just use the free tier to experience ad-free search focused on the user. I left Google to join Neeva so I could be part of building a better search experience for people. Check us out. Would love feedback.
There are only two choices. Either Google are no longer interested in finding the best answers to your queries. Or as you point out: They don't care, that is no longer what they do.
I cannot imagine it being easy, but I find it hard to believe that Google aren't able to weed out the worst offenders. For a large number of queries the results are all SEO spam. Try searching for anything related to health and get a useful result. It's no longer possible.
Google has SO much money, they could probably afford to curate the web and white-list websites one at a time and still make money. Personally, I think Google should get out of the search-indexing market entirely and rely on website submissions only.
At this point, it's no different than an appstore in terms of volume. Ironically that's where we're heading with most websites being "3.0" and just SPAs and now PWAs even.
> Try searching for anything related to health and get a useful result. It's no longer possible.
Medicine researching on the internets always was among the hardest topics. And anyone who really understands Medicine has to write on any advice not to consider it as a real medical advice... I agree that google has degraded down to unusable in comparison to non-global searchable websites (like SO) but Medicine is very bad example to your point.
At the same time, it would be the easiest to solve with something like a "white list" (I know, not a good term to use these days). With one stroke you shut out all the bad actors.
In fact, I wonder how far a search engine could go with a curated list of "good actors" you exclusively serve up results from. Add a "search everywhere" link for people that prefer SEO hell.
When Google started, the web was honest and innocent. People linked to the pages they liked, and Google could crowd source those links and raise up the most popular pages.
Now most of the web is lies meant to confuse search engines. Figuring out what pages are good is no longer a feasible task for software.
> Figuring out what pages are good is no longer a feasible task for software.
Of course it is. It shouldn't be at all difficult to train a large natural language AI model on SEO spam pages. But nobody will do it unless they can figure out a business model that makes it pay.
Or a non-profit organization is formed to do search spidering for the benefit of the whole world, much like Wikipedia or PBS.
I don't doubt the narrative as you relay it, but I question how hard this can be for one of the largest corporations with one of the largest engineering pools in the world.
Some years ago there were some serious Warnings issued from Google: Make your website fast and small! Popups will be PUNISHED! as will mobile-unfriendliness!
But in hindsight, this was a ploy from them to try and upsell AMP, which was in a lot of cases heavier than non-AMP websites, and it was a means to keep people inside of their ecosystem - and inside of their advertising network, because not all websites Google linked to had Google adverts on it.
They've given up on AMP. They've given up on improving search results quality. Dare I say they've given up on improving the web? I mean they're still putting some money into Chrome and HTTP/3 and the like.
> I mean they're still putting some money into Chrome and HTTP/3 and the like
Looking at the misnamed "Privacy Sandbox", Chrome is not just there to improve the Web (and therefore google usage), but also to control it (and steer advertisers to Google even if it undermines the Web as a whole).
I use Safari for everything except front-end web development on MacOS now. I started because it's more energy efficient but I got quite used to the UX and now I'm glad I switched.
I'm hopeful there are a lot of Mac users like me out there.
I'm using Google less and less in favor of Youtube. It's not always convenient to get answers in video form, but I'm finding it's better for more involved answers. Also, google is just littered with Answers/Quora/StackOverflow/etc dodgy clones that make searching questions a nightmare.
While I overall agree, I ironically find the search engine on YouTube to be somewhat limiting when I'm looking for specific and/or fresh things...
For example, I wanted to watch a bunch of videos of Mark Zuckerberg speaking from before 2010 and the YouTube search engine only showed me a limited amount of things I had already watched— it even had unrelated suggestions below those results based on my viewing habits.
So I went to Google and searched for videos and interviews of Zuck and found a bunch which were not easily surfaced on YouTube itself.
Same thing with Ina Garten/Barefoot Contessa cooking videos.
Now that I've found some fresh videos, the homepage recommendation algorithm has gotten better at surfacing new content related to these topics.
They also have no focus, as you and a reply essentially say.
For example, all the stupid aliasing of search terms, and dropping of search terms.
This is helpful if you want to optimize for "sorta close, sell things!" such as "maytag A4453 fix timer", but you want advertisers to latch on to maytag=washing machine, and ignore the rest.
And it also helps when websites with google ads on them them get higher SEO, because suggesting that site might get a click, and profit!
EG, suggest a blog/spam site, because google ads are on it.
And it also helps with google voice search! Did the speaker mean here/hear? Did they mean Danny/Daniel/Danson when they said Dan?
Google has too much genericization in its search, and it is for profit naturally, but I think it is lazy profit, and it is becoming worse yearly.
It is sad because I spent nearly 10 years at Google and nearly all the people I met were well-meaning, seemingly harmless, engineering-oriented, and data-driven. But the configuration of people that makes up the superorganism that is Google nowadays is driven by both local incentives and a broad top incentive to grow. Scale to more computation, more storage, more bandwidth, and with lower overhead--morally, those are neutral to positive, IMHO, but they are put in service of the real goal at the end of the day: swallow more money. More revenue, more markets, more ads, more engagement, more watch time...muscle out the competition, dominate. 20% YoY growth for nearly 25 years running, and suddenly you get huge, scary numbers. And you have to do insane things to keep that exponential curve from regressing to an S-curve like the laws of physics dictate it must.
If you believe they are neither vices as well, then I can see your point — they just are.
Perhaps this is the point you are making: if one is data-driven and engineering-oriented without regard to the consequences, that could be seen as a vice.
Despite the amazing engineering/science, it is hard to come to peace with von Braun or Teller (just to pick two examples).
Yes exactly. But until this is clear and accepted, these terms are apt to be misdirections—for example here, in a discussion trying to understand how things went so wrong at a leading tech company.
Hopefully we can get to the point of putting purpose (I think “mission” is a bit too hackneyed and neat to work here) first in discussions of culture, whether corporate or engineering culture.
I think most of us want to view "engineering culture" as just problem solving, amoral. If these companies we worked for didn't have such an outsized effect on the greater world we live in that would probably be a reasonable way to view engineering culture.
The overall problem -- and this is not specific to Google -- is that managers at all levels are incentivized to grow head counts.
Google has at least 10x as many employees as it needs. That's based on decades of revenue being decoupled from costs, and from monopoly profits. It's the double-whammy of being in tech (where costs and revenues don't couple since incremental costs are close to zero), and having a super-profitable monopoly.
You can't fire 90% of the workforce; even morale and culture problems aside, at this point, the organization is structured to need too many people. On the other hand, organizing over 100k employees turns into this superorganism problem.
At >100k employees, the influence of anyone -- up to and including the CEO -- is very, very limited. Dynamics take over.
I've been bearish on Google for a long time, but I haven't shorted the stock. Market irrationality can last a lot longer than my wallet. Fundamentally, though, one could muscle through building a new, better Google for much less than the valuation of Google.
Google could also be worth a lot more with a breakup. There is negative synergy between, for example, advertising, Google Workspace, Android, hosting, etc. If those were independent businesses, with partnerships around things like data sharing, the total value could be much higher. Alphabet was a move in the right direction, but the move should have been much deeper. Right now, Alphabet is Google + a bunch of tiny startups.
> The overall problem -- and this is not specific to Google -- is that managers at all levels are incentivized to grow head counts.
No one is incentivised to grow head count. They are incentivised to do more, produce more revenue, and expand the reach of the company; most of these goals require the application of more smart people to a problem. They are not getting a promotion or salary bump because their group increased from 200 to 300 people over the previous quarter, they are receiving rewards for being able to effectively direct those people in order to achieve some specific goal. Increasing head count by 50% while delivering 250% growth in revenue/engagement/view/etc is what is being rewarded.
Respectfully, I think this is a little bit naive. While the job ladder doesn't explicitly specify org growth as a requirement to get ahead, leading more people is a classic way to get ahead as a relatively junior manager. If a manager's peers are all more senior but are leading similar sized or smaller orgs, it will seem natural that the junior manager should get promoted to the same level. Building orgs is a classic way for managers to get ahead.
Actually delivering value is a huge advantage too, but as far as I've seen is not prerequisite for getting ahead.
> Actually delivering value is a huge advantage too, but as far as I've seen is not prerequisite for getting ahead.
Appearing to deliver value is strongly advantageous. Delivering value helps you do that. Other methods:
- Good salesmanship (e.g. making easy things look difficult)
- Stealing credit (can be personal, or e.g. for market conditions)
- Lying (e.g. cooking books)
Delivering value is probably the easiest way to appear to deliver value at the IC level. The higher one moves in the corporate ladder, the more important the others become.
- If I have a job at Google as an IC, I can look for IC jobs.
- If I have a job managing 10 people, I can get a manager-level job.
- If I have a job managing 100 people, I can get a director-level job.
- If I have a job managing 1000 people, I can look for a VP-level job.
- If I have a job managing 10,000 people, I can look for a C-suite job.
Average tenure in a position in the tech industry is around 3 years. No one will know or care what objectives a Google manager achieved, or whether those were hard or easy to achieve. If you want to grow rapidly in your career, the things which matter are:
- Connections, network, and references
- What shows up on an interview (e.g. self-improvement)
- What shows up on your CV (e.g. how many people you managed, or the brands you worked on)
The place techies get stuck in career growth is by trying to do the "right thing." By the time you're trying to rise to the top of a corporate ladder, the competition is extreme, and the people who succeed play the game as optimally as they can.
I might be able to look one level up. An IC might be able to manage a small team. However, your odds of having an IC move directly into the C-suite require nothing short of a miracle (a Nobel prize, being the author of Linux, or something similar).
This is why I switched to Neeva, I want my search engine to make products for me that help me find the best results for what I am looking for. When an advertiser is the customer instead of the end user the entire product road map is just trying to take you back to that search results page as many times as possible.
Neeva also lets me index my personal documents and includes those in search results as well. They are an early startup but I am excited about what they can build when they are not beholden to advertisers.
Another Neeva user here... yeah, the personal document indexing was far cooler than I expected. I held off doing it for it a while but glad I gave it go. Just the GitHub indexing alone is cool. Can do some fun hacks with that too. I think they need to surface these personal results even better in the web results than they do (and I hear that is in the works?? hope it is).
OT, but Amazon is in the same boat. Their problems with counterfeit merchandise and fly-by-night Chinese junk manufacturers have been going on so long that it's become clear they don't want to fix the problem because it's now a major revenue source for them. Apparently someone at Amazon decided the Aliexpress marketplace was more lucrative than selling good products.
They don't care about fixing it because people still flock to them.
And who can blame us? Other sites still suck worse. I can't believe how bad walmart.com still is - the user experience is night and day. Walmart - the retailer embracing computerized tracking of spending by credit card number and other advanced data analytics a full decade before Amazon was even a glimmer in Bezo's eye took forever to realize the power of using their stores as local fulfillment centers? Provides a 3rd party marketplace that's less reliable than craigslist? Who the hell is in charge of online operations at these big companies and how do they still have jobs?
Then again just look at Sears - the company that literally invented mail order *in the 1800's* and how they completely missed the Internet - the ultimate mail order environment.
This makes no sense. You make it sound like Google has no financial incentive to fix this problem, which is patently false. Google isn't selling the organic search results, which is what this article talks about.
The problem is that Google is engaged in a war with SEO optimizers, and losing (or at least, barely breaking even). No business model will fix that. It's a hard problem that any successful search engine will struggle with.
"Google" is 100+k people. "Google" doesn't respond to incentives; individuals do.
A typical employee at any large company is incentivized to:
- Maximize their salary
- Maximize their future earnings potential (e.g. CV)
- Have fun
- Have work-life balance
... and so on. That goes all the way up to the CEO. Pretending a corporation thinks and responds to incentives is folly. Running a corporation like Google is like herding cats. If everyone marched in the same direction, Google would cut through SEO optimizers like a chainsaw through butter. There is no wayh to do that with 100+k people.
Google fundamentally has no financial incentive to fix any problem, since organizations aren't intelligent, sentient beings which act on incentives. They have dynamics from the interactions of 100k+ individual agents.
There is no magical fairy which makes large organizations act in their own best interests or even try to survive. The only reason they do survive is that's as true of their competitors as themselves.
Did you write that, or did your 85 billion (approx.) neurons?
Corporations are intelligent, sentient entities which act on incentives. They don't have a conscious sense of self, but then neither do many natural organisms. That doesn't keep them from responding to their environment as if they're aware of it, and acting in incentive-driven ways.
Corporations seeking to maximise profits are exactly like humans attempting to maximise income. The problem is that corporations - like humans - have the wrong incentives, further diminished by limited predictive ability and poor heuristics.
So maladaptive behaviours emerge and stick. Both humans and corporations find a nice local minimum and don't move from it.
More intelligent behaviour would be able to predict and avoid existential threats without having to experience them first. But it's a question of predictive ability, not a binary absence of all intelligence.
So it's not that Google can't do better because it has too many employees. It can't do better because its cultural heuristics - which include it internal culture and the external culture it operates in - don't allow it to.
A megacorporation doesn't act like my 85 billion neurons. A corporation acts like a forest of trees, or at best, an anthill. Corporations (and simple organisms) respond to their environment, but not necessarily in incentive-driven ways. Often, corporations move opposite incentives not because of some local minimum or whatever, but because it, for example, personally benefits the CEO, a few board members, or a group of employees.
I can set up "incentives" for an anthill, in the form of a mixture of sugar and borax. Emergent behavior takes over.
The behavior of corporations, at the scale of Google, Facebook, or similar, is almost entirely emergent dynamics. Same thing for countries, for that matter.
But does Google care about the organic search result quality?
For many search terms, the entire above-the-fold and sometimes multiple scroll heights are filled with:
- Ads
- Info cards
- People Also Ask
- Map/Business info
- Videos
- Related Products
- More Ads
The organic search results start after that and on mobile they may as well not exist.
If everyone is clicking the directly or indirectly sponsored/algorithmic content then what incentive does Google have to give you good, straightforward links to relevant webpages?
I am not a Googler, but I know some Googlers (including at least one that worked on search) and... Google doesn't seem to work like that? The search team doesn't make take direction from the adsense team.
I think they are genuinely struggling in an adversarial environment. Imagine every hacker in the world specifically focused on your system and the payoff is millions or even billions of dollars.
I think that’s totally reasonable as far as the organic results go.
But clearly there are also a lot of motivations to de-emphasize the organic results on the search page in favor of ads, “algorithmic answers” that give you an answer without clicking away from Google, and redirects to Maps, Youtube, and Shopping which are revenue-generators for Google.
Maybe that also comes with a budget de-emphasis for fighting the war in the organic results vs surfacing relevant content from these other sources.
While obviously not universally true, this bothers me about a large swath of current UX culture and it’s pushing me towards other types of design. Anyone who’s heard a UX conference keynote in the past ten years would think UX folks go into work every day making unbiased data, using it to identify users’ genuine needs, and championing those needs at every stage of product development. Sure, the suits call the shots, but does our software landscape indicate anybody really does that?
Many product folks just don’t seem honest, even with themselves, about what benefits users. Do you really think your users benefit from ”more relevant” advertisements? Because your users don’t. Are new services and content so beneficial to users that they’d forfeit their privacy to fund it? If so, why obfuscate your having made that choice for them? Why not make it opt-in? Does that dialog box popping up just at the right time really give that overstimulated and frustrated user a choice? Either answer those kinds of questions honestly or admit that you’re just finding the smoothest path to maximize revenue. The hypocrisy is infuriating.
This is nitpicky but the tire changing page doesn't actually fit your rational for choosing .edu pages
>Professors are paid generous salaries to share knowledge with the paying customers of the University (students). So, let’s find some paid unpaid University knowledge on how to change a tire by using our new trick!
But the bottom of the page says
>Prepared by: [students] For Dr. Bruce Magee's English 303 (Technical Writing) Class, Winter 1997-1998.
The professor is actually an English professor and the page was written by students as an assignment. I looked into this because I was wondering, "what would a professor be teaching that he's writing notes on tire changing?". There's probably some professor out there that does that in an academic context, but there's probably a lot of things that no professor actually gets paid to write about. Maybe they would write about something miscellaneous like that anyway and put it online, but they would be doing so out of the culture of web savvy professors having personal pages where they upload stuff they think is helpful.
Yes, it matters - a writing assignment would be graded on writing quality and style. I would assume the writing professor is not a tire changing expert. You're there because you don't know how to change a tire, so there's no way to know if the information is correct.
It's certainly better than nothing but there could be specific gotchas for your situation. Most common would be needing certain tools or techniques for your model of car. It'd probably be best to either look up the manual for your car online or find a youtube video for changing the tires for your model of car.
It isn't nitpicking at all, I had the same question and glad you uncovered this answer. I think site:*.edu is just one tool in your search toolbox that you should know about, but I wouldn't expect it to work for every topic. Still, I had never thought of it so I'm glad I read this article.
This has recently been my desperation move when e.g similar bugs to what I'm facing were solved a decade ago (but were irrelevant) were coming out as top results rather than bugs for a recent version
Consider using alternative SE's like you.com or neeva, they give you the ability to rank sources and e.g. set Reddit as a preferred site so you don't have to type site:reddit.com every single time. Works great and saves time.
That's how a search engine should work: it returns only what you asks of it, knows how many results there are, can be forced to use your query verbatim.
> Oh, and also, Miller is hiring if you’re interested in joining the team. I want to change my tire! Get off my back, and stop trying to make money off me for one second while I solve this tire crisis.
So, you want someone to help you and you are pissed when they don't want to do it for free?
No. We are not looking for hire someone to do a job. We just want to be able to search for content online without middlemen interjecting themselves in the hopes of monetizing the interaction.
The millions of people that write and produce content without being tied to this Protestant mentality that something is only worth doing if it is profitable.
In fact, I'd wager that if we had a way to filter for content that is written without the goal of being monetized, we'd have a better web and us consumers would be more inclined to reward and give back to the creators generously.
The Kagi search engine which is close to entering public beta has lenses which tailor the search experience. There is a non-commercial lens. The lenses are basically on or off all the time and affect the overall search results.
Where did you find this feature? As a beta user, I have a hard time finding documentation for the features in Kagi. Ironically, searching about Kagi in Kagi is not helpful.
I thought this was an interesting topic, so I read a bit on Protestantism and Protestant work ethic and what makes things worth doing. [1, 2] The interpretation that something is only worth doing if profitable seems somewhat uncharitable but does seem to be possibly rooted in truth.
The irony is that the garage is doing a fantastic job up until that point.
If someone is so clueless about changing a tyre that the best place they can start is google, then organically taking them to the garage's page is a win. Maybe they'll find the information useful, maybe they'll decide they're out of their depth and call the number on the screen.
For them to show up as the first organic result is a huge win. All they have to do with that win is not screw it up. And 9 meg of trackers, a page full of popups, and pointless CTAs (do you want to hire someone who can't change a tyre?) is screwing up that easy win.
You don't have to be scummy to turn search results into profit. That garage came so, so close of being a great example, before they threw it away and made a bad example.
I came to find out how to change a tire, not to get a job in dealership. If the page said "we have 5 different guides on how to change a tire with different level of details, so you can choose the one you preffer. Just pay us 5$ to get download link" without forcing an account, it would be much better.
The problem is that there are people who have published free, no-bullshit guides on how to change a tire for the sake of spreading knowledge, and these results should be prioritized over businesses who have a profit motive.
Ironically, it's the ones who were doing it for free who tended to excel at providing information, and the ones who're busy monetising their presence who've revealed the Sadim Touch, turning all that's gold to shit.
There are countless small(er) subreddits where a Google 'site:' search works wonderfully. Often, I don't specify a subreddit at all if the query itself is narrow enough.
Reddit has very strong biases and its own bubbles as well. Don't expect to be exposed to a wide variety of viewpoints there, but what appeals to a very narrow demographic (18-30 year old low-income white progressive American males)
Unfortunately, I find Reddit results to generally be of very low quality. For tech related searches, stackoverflow and direct links to documentation are far superior. Where reddit is occasionally useful is when searching for things like "what's the best service for X?", but even then I find myself wading through many posts and many comments on those posts before I come across a sliver of useful information.
There's a Firefox add-on that allows you to block specific sites from Google's search results... not every practical for the whole Internet but it is for the worst offenders. (https://addons.mozilla.org/en-US/android/addon/ublacklist/)
To be fair my result in Germany is a very good article that even has a video and a good schema. Before the result I even have google's automatically generated summary of the page that's quite good to understand if the page's actually useful.
I hate Google Search for many reasons, but this example is really ill-conceived.
There is no perfect answer for a query.
There can only be contextually meaningful answers.
They can be from highly relevant to far off.
It is very hard to prove a search engine as bad in general.
It may not work for you. But it may still work for a lot more.
Google has indexed the entire Web. This is as large a span as one can take.
It's the aggregate result of the sort of hiring they do in there. Take for example they rejected the guy who created homebrew because he couldn't invert a binary tree to the satisfaction of some jobsworth in there (1). It's jumped the shark and gone full on corporate bs, not because of any changes to what they do, but as a result of who's making the decisions in there these days.
I feel like Google could filter out a lot of SEO spam by simply ignoring or downranking heavy sites. Everything that loads more than "n MB" is probably bloated and made to trick you into buying something or generating traffic without providing value.
Another method could be to analyze how many external requests a page makes and put that into the calculation.
Of course, there are exceptions, but I think this might help.
Downranking these sites will overwhelmingly mean downranking those who spend money on Google ads. It will not happen because it is exactly contrary to Google making more money.
Of course Google has PageSpeed, which supposedly influences rankings by downranking sites that are very heavy to load, so I mean, that's an aspect to take into account, but I still believe the above to be largely true.
Thought the same, but it would be unfair to web sites that are actual web apps and are JS-heavy.
Google is hopeless anyway, so here's something for the next search engine that wants to bring back the Web: downrank by the number of 3rd party domain requests, attempts of tracking, and possibly also popups.
If you show an example of something that pops up right after the initial load, and that a user actually desires, we'll think of ways. I can't think of any right now. Things should pop up in response to a click, generally.
The presence of ads, analytics and affiliate links are a good proxy for "spam site" and is very hard to game, so downranking them is a trivial solution.
Google will never do it because those ads & analytics can be Google's and benefit their bottom line.
I was confident they already included page speed in their rankings, but who knows what the status of that is? After some (ironically) googling, I've found three blog posts from May 2020 [0], November 2020 [1] and April 2021 [2] announcing it, the last one saying it went live in August 2021. But I was sure they talked about it years ago, hence the PageSpeed Insights and Lighthouse tools.
Those sites generate plenty of value ... for Google. Even if they're not showing Google adverts, those sites normalize the idea that "websites have adverts", so users don't question it on other sites.
If Google started returning sites with low numbers of adverts first then users would start demanding more sites with fewer ads. Google really don't want to show fewer adverts.
People literally curate lists of websites that are only spam, and provide no value in search results[1]. Even if Google only cared a little bit about blocking the worst 1% of websites they don't already block, they would simply copy these lists, or use them as a starting point. Let alone devoting actual resources of their own. But they don't even do that.
Ok. I bit the bullet and typed exactly those words: "how to fix a flat tire".
This is what I got in return:
Row 1: 2 buttons marked "on a bike" and "scooter"
Row 2: single scrollable row of images of tire fixing kits with the prices.
Row 3: step by step recipe on how to fix flat on a car (7 steps long) followed by the link to the corresponding article and small image on the right side.
To me it looks straight to the point and one can't bitch about single row of ads of a relevant products.
lol @ musk thinking cookie settings should be in the browser. Does he not know that they used to be until a certain advertising company took it out of their browser, and Mozilla then followed?
The mentioned queries provide quite good search results for me
The first link clearly explains how to fix a tire and right underneath the first link, there's a link to Youtube video with described sections which you can click on immediately
The sites are full of tracking and ads, but that's hardly Google's fault imo
Seems like a long way to go to you know, not just switch search engines. Google's had a nice run so did yahoo, hotbot, excite, alta vista... the question isn't how to fix Google. The question is...
I switched to using kagi.com a while ago, and I've never looked back. I've tried to use duckduckgo before but just ended up using '!g' most of the time.
Kagi is in a whole other league. It's on par with, if not better than google search. I highly recommend it. When they introduce the paid tier I'm sure as hell going to subscribe to it. At this point I would trade my Netflix subscription for it.
There’s no way to test without giving your info away and taking a long survey from what I remember. It’s kind of like posting Facebook links I suppose.
Our goal at this stage (invite only beta) was not to get users who just want to try it out, but who actualy wanted to contribute by reporting bugs and suggestions as beta-testers. This is why we have a survey form, as completing it signals wilingness to invest time in the product. Soon (in two weeks) we will move to public beta.
Even then we won't have search without an account, among other things because bots would probably kill us.
I've been using it during the beta and plan on paying for it once they start charging. Previously I had tried switching to DuckDuckGo, but found myself using !g half the time because I knew the DDG results would be garbage compared to the Google results.
I haven't found Kagi's results to be significantly better than Google's results, but unlike DDG they're not hugely worse than Google. It's all the extra stuff that makes me want to use Kagi instead of Google:
* Ability to banish sites from results forever. E.g. I never want to see the Concordia Lutheran High School's website when I search for things in the Common Lisp HyperSpec with clhs.
* Ability to weight sites higher. E.g. there are two mirrors of the Common Lisp HyperSpec that appear in results, one at http://clhs.lisp.se and one at http://www.lispworks.com/documentation/HyperSpec/. The lisp.se mirror is hilariously slow for me (5-10 seconds to load) while the lispworks one is instant. Google always ranks lisp.se first which tricks me into clicking it and wasting my time. Kagi lets me weight that site lower so the lispworks site comes first. Banishing lisp.se would also work, but just upweighting it means I can still find it if the other mirror ever goes down or something.
* I reported a minor bug to Kagi and they fixed it within a few days. I doubt that would happen with Google.
* Supports DDG-style bang queries, which I use occasionally.
* No goddamn ads/tracking (I use uBlock Origin, but it's the principle of the thing). I've almost entirely de-Googled my life at this point (Youtube is the only holdout for me).
So it's not that it gives better results than Google, it's that the results aren't any worse and the extra quality-of-life features are really nice.
Basically everything you wrote above is what I'd say - except I haven't submitted a bug report or feedback to Kagi...
Basically, when I used DDG, I found myself frequently using !g to search Google instead. For Kagi, I haven't really found myself doing that anywhere near as often.
So yeah, you have to fill out a survey (took a couple of minutes and I got an invite the same/next day), so it's not the easiest to recommend right now. Two weeks and they go free+premium according to an email today.
I think it's just that I, like a lot of HN, have been getting super disillusioned with Google lately - and most importantly the preponderance of obviously machine-generated zero-value spam pages for everything I search for - so I was desperately trying to find something better. DDG constantly disappointed for my searching. Kagi doesn't do that. So I'm happy. $10/month happy? I'm not sure.
Generally, there's a growing group of people who are feeling about Google the way a lot of us felt about Facebook 10+ years ago - it's time to stop relying on it and maybe cut it out entirely (and yes, I know there are plenty of people who felt this way about Google for years already). So I pay for Fastmail now. I guess I'll likely be paying for Kagi now. Quality is getting expensive, but at least I'm paying with my money rather than my data.
Not only requires a login, but also Joe Regular cannot use it - beta means here closed invite-only, so it's just an internal tool for some group of people. So any news about it should be taken with a grain of salt.
Also, it predicts a video is best to answer this question and gives 3 YouTube thumbnails listed at the top of the results. They all seem generally helpful based on the title.
These top results are identical to the results I'm getting from metager.org which is just a meta search engine. Both those results came from Bing in the case of metager
I really like what I've read about Kagi, and I'll likely end-up paying for it too, but I _hate_ the metered fee.
I've never run a search engine, but I don't imagine that offering "Unlimited" searches where "unlimited" means "do not abuse" would meaningfully hurt their margins. As a benefit, you avoid losing customers that then need to ask "Do I make more than 5*80 searches a month? How many is that a day? What do I if I go over?". It also just feels like I'm being nickle-and-dimed.
I love that they're charging for the service, it makes me feel like they're building something sustainable. I just don't love the fee structure.
They're not going to do a metered plan. From an email sent today:
> Kagi will come as a free version with limited use; and an unlimited use, paid option at $10 a month, both versions having great search results with less spam and completely ad-free, tracking free, and with none of your search data being retained
$10/m for a household, if this is as good as people are saying, would be a no-brainer subscription for me.
Not paying $10/m per person in my household, though. That'd be way too high.
[EDIT] Though I guess with no retained search history, we could just share one account anyway, so it doesn't matter. I was thinking about keeping those separate. Unless there are per-user settings that we might not all want to share.
Yeah I've running Kagi recently and in most areas it outshines Google.
Love how I can easily banish garbage seo-only sites from results
I appreciate how it groups results into a more compact section if they're from forums or review sites. The UX give you the ability to scan for relevant links quicker
I've been using Brave Search for months now, much better than DDG. I had a Kagi invite sitting in my inbox, just signed up and set as default search.
Will give it a spin, but so far I like what I see.
EDIT: I immediately boosted all `github.com` and `reddit.com` results. That's a killer feature!
EDIT 2: better than Brave Search on mobile: there's a search button to enter a new search, and the textbox doesn't move around like Brave or Google do causing you to click on the first suggestion. I bloody hate that. Well done Kagi.
437 comments
[ 2.9 ms ] story [ 466 ms ] threadEdit: yeah that's good advice too. Though the *.edu trick also seems to be pretty useful so far.
"what is an iris" when wondering about anatomy brings up Wikipedia as the first link. Perfect.
With the .edu trick the first link is about the flower, and there's nothing about the human eye anywhere on the first page of results.
First listing (eye): https://webarchive.library.unt.edu/web/20120921215826/http:/...
Second listing (flower): http://www.ladybug.uconn.edu/FactSheets/iris.php
https://www.umkelloggeye.org/conditions-treatments/anatomy-e...
e: ah it looks like it is a .edu that redirects to a .org
https://www.kellogg.umich.edu/patientcare/conditions/anatomy...
First result https://webarchive.library.unt.edu/web/20120921215133/http:/...
Short but description of the iris and the function in the eye.
For textual links 2 out of 9 responses for eye irises. and 7 out of 9 for the flower.
embedded in the same page is 'Images for what is an iris site:.edu' and 10 out of 10 of those links are for eye irises.
Switching over to image search the first 20 are all about eye irises.
Bringing awareness
Google spends billions of dollars a year to give everyone their own personalized set of bad search results.
That's the justification for its invasive tracking.
Google: "We to build a dossier about you, your life, your friends, and your family members in order to give you better results!"
Us: "So... they could be worse?"
I had to look up some govt pdfs and tables recently and the government sites hosting the files hardly ever show up on the first page on Google anymore. It's a sad story.
If it had a filter to only show pages that didn't have cookie popups then maybe I'd look at it again.
[1] https://kagi.com/faq
Maybe not dominate the market, take over the world, but not every thing has to be that.
Imagine a search engine that truly is private, containerized search and does not need to sell you anything. It only wants to serve you true relevance and accuracy. That engine also skips over all monetized sites that serve more than 5-10% ads. Prefers cookieless sites. That search engine would be bliss.
The Bloomberg Terminal is extremely expensive and yet companies pay for it because it makes their traders' jobs so much more efficient. I don't see why companies wouldn't do the same for a search engine that filters out all the bullshit so their employees don't have to waste their time wading through it.
Google has 4.3B users, so 1% of that is 43M. With a $5 subscription, Kagi would have $215M in monthly revenue, enough to pay for 14,000 developer salaries.
A more reasonable target might be 500k users at, let's say, $10/mo.
(Kagi founder here)
I'm a Kagi private beta tester and love it. Will become a paying customer.
One question: I do remember a page that told me how much I searched and what that would mean once Kagi becomes a paid service. I can't find that info anymore. In light of the mail us Kagi users got today, does that mean that it's going to be a flat rate for starters?
We do not yet know if the flat fee will be viable for the company and we hope to learn that very soon.
If you want to save your search history perhaps you can open a suggestion on https://kagifeedback.org
A listing of top global educational institutions would only have a few thousand entries.
It's not without some irony that I note that the early, and much belated, Internet was largely edu domains, along with a handful of tech firms, government agencies, and military entities.
https://www.goodyear.com/en-US/learn/tire-care-maintenance/h...
On Kagi you have to go until the 10th result to get something from Bridgestone, which explains the same thing while being less aggressive about selling tires:
https://www.bridgestonetire.com/learn/maintenance/how-to-cha...
So the trade-off seems to happen between seeing fewer ads and having better tolerance for imprecise queries.
Or let Musk buy them.
Every site would have to run their own data analysis and sell their own ads. Advertisers would be unlikely to take Google or anyone else's word, so audits would be necessary but difficult to conduct. It wouldn't completely end online advertising but would turn it back into something closer to the old traditional model where those selling ad space had a handful of semi-generic personality models for their readers/users and advertisers would select one that best matches the profile of their target customer. It destroys the economics of centralized data collection.
There are many other laws that could accomplish a similar end but through different means. The biggest barrier would be getting politicians to not care about the demands of huge tech companies that dump endless amounts of money into lobbying and campaign funding.
The only way you have not listed is gov't funding. And i dont think i would trust a gov't run search engine either. May be a well meaning librarian institution could make it work.
> I'm quite sure a subscription model would fail.
this hasn't really been tried yet - it could work, who knows?
"...offers a freemium model (free basic and premium for $5/Month which includes paid versions of VPN....."
Or as it reads?
https://neeva.com/blog/introducing-neeva-free-basic-and-neev...
> How we make money
> We plan to fund Andi through a freemium business model, with free anonymous search for everyone for ever, and paid plans for professionals and businesses, including APIs and using Andi for corporate information, with supplemental revenue from anonymous referral link attribution. We're focused on building the product right now, and will figure out the details once we're further along.
> We are committed to staying 100% advertising free. Commercial considerations or partnerships will have zero impact on our search results or recommendations. Our recommendations are always made based on the best results we can find for our customers, and are never subject to commercial influence.
> By sharing search revenue with content producers and media companies that join our network, our mission is to provide funding to great content and reduce media's reliance on invasive ad-tech. Contact us at info@andisearch.com to get involved!
There are only 2 employees[1] of the company so there is a low amount of revenue they need to be self-sufficient
[0]: https://andisearch.com/about/
[1]: https://www.ycombinator.com/companies/andi
Do I have to build the startup myself?
What I really want is a decentralized, non-commercial, open source web. As part of this I would like to see search and communications moved to the client, rather than the cloud. This seems like it would take a huge effort to build, however, and even more monumental effort to bring people into the network. So it feels like a pipe dream at this point. I do think there are a lot of other people out there craving for something similar, based on nostalgia for the old days (90's and earlier).
Indeed. The internet in general and the beginnings of the web were a very different place back then. I remember about 1996 or so seeing a URL on a Pepsi can and thinking, "oh crap, it's over."
There's a lot more to search than just search in the traditional sense, and I think multiple cooperating services offering small complimentary functionality slices could probably offer reasonably similar capabilities (save for like image search, maps and translation) with a significantly smaller hardware footprint.
My own search engine is fairly janky, but a lot of its problems are understood and could probably if not be fixed, at least mitigated. That's one functionality slice, searching documents. You could have a slice for forums, one for reviews, one for high value sites like stackoverflow and wikipedia, one for facts, one for popular links in social media and news, one for source code, one for mailing lists, and so on.
As a whole, I think you might actually be able to put together an actually useful information gateway this way. Would be a lot of work, but I think the amount of work probably exceeds its difficulty.
Google has an impossible task in trying to answer a query like "Portable air mattress reviews", because 99.99999% of the results are SEO-optimized crap
On the other hand, they have an incentive - increased competition, they can still monetize better pages, search is what makes them money.
1 and 2 are trivial to detect. 3 is harder but still not impossible given Google's machine learning capabilities and existing datasets.
Downrank or ban ads & affiliate links and you remove any incentive for creating spam sites in the first place.
This would also put pressure on content creators to tone down the ads or explore alternative monetization models.
I also spend 50£/month on Patreon to support the creators I watch regularly.
My loosely held theory on the wide variance in people's judgement of DDG is that some people really like/have come to rely on how much Google personalizes results based on all the other data they have on you, even if they are in the "Google has gotten bad" camp. DDG doesn't do any of that: it's much more like going back 20 years where you have to put all the context you care about into a query, you're not searching "in a bubble". For me, that's a very positive feature, but others may find it frustrating and not even be able to articulate why they're not happy with the results. DDG does do a lot of the nice "enhancement" stuff Google pioneered long ago, like embedding the intro of the Wikipedia page in the sidebar for a query with a strong match, e.g. a person's name. So for me DDG is just the right level of "smart": it understands common queries well enough to immediately highlight useful details, but it doesn't try to be so smart that you as a user stop being able to understand what it's doing under the hood.
Also, I think Google could make a ton of money if they sold one ad per language at a time. As in, for the next 5 minutes the ad for all English speaking users will be for coca-cola, and then it will be for boscovs. Just like TV ads.
That is to say, a search engine could sell advertising without betraying it's users or compromising its own quality. It is possible, but would need constant vigilance against the temptations of greed.
I cannot imagine it being easy, but I find it hard to believe that Google aren't able to weed out the worst offenders. For a large number of queries the results are all SEO spam. Try searching for anything related to health and get a useful result. It's no longer possible.
At this point, it's no different than an appstore in terms of volume. Ironically that's where we're heading with most websites being "3.0" and just SPAs and now PWAs even.
Medicine researching on the internets always was among the hardest topics. And anyone who really understands Medicine has to write on any advice not to consider it as a real medical advice... I agree that google has degraded down to unusable in comparison to non-global searchable websites (like SO) but Medicine is very bad example to your point.
In fact, I wonder how far a search engine could go with a curated list of "good actors" you exclusively serve up results from. Add a "search everywhere" link for people that prefer SEO hell.
When Google started, the web was honest and innocent. People linked to the pages they liked, and Google could crowd source those links and raise up the most popular pages.
Now most of the web is lies meant to confuse search engines. Figuring out what pages are good is no longer a feasible task for software.
Of course it is. It shouldn't be at all difficult to train a large natural language AI model on SEO spam pages. But nobody will do it unless they can figure out a business model that makes it pay.
Or a non-profit organization is formed to do search spidering for the benefit of the whole world, much like Wikipedia or PBS.
But in hindsight, this was a ploy from them to try and upsell AMP, which was in a lot of cases heavier than non-AMP websites, and it was a means to keep people inside of their ecosystem - and inside of their advertising network, because not all websites Google linked to had Google adverts on it.
They've given up on AMP. They've given up on improving search results quality. Dare I say they've given up on improving the web? I mean they're still putting some money into Chrome and HTTP/3 and the like.
Looking at the misnamed "Privacy Sandbox", Chrome is not just there to improve the Web (and therefore google usage), but also to control it (and steer advertisers to Google even if it undermines the Web as a whole).
They’re killing Firefox with it (RIP) and successfully killed IE with it (rest in piss).
We’re getting to a desktop browser hegemony again but I doubt a scrappy upstart will be able to save us.
I'm hopeful there are a lot of Mac users like me out there.
For example, I wanted to watch a bunch of videos of Mark Zuckerberg speaking from before 2010 and the YouTube search engine only showed me a limited amount of things I had already watched— it even had unrelated suggestions below those results based on my viewing habits.
So I went to Google and searched for videos and interviews of Zuck and found a bunch which were not easily surfaced on YouTube itself.
Same thing with Ina Garten/Barefoot Contessa cooking videos.
Now that I've found some fresh videos, the homepage recommendation algorithm has gotten better at surfacing new content related to these topics.
For example, all the stupid aliasing of search terms, and dropping of search terms.
This is helpful if you want to optimize for "sorta close, sell things!" such as "maytag A4453 fix timer", but you want advertisers to latch on to maytag=washing machine, and ignore the rest.
And it also helps when websites with google ads on them them get higher SEO, because suggesting that site might get a click, and profit!
EG, suggest a blog/spam site, because google ads are on it.
And it also helps with google voice search! Did the speaker mean here/hear? Did they mean Danny/Daniel/Danson when they said Dan?
Google has too much genericization in its search, and it is for profit naturally, but I think it is lazy profit, and it is becoming worse yearly.
And it is all their fault, no one else's.
At some point I really hope that we can all recognize that “seemingly harmless”, “engineering-oriented”, and “data-driven” are not virtues.
Perhaps this is the point you are making: if one is data-driven and engineering-oriented without regard to the consequences, that could be seen as a vice.
Despite the amazing engineering/science, it is hard to come to peace with von Braun or Teller (just to pick two examples).
Hopefully we can get to the point of putting purpose (I think “mission” is a bit too hackneyed and neat to work here) first in discussions of culture, whether corporate or engineering culture.
Google has at least 10x as many employees as it needs. That's based on decades of revenue being decoupled from costs, and from monopoly profits. It's the double-whammy of being in tech (where costs and revenues don't couple since incremental costs are close to zero), and having a super-profitable monopoly.
You can't fire 90% of the workforce; even morale and culture problems aside, at this point, the organization is structured to need too many people. On the other hand, organizing over 100k employees turns into this superorganism problem.
At >100k employees, the influence of anyone -- up to and including the CEO -- is very, very limited. Dynamics take over.
I've been bearish on Google for a long time, but I haven't shorted the stock. Market irrationality can last a lot longer than my wallet. Fundamentally, though, one could muscle through building a new, better Google for much less than the valuation of Google.
Google could also be worth a lot more with a breakup. There is negative synergy between, for example, advertising, Google Workspace, Android, hosting, etc. If those were independent businesses, with partnerships around things like data sharing, the total value could be much higher. Alphabet was a move in the right direction, but the move should have been much deeper. Right now, Alphabet is Google + a bunch of tiny startups.
No one is incentivised to grow head count. They are incentivised to do more, produce more revenue, and expand the reach of the company; most of these goals require the application of more smart people to a problem. They are not getting a promotion or salary bump because their group increased from 200 to 300 people over the previous quarter, they are receiving rewards for being able to effectively direct those people in order to achieve some specific goal. Increasing head count by 50% while delivering 250% growth in revenue/engagement/view/etc is what is being rewarded.
Actually delivering value is a huge advantage too, but as far as I've seen is not prerequisite for getting ahead.
Appearing to deliver value is strongly advantageous. Delivering value helps you do that. Other methods:
- Good salesmanship (e.g. making easy things look difficult)
- Stealing credit (can be personal, or e.g. for market conditions)
- Lying (e.g. cooking books)
Delivering value is probably the easiest way to appear to deliver value at the IC level. The higher one moves in the corporate ladder, the more important the others become.
- If I have a job at Google as an IC, I can look for IC jobs.
- If I have a job managing 10 people, I can get a manager-level job.
- If I have a job managing 100 people, I can get a director-level job.
- If I have a job managing 1000 people, I can look for a VP-level job.
- If I have a job managing 10,000 people, I can look for a C-suite job.
Average tenure in a position in the tech industry is around 3 years. No one will know or care what objectives a Google manager achieved, or whether those were hard or easy to achieve. If you want to grow rapidly in your career, the things which matter are:
- Connections, network, and references
- What shows up on an interview (e.g. self-improvement)
- What shows up on your CV (e.g. how many people you managed, or the brands you worked on)
The place techies get stuck in career growth is by trying to do the "right thing." By the time you're trying to rise to the top of a corporate ladder, the competition is extreme, and the people who succeed play the game as optimally as they can.
I might be able to look one level up. An IC might be able to manage a small team. However, your odds of having an IC move directly into the C-suite require nothing short of a miracle (a Nobel prize, being the author of Linux, or something similar).
In any public company the CEO is just another employee. Maybe he even has less freedom than others.
Yes, responsibility and compensation are vastly greater, but in the end you don't really realize personal goals.
Neeva also lets me index my personal documents and includes those in search results as well. They are an early startup but I am excited about what they can build when they are not beholden to advertisers.
And who can blame us? Other sites still suck worse. I can't believe how bad walmart.com still is - the user experience is night and day. Walmart - the retailer embracing computerized tracking of spending by credit card number and other advanced data analytics a full decade before Amazon was even a glimmer in Bezo's eye took forever to realize the power of using their stores as local fulfillment centers? Provides a 3rd party marketplace that's less reliable than craigslist? Who the hell is in charge of online operations at these big companies and how do they still have jobs?
Then again just look at Sears - the company that literally invented mail order *in the 1800's* and how they completely missed the Internet - the ultimate mail order environment.
Ugh.
The problem is that Google is engaged in a war with SEO optimizers, and losing (or at least, barely breaking even). No business model will fix that. It's a hard problem that any successful search engine will struggle with.
A typical employee at any large company is incentivized to:
- Maximize their salary
- Maximize their future earnings potential (e.g. CV)
- Have fun
- Have work-life balance
... and so on. That goes all the way up to the CEO. Pretending a corporation thinks and responds to incentives is folly. Running a corporation like Google is like herding cats. If everyone marched in the same direction, Google would cut through SEO optimizers like a chainsaw through butter. There is no wayh to do that with 100+k people.
Google fundamentally has no financial incentive to fix any problem, since organizations aren't intelligent, sentient beings which act on incentives. They have dynamics from the interactions of 100k+ individual agents.
There is no magical fairy which makes large organizations act in their own best interests or even try to survive. The only reason they do survive is that's as true of their competitors as themselves.
Corporations are intelligent, sentient entities which act on incentives. They don't have a conscious sense of self, but then neither do many natural organisms. That doesn't keep them from responding to their environment as if they're aware of it, and acting in incentive-driven ways.
Corporations seeking to maximise profits are exactly like humans attempting to maximise income. The problem is that corporations - like humans - have the wrong incentives, further diminished by limited predictive ability and poor heuristics.
So maladaptive behaviours emerge and stick. Both humans and corporations find a nice local minimum and don't move from it.
More intelligent behaviour would be able to predict and avoid existential threats without having to experience them first. But it's a question of predictive ability, not a binary absence of all intelligence.
So it's not that Google can't do better because it has too many employees. It can't do better because its cultural heuristics - which include it internal culture and the external culture it operates in - don't allow it to.
I can set up "incentives" for an anthill, in the form of a mixture of sugar and borax. Emergent behavior takes over.
The behavior of corporations, at the scale of Google, Facebook, or similar, is almost entirely emergent dynamics. Same thing for countries, for that matter.
For many search terms, the entire above-the-fold and sometimes multiple scroll heights are filled with:
- Ads
- Info cards
- People Also Ask
- Map/Business info
- Videos
- Related Products
- More Ads
The organic search results start after that and on mobile they may as well not exist.
If everyone is clicking the directly or indirectly sponsored/algorithmic content then what incentive does Google have to give you good, straightforward links to relevant webpages?
So why would Google serve the more direct and helpful result from the edu domain!
I think they are genuinely struggling in an adversarial environment. Imagine every hacker in the world specifically focused on your system and the payoff is millions or even billions of dollars.
But clearly there are also a lot of motivations to de-emphasize the organic results on the search page in favor of ads, “algorithmic answers” that give you an answer without clicking away from Google, and redirects to Maps, Youtube, and Shopping which are revenue-generators for Google.
Maybe that also comes with a budget de-emphasis for fighting the war in the organic results vs surfacing relevant content from these other sources.
Many product folks just don’t seem honest, even with themselves, about what benefits users. Do you really think your users benefit from ”more relevant” advertisements? Because your users don’t. Are new services and content so beneficial to users that they’d forfeit their privacy to fund it? If so, why obfuscate your having made that choice for them? Why not make it opt-in? Does that dialog box popping up just at the right time really give that overstimulated and frustrated user a choice? Either answer those kinds of questions honestly or admit that you’re just finding the smoothest path to maximize revenue. The hypocrisy is infuriating.
Give me a break.
>Professors are paid generous salaries to share knowledge with the paying customers of the University (students). So, let’s find some paid unpaid University knowledge on how to change a tire by using our new trick!
But the bottom of the page says
>Prepared by: [students] For Dr. Bruce Magee's English 303 (Technical Writing) Class, Winter 1997-1998.
The professor is actually an English professor and the page was written by students as an assignment. I looked into this because I was wondering, "what would a professor be teaching that he's writing notes on tire changing?". There's probably some professor out there that does that in an academic context, but there's probably a lot of things that no professor actually gets paid to write about. Maybe they would write about something miscellaneous like that anyway and put it online, but they would be doing so out of the culture of web savvy professors having personal pages where they upload stuff they think is helpful.
It's certainly better than nothing but there could be specific gotchas for your situation. Most common would be needing certain tools or techniques for your model of car. It'd probably be best to either look up the manual for your car online or find a youtube video for changing the tires for your model of car.
It's good, but not really _that_ often useful.
(you.com founder here)
site:news.ycombinator.com reddit.com lwn.net servethehome.com head-fi.org doom9.org ....
https://hn.algolia.com/
Car troubles? YouTube chrisfix car_problem
Knowlage stuff? Wikipedia query
Programming question? Stackoverflow question
... and so on
So, you want someone to help you and you are pissed when they don't want to do it for free?
In fact, I'd wager that if we had a way to filter for content that is written without the goal of being monetized, we'd have a better web and us consumers would be more inclined to reward and give back to the creators generously.
https://duckduckgo.com/bang
1. https://en.wikipedia.org/wiki/Protestantism#Protestant_cultu...
2. https://en.wikipedia.org/wiki/Protestant_work_ethic
If the "business people" were to disappear we'd do just fine. The whine of incessant marketing does tend to disturb the peace.
If someone is so clueless about changing a tyre that the best place they can start is google, then organically taking them to the garage's page is a win. Maybe they'll find the information useful, maybe they'll decide they're out of their depth and call the number on the screen.
For them to show up as the first organic result is a huge win. All they have to do with that win is not screw it up. And 9 meg of trackers, a page full of popups, and pointless CTAs (do you want to hire someone who can't change a tyre?) is screwing up that easy win.
You don't have to be scummy to turn search results into profit. That garage came so, so close of being a great example, before they threw it away and made a bad example.
(Sadim is the revers of Midas.)
Reddit is what the internet used to be back when it was useful.
So you expect them to write an article on how to fix your tyres for free? And what about the hosting costs?
(1) - https://news.ycombinator.com/item?id=15985775
Another method could be to analyze how many external requests a page makes and put that into the calculation.
Of course, there are exceptions, but I think this might help.
Of course Google has PageSpeed, which supposedly influences rankings by downranking sites that are very heavy to load, so I mean, that's an aspect to take into account, but I still believe the above to be largely true.
Google is hopeless anyway, so here's something for the next search engine that wants to bring back the Web: downrank by the number of 3rd party domain requests, attempts of tracking, and possibly also popups.
Or do you mean sites like reddit, facebook, etc. by "apps"? In that case, downrank them seems fine - they can always provide lighter version.
Google will never do it because those ads & analytics can be Google's and benefit their bottom line.
[0] https://developers.google.com/search/blog/2020/05/evaluating...
[1] https://developers.google.com/search/blog/2020/11/timing-for...
[2] https://developers.google.com/search/blog/2021/04/more-detai...
Those sites generate plenty of value ... for Google. Even if they're not showing Google adverts, those sites normalize the idea that "websites have adverts", so users don't question it on other sites.
If Google started returning sites with low numbers of adverts first then users would start demanding more sites with fewer ads. Google really don't want to show fewer adverts.
[1] e.g. https://iorate.github.io/ublacklist/subscriptions
This is what I got in return:
Row 1: 2 buttons marked "on a bike" and "scooter"
Row 2: single scrollable row of images of tire fixing kits with the prices.
Row 3: step by step recipe on how to fix flat on a car (7 steps long) followed by the link to the corresponding article and small image on the right side.
To me it looks straight to the point and one can't bitch about single row of ads of a relevant products.
The first link clearly explains how to fix a tire and right underneath the first link, there's a link to Youtube video with described sections which you can click on immediately
The sites are full of tracking and ads, but that's hardly Google's fault imo
what's next?
Kagi is in a whole other league. It's on par with, if not better than google search. I highly recommend it. When they introduce the paid tier I'm sure as hell going to subscribe to it. At this point I would trade my Netflix subscription for it.
Even then we won't have search without an account, among other things because bots would probably kill us.
(kagi founder here)
I haven't found Kagi's results to be significantly better than Google's results, but unlike DDG they're not hugely worse than Google. It's all the extra stuff that makes me want to use Kagi instead of Google:
* Ability to banish sites from results forever. E.g. I never want to see the Concordia Lutheran High School's website when I search for things in the Common Lisp HyperSpec with clhs.
* Ability to weight sites higher. E.g. there are two mirrors of the Common Lisp HyperSpec that appear in results, one at http://clhs.lisp.se and one at http://www.lispworks.com/documentation/HyperSpec/. The lisp.se mirror is hilariously slow for me (5-10 seconds to load) while the lispworks one is instant. Google always ranks lisp.se first which tricks me into clicking it and wasting my time. Kagi lets me weight that site lower so the lispworks site comes first. Banishing lisp.se would also work, but just upweighting it means I can still find it if the other mirror ever goes down or something.
* I reported a minor bug to Kagi and they fixed it within a few days. I doubt that would happen with Google.
* Supports DDG-style bang queries, which I use occasionally.
* No goddamn ads/tracking (I use uBlock Origin, but it's the principle of the thing). I've almost entirely de-Googled my life at this point (Youtube is the only holdout for me).
So it's not that it gives better results than Google, it's that the results aren't any worse and the extra quality-of-life features are really nice.
Basically, when I used DDG, I found myself frequently using !g to search Google instead. For Kagi, I haven't really found myself doing that anywhere near as often.
So yeah, you have to fill out a survey (took a couple of minutes and I got an invite the same/next day), so it's not the easiest to recommend right now. Two weeks and they go free+premium according to an email today.
I think it's just that I, like a lot of HN, have been getting super disillusioned with Google lately - and most importantly the preponderance of obviously machine-generated zero-value spam pages for everything I search for - so I was desperately trying to find something better. DDG constantly disappointed for my searching. Kagi doesn't do that. So I'm happy. $10/month happy? I'm not sure.
Generally, there's a growing group of people who are feeling about Google the way a lot of us felt about Facebook 10+ years ago - it's time to stop relying on it and maybe cut it out entirely (and yes, I know there are plenty of people who felt this way about Google for years already). So I pay for Fastmail now. I guess I'll likely be paying for Kagi now. Quality is getting expensive, but at least I'm paying with my money rather than my data.
And for this I will HAPPILY pay $10 a month. It's the most freaking annoying thing about Google. Obvious link farm sites with zero way to remove them.
Here are the queries from the original article in Kagi:
Top result:https://www.wikihow.com/Fix-a-Flat-Tire
Also, it predicts a video is best to answer this question and gives 3 YouTube thumbnails listed at the top of the results. They all seem generally helpful based on the title.
Top result:https://www.thepearlsource.com/blog/facts-about-pearls/how-a...
First result is
https://www.outsideonline.com/outdoor-gear/cars-trucks/how-t...
2nd, 3rd, and 4th resultsa are videos
5th is wikihow (which by the way IMO is often a really bad site. There is lots of junk content on wikihow)
https://www.outsideonline.com/outdoor-gear/cars-trucks/how-t...
Which is equally good.
In case of code.you.com vs. Google: 1. Write a recursive max function for lists in Scala
Not much different from Google, but surely a search dedicated for code only feels better.
I've never run a search engine, but I don't imagine that offering "Unlimited" searches where "unlimited" means "do not abuse" would meaningfully hurt their margins. As a benefit, you avoid losing customers that then need to ask "Do I make more than 5*80 searches a month? How many is that a day? What do I if I go over?". It also just feels like I'm being nickle-and-dimed.
I love that they're charging for the service, it makes me feel like they're building something sustainable. I just don't love the fee structure.
> Kagi will come as a free version with limited use; and an unlimited use, paid option at $10 a month, both versions having great search results with less spam and completely ad-free, tracking free, and with none of your search data being retained
https://kagi.com/faq#cost
Not paying $10/m per person in my household, though. That'd be way too high.
[EDIT] Though I guess with no retained search history, we could just share one account anyway, so it doesn't matter. I was thinking about keeping those separate. Unless there are per-user settings that we might not all want to share.
(kagi founder here)
Love how I can easily banish garbage seo-only sites from results
I appreciate how it groups results into a more compact section if they're from forums or review sites. The UX give you the ability to scan for relevant links quicker
e.g. Searching for a generic "best wireless headphones" looks like this https://i.ibb.co/GPyb6Zp/kagi-exmple.png
Will give it a spin, but so far I like what I see.
EDIT: I immediately boosted all `github.com` and `reddit.com` results. That's a killer feature!
EDIT 2: better than Brave Search on mobile: there's a search button to enter a new search, and the textbox doesn't move around like Brave or Google do causing you to click on the first suggestion. I bloody hate that. Well done Kagi.