The truth is that normal people like abusive software. Using software that gives them freedom also gives them the responsibility to understand the behavior of the machine which is something they do not want. No amount of evangelism or possibly even education will fix this. IMO for their own good people who behave this way should not be allowed to use computers and should delegate the task to people who are willing to think through the consequences of using a particular piece of software.
Allowing normal people to use computers is cruel in the same way making a dog order its food over the telephone is.
i use opera on android, firefox in windows, safari on mac and duckduckgo for everything and have been for YEARS. I'm offended you're including me in your cohort. lol
Im so used to the UI (down to the fonts, things I can't even name), that DuckDuckGo looks off to me. So although it is set as my standar search engine, I g! most of the time.
I would disagree. It is nice at the start but it shows the same things on the same search result forever.
That is not something that you'd want in a Porn search engine, you'd want it to have some amount of randomization.
All I know is that I've fed bing some of the most obscure keywords to find what I'm looking for and it's been like "oh I know exactly the video you're looking for I've got you".
Curious thought, isn't in this area that the most important thing to avoid showing to the user certain undesirable result by accident? Not implying that bing ever failed to do that.
Google results are the one thing keeping me on Google. Not sure why getting the almost indisputably best results would be bad. Even if they are getting worse, they're still miles ahead of anyone else.
I've been waiting for a Kagi invite for a while because I've heard good things on HN about it. Finally got one a couple weeks ago. Finally switched off of it and back to Google yesterday. The results weren't even in the same ballpark, despite me really wanting it to succeed.
My standard line on this is that Google today is not as good as Google of (at this point more than) 10 years ago, but it is still the best available option today because nothing is as good as or better than Google of (at this point more than) 10 years ago.
That said there have been a stack of new search engine posts on HN in the last few months and I may have to update my priors once I’ve had a chance to actually investigate the new options.
EDIT: Maybe I should note that I’ve also been relying a lot more on Reddit too in the past year since Apollo has a decent search interface for Reddit and I’ve gotten used to processing new subs quickly and getting information out of them. If nothing else I usually at least have a stack of new terminology to feed my search queries elsewhere.
I think it's also only going to get worse when the amount of bad content and blog spam is 1000 times the ratio it is today due to really human-like AI writing. I am sure Google will find ways to detect this, but it will be a cat and mouse game for decades because at some point we won't be able to tell apart bad, lazy human writing from AI writing.
At some point the internet has to go to a circle of trust model with real identities tied to online content of any sort. I see no other way to curb this pending disaster than being able to block bad actors and bad actors having very limited means to publish under an alias.
I tell everyone to use Bing. It's decent, but better than using Google. Google invades our privacy, keeps users hostages for money. I wish Amazon has built a search engine so that Google's tyrannical regime on internet ends.
Can't they just migrate off of it? I guess the only part being held hostage according to the article are their Play Store purchases, if even that (the article says the purchases' fate is unclear).
To me it seems Google has given ample time to either migrate off of it or to become a customer.
- Its location feature is better than DDG or anything else - It has gotten worse a bit, but if I tell it to show me results from Colombia, 95% of the results it will return are from actual websites from Colombia. DDG will throw anything from Latin America, for example. Not to mention the disaster with Bing.
- Its image search feature is still more precise than others. Reverse search won't return sometimes what I'm looking for and I have to resort to Yandex/TinEye/Bing, but still. Oh, and it can search for SVGs, which others can't.
- Double quotes aren't returning exact matches, but other operators are working fine as far as I can tell. Filetype operator is great and way bigger than DDG's, cache operator is great for looking for a cached version of a website that is not working at the moment, the minus operator still works (sans the advertisements).
The moat is worse than we think. New search engines are not only hobbled by the bandwidth and processing power and storage required to spider the web, but by the websites who will preemptively disallow them because they're not Google.
I can't imagine trying to build a new search engine when the landscape is intentionally (if justifiably) hostile to new search engines.
Elaborate? It is the best thing web has for crawling rules. Yea ofc crawling rules can be broken and violated but all serious internet search engines respect robots.txt.
Honestly I think people are completely underestimating the difficulty of a good search engine. Google was better ten years ago because search was an easier problem to solve back then. End of story. Nobody is coming along with a better search engine. There is too much spam, content gaming, and money to be made by hacking search.
These posts should almost be blocked from hacker news. ITs a fantasy. Its like saying that democracy has failed so lets replace it, replace with what? Its the best we can get given the alternatives, and its flaws will always be exploited.
Democracy would be best replaced with cashless society. Society based on everyone value of doing what they love doing most (aka hobbyist), and virtually everything else replaced by robots and technology we already have. This is what's coming eventually, but its not something that can be installed on the top of your "operating system"; you will have to format the whole hard-drive (civil war)
Well, ideally people could choose to do whatever they liked to provide for society. Everyone would have a role to play. So those that like to farm (raised on a farm, to be proud of their work) would farm, those that want to commit their life to science could attempt to advance their field. Someone might find an invigorating pride in making sure wastewater treatment is running well, but they don't know how to get into the field and are stuck working a job they don't like for money, currently. Artists could create culture. As long as you are doing something to provide for society, society will provide for you.
There will be people that do not have favorite hobbies. There will be people with no ambition who benefit from being told what they can or should do to contribute. Those people, in order to have their needs met could choose to take on a Necessary Task, even if it isn't their favorite thing. Maybe they learn to love it and excel at it.
Don't want to do anything to contribute to society despite being physically and mentally able? No needs met. Meet them yourself, you magnificent lone wolf. Want your needs met but don't want to do Necessary Task that you hate? Find a purpose and pursue it.
I don't know how to break it to you, but the system you're looking for that does all the things you're saying is capitalism.
Have the talent and desire to be a heart surgeon? Go for it, and you'll be rewarded. Want to make life sized statues of the Beatles out of used printer cartridges? Eh, fine but maybe also get a day job.
Ah yea. Let me just pay to put myself through med school independently while I struggle to pay for food and housing.
Sorry but Capitalism is a broken system that is not meeting mine or million's of others needs, much less enabling us to pursue what we're actually passionate about. I at least know that I am passionate about building and supporting technology but I still struggle to pursue it professionally because I had the poor luck to be born without the resources to get a piece of paper that says I know how.
No. Google is an actively hostile experience. Try it without adblock and get a taste of how most people get treated by Google. And for the search results, Kagi is already better for 90% of my queries.
It's much more difficult now to build a competitive search engine, but saying it's impossible and discussions should be banned is toxic. (And already basically proven wrong with existing competing search engines.)
I think that with current technology, beating google at the scale they run at is impossible. I think with advances it NLP, its possible. But right now, its pie in the sky.
Kagi's better than Google for me. Proof is that initially I only had it set up as default search on one device but I've progressively switched them all over out of annoyance with crap google results. I haven't yet had to go to google once to finish a search I started on Kagi.
Came here to mention kagi.
Very happy user here. The no-spam results make it so much more useful as a tool.
Also, "block this site from myresults forever" gives so much more agency to the user than anything Google has release in years. Googles "we know better" just reads as a big middle finger to me.
Wow, they have that? That’s what I always wanted from Google and they never delivered. Im sure they had reasons. I really don’t think they want to empower their users, they are even hostile to search users and content creators, amp comes to mind and am glad it dissapeared already. Meanwhile I’be been using duckduck but i’ll give kagi a go.
No, Kagi (unlike Google and DDG) will just stop providing results, it won't make up others with very tenuous links to your search terms like the others do.
Just tried that. I did have to put "ukrainian generals" into quotes (which makes sense, that's how it's supposed to work) but then it just showed me 5 results total, all of which contained exactly that phrase.
Kagi requires users to sign up (even though it's currently free?) which is 1000x more user-hostile than anything Google does and makes it a nonstarter as far as I'm concerned.
Kagi is intended to be a paid service when it launches. This is something I actively want. It should make them the opposite of user-hostile. Their users will be the source of their revenue so they will need to provide value or lose them. Login is a necessary part of that. I'm happy to take both that inconvenience and the cost.
I used it for awhile and did like it quite a bit - I found myself having a bit of anxiety though worrying if I was going to hit the search limit with my more trivial queries. Google really has us psychologically
Is there a search limit for Kagi during beta? I don't see it documented anywhere, and I've been using it as my primary search engine for a few months and haven't seen any sign of a limit.
I wouldn't be surprised if there's some limit to prevent abuse, but it hardly gets in the way of my normal usage.
According to a discussion I had here previously with a Kagi user, when they go out of beta they plan to offer a $10/month plan with a quota of 20 searches per day (and $0.015 per each additional search), and an unlimited plan for $20-30/month. Additionally, every time you change something related to your search query (such as clicking "Images", changing the sort order, applying filters, or blocking a website) counts as an additional search, so you could go through your quota pretty fast.
Apparently all of this is to offset the costs from paying Google and Bing to run searches for them (the site basically takes results from Google and Bing and then combines them together with their own special sauce) but with a pricing model like this it seems like Kagi will remain a niche tool solely used by the wealthy.
Hm, 30 USD/month for good search is not that much - and certainly not "wealthy" territory. Remember that many people pay subscription services they use a lot less (18 USD for Netflix, anyone?).
The main paradigm shift necessary will be to pay at all for something that was "free" and is provided "free" to this day - even if the quality is worse.
Monthly subscription and quotas? While I respect and understand the goal I can’t help but feel like we are paywalling what the web has been for the longest time and should be “out of the box”.
And as of now, they are listening to users very attentively. Just a few hours ago I suggested they clarify the defualt "Programming" filter since it only searches q&a sites (i.e Stack Overflow) and they have already changed the name to "Programming Help" to make it more clear.
Google uses ads to pay for the service, which is available without logging in. Those ad systems can track you. On the other hand, a paid service (even if it is currently in "free" mode) will likely get rid of ads, but requires some authentication to know you are you.
I'm logged into Google all the time. I do this to get sharing working across Chrome. Others doe the same with Firefox. Some don't, because privacy.
I hear you think logins are 1000x more hostile than paying with your eyeballs. How would you suggest resolving this clearly conflicted view with logging in vs. how the revenue is made on a given site?
It used to be that if you used a paid service, then that service wouldn't throw you to the wolves of internet marketing/tracking companies. That alone made logins 1000 times less hostile.
But that is changing. Internet marketing companies are now teaming up with websites so the websites to their dirty work for them and you end up getting tracked anyway -- but with much higher certainty as to who you are. That means that now, yes, logins are more hostile.
And they also intend to charge $10 per month (and that's just for 20 searches per day, which is a WTF low number). There's no way I'm paying that kind of money.
I use Google without AdBlock whenever I am in a private browser in addition to going without adblock on both my phone and tablet. I agree, it isn't that much different. At least with search, that is.
Kagi is probably a lot more usable, right now. Nobody’s trying to game it.
As long as it remains niche, it might actually stay that way indefinitely. I’m just worried about Google taking inspiration from them. I bet whatever “tricks” they use now would get a lot less effective if they did.
Do I have to wait for their invitation to join the beta list before I can try Kagi? The comments here are promising and I tried to take a look for myself but apparently only beta users are allowed in at the moment.
What’s the story behind Kagi - are they somehow related to Google?
To my knowledge besides the old experiments with client side search, Google has never allowed (even paid) api access to their search. I thought they did not do this, because it would have allowed somebody to jump start a search engine that might eventually become a competitor.
Kagi seems to be open about their Google relationship, so I assume they have agreement in place.
This even feels like some experiment from Google to create a premium, paid, search product (like what they did with YouTube).
> Honestly I think people are completely underestimating the difficulty of a good search engine.
No, they are underestimating the difficulty of funding a good search engine.
I liked the runnaroo search very much as did several of my friends. The guy who ran it couldn't fund it. He shut it down.
Altavista (Yeah, that far back) had a nice feature where it would draw a cluster graph of your search results. So, if you searched for "python", it would show your results but would also draw a little graph and you could see that "Hey, there are two clusters here--programming and reptiles." You could then click on the "programming" node and the "reptiles" cluster would go away. It allowed you to drill through irrelevant stuff really quickly.
Note how that feature doesn't exist today--in spite of orders of magnitude more programmers being thrown at search, graph algorithms, and nifty Javascript web UIs. I wonder why ...
(/sarcasm in case you missed it. I don't wonder why. Such a feature would let you drill through irrelevant Ad and SEO garbage too quickly and would impact Google's revenue.)
Honestly I think people are completely underestimating
the difficulty of a good search engine
I suspect that is not difficult so much as expensive.
Boutique search engines pop up all the time here on HN, but they can't compete fairly against Google, without the resources to crawl a billion webpages day after day.
Source? But it's definitely possible nowadays when RAM is cheap and modern Google makes the Internet feel incredibly smaller than it was 20 years ago.
At the time you would get tons of results from a myriad of small blogs, forums, niche websites. Nowadays it's Pinterest, blogspam and more SEO optimised algorithmic crap. If you're lucky you get a forum result that might actually be relevant.
Given the joke that if you can't Google it it doesn't exist, it follows that the Internet has considerably shrunk in the past two decades.
It looks like I was mistaken. They don't keep all of the documents in RAM, but I'm pretty sure they keep the search index (100 petabytes) in RAM. I'm struggling to find a source but I remember hearing it in a talk given by a Googler on YouTube. I also read somewhere that moving their whole index to RAM is how they brought their response times down from a couple of seconds to a couple hundred milliseconds.
> Nobody is coming along with a better search engine
https://neeva.com/ better than pretty much any solution (Qwant, DDG, etc) I've personally tested. It also indexes specific websites like StackOverflow, GitHub, and GMail.
Edit:
Neeva does require an account to create because eventually the product is going to require a subscription.
I don't mind giving my email or making an account after I have any inkling that the product might be worth deleting a few spam emails. Neeva was giving me a modal popup in my first minute; that's never going to get my email.
(Founder of Neeva here) Appreciate the input and hear your frustration. This is a carryover from changes we made to introduce the free tier earlier in the quarter. We are making imminent improvements on modal frequency and getting them out of your way. Stay posted.
Yo! Neeva is a dope product, imo, but could I offer some suggestions on how I use Neeva?
1. I have more than one GitHub account (one is for work, one is for personal stuff). I would like to somehow keep them separate, but at least being able to add multiple GitHub accounts for indexing would be cool.
2. Could Neeva offer something for segmenting the parts of my online presence? For instance, there's personal me and then there's work me. I'd like to turn a different profile on when I'm doing different things.
3. I don't know if it's possible to partner with ProtonMail, but I'd love to index that and my Proton Calendar as well.
Also didn't know the free tier is permanent, but I think that's probably a good move. Thanks for what you do!
Hi! Thanks for your notes here. I'll take these suggestions back to our team. re: 2. Being able to connect two accounts is interesting -- right now I solve this by having two Neeva accounts, one associated with my work email and one with my personal email.
I experimented with neeva, because I'd really like an alternative to Google, even a paid one. However, I found its results pretty disappointing. At least for my work, I had to go back to Google, because I don't want to waste time "on the clock."
Neeva has a free tier (with only access to the search engine, so people can still test the search aspect), but when you pay for it now you ALSO get LastPass premium and Bitdefender VPN Premium as well, along with all their integrations into your system.
If you trust these people are doing what they say, it's a pretty good deal for securing your internet spaces and trying to get away from google, imo.
1) New system comes out that indexes/controls/regulates a naively created dataset
2) Data consumers adopt that system and experience benefits
3) Data suppliers learn rules of system and take advantage of it to improve the positioning of their data, thus breaking the intent of a system built on the assumption of naive creation
4) Users complain about the broken system
5) New entrants realize that the original system actually solved the core problem really well, and there are no easy ways to solve the 'gaming the system' problem
6) Flawed system remains the best available option indefinitely
It's like entropy, there's just no fighting it.
EDIT: And to extend this beyond Google - do you see a lot of long text blocks in 7 second tik tok videos? That's because the creator found a way to game the algorithm.
Let's say someone writes a blog article. They publish it on Wordpress with a bunch of unoptimised defaults. Some time after, someone comes along and says "Hey, the search engine will rank your post more highly if you include the name of the post in the URL instead of just having it be blog.example.com/posts?id=1234", so they change the URL. Is that SEO? Should that result in the site being de-listed/banned? It doesn't meet your definition of "provide website content that is more relevant/interesting for the viewer".
And where is the line between "provide website content that is more relevant/interesting for the viewer" and SEO? If I realise that (to take an example from something found on Google) I could take the sentence "Identify the best customers and convert more" and rewrite it as "Marketing automation helps you identify the best customers and convert more", is that SEO because I'm intentionally adding a keyword, or is that making the content more relevant to the user by being more explicit in what I'm saying?
To my moral standard, all of this is manipulative if the intention is to manipulate the Google ranking instead of seeing an improved Google ranking as a result of delivering better service to the visitor.
This does of course not imply that these things can be proven, which implies that no action can be taken. Not everything that is immoral is punishable.
I'm sure there are better examples, but this specific one actually does make the experience better for the reader. People often return to good blog posts and it's a lot easier to remember a title than a numerical id. If you can remember the title and know the url of the blog and it has a simple naming convention, you can bypass the search engine entirely and just go straight there. Sports sites are really good about this. <URL>/<LEAGUE>/stats always gets you the stats. <URL>/<LEAGUE>/standings gets you the standings. If you're curious how a team is doing or who currently leads in scoring or whatever, you know where to go without needing assistance from search.
Sites with people hired to optimize the robots.txt file are ranked higher than those without. If we ban all websites that hire people to optimize their robots.txt, then a lot of the web isn't indexed and the search results suffer.
Or if we ban every creator on tik tok who posts a lot of text in a short video, then a lot of creators are going to be de-platformed and moved elsewhere.
The point of what I wrote is that 'humans are really god at following rules, so much so that they often are able to manipulate rules to their advantage, without breaking said rules.
That's why this is such a hard problem to counter.
You're ignoring the only insightful part of the article: if a search engine succeeds at all, it cannot ban anyone, because some antitrust bureaucrat somewhere will cluck about it.
If this were true, wouldn't Google display all million pages that are vaguely relevant to my query on the first screen?
Their only job is to stack order the internet. Down ranking a site to the 1,000,000th result is the same as banning them. They necessarily do this all the time (like for every search with more than 10 potential hits).
For bot detection, when was the golden age? I'd say that, rather than follow the above description, it started as non-existent and there was basically a smooth descent downward to where we are now.
There were many multiples less people online. The internet economy was almost nothing back then.
There’s no comparing how things were in the 90s and early 00s to now.
Every social media company before Facebook faltered or began losing a good chunk of their user base fairly quickly. Until that stopped happening with Facebook and IG for over a decade now.
Reddit’s position has also been here for a decade now.
It’s actually because I’m not in an english speaking country that Google is failing me and screwing my searches for anything than can be somewhat pushed to local results…
I’m glad it works for you, it just reminds me how different are everyone’s expectations of good search results.
"pizza near me" - Google suggested a well rated place within 5 miles. DuckDuckGo suggested a pizza place 91 miles away??
"busted kids lip" - Google says gauze and a cold pack. DuckDuckGo says salt water??
"best monitor for mac" - Google says Dell Ultrasharp with 4.5 stars and 738 ratings on amazon. DuckDuckGo says BenQ 4 stars with 174 ratings on amazon?
I believe blocking an honest discussion is never going to do more good than harm. The original post doesn't even suggest that we should overthrow our Google overlord to replace it with whatever we find interesting. It's just pointing various limitations and issues that Google users are facing right now. And apparently from the replies here, there are even more aspects that people are have troubles with.
It's true building a good search engine is an gigantic undertaking but if Google (or someone else entirely) is aware of the current issues, they may have ideas to tackle those problems and then we can all share a better internet.
Same for the argument with democracy and any other similar arguments, pointing out problems within our current institutions doesn't remotely mean that we want to abolish everything and start from scratch.
I strongly suspect that's a very large part of what is making everything worse. Or, at least, there does seem to be a correlation in time between the implementation of ML and the degradation of the quality of results.
If Vladimir Putin sucks, why is everyone letting him be in charge over there? If Donald Trump sucks why did we ele-- If Barack Obama sucks-- The FBI sucks. Facebook sucks. C sucks. The WWW sucks. Fiat currency sucks.
Maybe the real truth is "the top N% of everything sucks?"
I actually don’t use google (search or other services aside form YouTube) very often. As for “everyone”, I assume habit and momentum. People are used to it so it’s hard to change to something else. Many people also don’t know what alternatives there are.
This article is really not that good, it suggests a bunch of obvious reasons why Google has a moat (Chrome, everyone's personal data) and misses some others (brand recognition). Then it goes and an explains reason why OP thinks Google sucks (to be fair I agree with him, but they're pretty subjective). Finally, OP claims that people are abandoning Google. Their single data point is that since 2009 there have been fewer searches for mortgages, because that couldn't possibly have been the result of an mortgage-driven recession in 2008.
469 comments
[ 87.4 ms ] story [ 1005 ms ] threadAllowing normal people to use computers is cruel in the same way making a dog order its food over the telephone is.
I would disagree. It is nice at the start but it shows the same things on the same search result forever. That is not something that you'd want in a Porn search engine, you'd want it to have some amount of randomization.
DuckDuckGo is ok too.
TL;DR there's no reason to default to direct Google.
I've been waiting for a Kagi invite for a while because I've heard good things on HN about it. Finally got one a couple weeks ago. Finally switched off of it and back to Google yesterday. The results weren't even in the same ballpark, despite me really wanting it to succeed.
Have you posted feedback about bad results on kagifeedback.org?
That said there have been a stack of new search engine posts on HN in the last few months and I may have to update my priors once I’ve had a chance to actually investigate the new options.
EDIT: Maybe I should note that I’ve also been relying a lot more on Reddit too in the past year since Apollo has a decent search interface for Reddit and I’ve gotten used to processing new subs quickly and getting information out of them. If nothing else I usually at least have a stack of new terminology to feed my search queries elsewhere.
At some point the internet has to go to a circle of trust model with real identities tied to online content of any sort. I see no other way to curb this pending disaster than being able to block bad actors and bad actors having very limited means to publish under an alias.
https://addons.mozilla.org/en-US/firefox/addon/g-search-filt...
https://iorate.github.io/ublacklist/docs
[1] https://www.ghacks.net/2022/01/20/google-ends-the-g-suite-le...
To me it seems Google has given ample time to either migrate off of it or to become a customer.
- Its location feature is better than DDG or anything else - It has gotten worse a bit, but if I tell it to show me results from Colombia, 95% of the results it will return are from actual websites from Colombia. DDG will throw anything from Latin America, for example. Not to mention the disaster with Bing.
- Its image search feature is still more precise than others. Reverse search won't return sometimes what I'm looking for and I have to resort to Yandex/TinEye/Bing, but still. Oh, and it can search for SVGs, which others can't.
- Double quotes aren't returning exact matches, but other operators are working fine as far as I can tell. Filetype operator is great and way bigger than DDG's, cache operator is great for looking for a cached version of a website that is not working at the moment, the minus operator still works (sans the advertisements).
I can't imagine trying to build a new search engine when the landscape is intentionally (if justifiably) hostile to new search engines.
These posts should almost be blocked from hacker news. ITs a fantasy. Its like saying that democracy has failed so lets replace it, replace with what? Its the best we can get given the alternatives, and its flaws will always be exploited.
There will be people that do not have favorite hobbies. There will be people with no ambition who benefit from being told what they can or should do to contribute. Those people, in order to have their needs met could choose to take on a Necessary Task, even if it isn't their favorite thing. Maybe they learn to love it and excel at it.
Don't want to do anything to contribute to society despite being physically and mentally able? No needs met. Meet them yourself, you magnificent lone wolf. Want your needs met but don't want to do Necessary Task that you hate? Find a purpose and pursue it.
Have the talent and desire to be a heart surgeon? Go for it, and you'll be rewarded. Want to make life sized statues of the Beatles out of used printer cartridges? Eh, fine but maybe also get a day job.
Sorry but Capitalism is a broken system that is not meeting mine or million's of others needs, much less enabling us to pursue what we're actually passionate about. I at least know that I am passionate about building and supporting technology but I still struggle to pursue it professionally because I had the poor luck to be born without the resources to get a piece of paper that says I know how.
It seems like everyone just thinks it's how you describe the contemporary world
No, what they're describing isn't capitalism. Not even close
It's much more difficult now to build a competitive search engine, but saying it's impossible and discussions should be banned is toxic. (And already basically proven wrong with existing competing search engines.)
* It blocks all the GitHub and Stack Overflow copycat sites out of the box.
* I can add additional sites to the block list with two clicks.
* It up-ranks (semi-)official documentation like the Python docs or MDN over blog spam sites.
* I can manually up-rank sites (and even pin them to the top of the results) if I'd like to see them more often.
Kagi is awesome because where Google thinks it can guess what you want, Kagi gives you an easy way to tell it what you want.
I was looking for this and it really annoys me how it thinks it knows what I want.
I wouldn't be surprised if there's some limit to prevent abuse, but it hardly gets in the way of my normal usage.
Apparently all of this is to offset the costs from paying Google and Bing to run searches for them (the site basically takes results from Google and Bing and then combines them together with their own special sauce) but with a pricing model like this it seems like Kagi will remain a niche tool solely used by the wealthy.
The main paradigm shift necessary will be to pay at all for something that was "free" and is provided "free" to this day - even if the quality is worse.
They need to support their efforts somehow. Instead of showing ads or selling your data, they are going to charge a fee to use it.
I'm logged into Google all the time. I do this to get sharing working across Chrome. Others doe the same with Firefox. Some don't, because privacy.
I hear you think logins are 1000x more hostile than paying with your eyeballs. How would you suggest resolving this clearly conflicted view with logging in vs. how the revenue is made on a given site?
But that is changing. Internet marketing companies are now teaming up with websites so the websites to their dirty work for them and you end up getting tracked anyway -- but with much higher certainty as to who you are. That means that now, yes, logins are more hostile.
The web continues to get smaller and smaller.
As long as it remains niche, it might actually stay that way indefinitely. I’m just worried about Google taking inspiration from them. I bet whatever “tricks” they use now would get a lot less effective if they did.
To my knowledge besides the old experiments with client side search, Google has never allowed (even paid) api access to their search. I thought they did not do this, because it would have allowed somebody to jump start a search engine that might eventually become a competitor.
Kagi seems to be open about their Google relationship, so I assume they have agreement in place.
This even feels like some experiment from Google to create a premium, paid, search product (like what they did with YouTube).
No, they are underestimating the difficulty of funding a good search engine.
I liked the runnaroo search very much as did several of my friends. The guy who ran it couldn't fund it. He shut it down.
Altavista (Yeah, that far back) had a nice feature where it would draw a cluster graph of your search results. So, if you searched for "python", it would show your results but would also draw a little graph and you could see that "Hey, there are two clusters here--programming and reptiles." You could then click on the "programming" node and the "reptiles" cluster would go away. It allowed you to drill through irrelevant stuff really quickly.
Note how that feature doesn't exist today--in spite of orders of magnitude more programmers being thrown at search, graph algorithms, and nifty Javascript web UIs. I wonder why ...
(/sarcasm in case you missed it. I don't wonder why. Such a feature would let you drill through irrelevant Ad and SEO garbage too quickly and would impact Google's revenue.)
we started out with topics and then moved to web, and are now folding topics into the web search experience, it's really hard stuff to get right
our first main filter was blogs, which is getting renamed to posts for a mix of reasons, adding forums shortly, along with other more specific topics
re: https://breezethat.com/ -- & better on laptop / desktop atm, premium version will be ad-free
Boutique search engines pop up all the time here on HN, but they can't compete fairly against Google, without the resources to crawl a billion webpages day after day.
At the time you would get tons of results from a myriad of small blogs, forums, niche websites. Nowadays it's Pinterest, blogspam and more SEO optimised algorithmic crap. If you're lucky you get a forum result that might actually be relevant.
Given the joke that if you can't Google it it doesn't exist, it follows that the Internet has considerably shrunk in the past two decades.
https://pdfs.semanticscholar.org/fc32/72302461b74217662085a8...
The original article's description of how Google search signals work doesn't even scratch the surface of all the signals it's using.
https://www.youtube.com/watch?v=IQuSuyUVkoE
However, I think Google has severely degenerated from just two years ago, when most of the problems were fully in effect.
Google is a bit of a product of the situation of scams being the easiest way anyone makes online.
https://neeva.com/ better than pretty much any solution (Qwant, DDG, etc) I've personally tested. It also indexes specific websites like StackOverflow, GitHub, and GMail.
Edit:
Neeva does require an account to create because eventually the product is going to require a subscription.
"To continue searching and access all of Neeva's features, create your free account. Already a Neeva member? Sign in"
Yeah, no.
1. I have more than one GitHub account (one is for work, one is for personal stuff). I would like to somehow keep them separate, but at least being able to add multiple GitHub accounts for indexing would be cool.
2. Could Neeva offer something for segmenting the parts of my online presence? For instance, there's personal me and then there's work me. I'd like to turn a different profile on when I'm doing different things.
3. I don't know if it's possible to partner with ProtonMail, but I'd love to index that and my Proton Calendar as well.
Also didn't know the free tier is permanent, but I think that's probably a good move. Thanks for what you do!
But essentially we all have de facto accounts on Google anyway.
They track and monitor usage and then map you to a user account when you finally login somewhere.
I'm totally fine with that being upfront and made clear from the get go!
If you trust these people are doing what they say, it's a pretty good deal for securing your internet spaces and trying to get away from google, imo.
Humans are incredibly good at solving engineering problems they can see from a mile away, although it takes time to solve.
1) New system comes out that indexes/controls/regulates a naively created dataset
2) Data consumers adopt that system and experience benefits
3) Data suppliers learn rules of system and take advantage of it to improve the positioning of their data, thus breaking the intent of a system built on the assumption of naive creation
4) Users complain about the broken system
5) New entrants realize that the original system actually solved the core problem really well, and there are no easy ways to solve the 'gaming the system' problem
6) Flawed system remains the best available option indefinitely
It's like entropy, there's just no fighting it.
EDIT: And to extend this beyond Google - do you see a lot of long text blocks in 7 second tik tok videos? That's because the creator found a way to game the algorithm.
Perhaps there is no border to be defined since nearly all SEO is abuse?!
The only acceptable SEO should be "provide website content that is more relevant/interesting for the viewer".
And where is the line between "provide website content that is more relevant/interesting for the viewer" and SEO? If I realise that (to take an example from something found on Google) I could take the sentence "Identify the best customers and convert more" and rewrite it as "Marketing automation helps you identify the best customers and convert more", is that SEO because I'm intentionally adding a keyword, or is that making the content more relevant to the user by being more explicit in what I'm saying?
This does of course not imply that these things can be proven, which implies that no action can be taken. Not everything that is immoral is punishable.
Sites with people hired to optimize the robots.txt file are ranked higher than those without. If we ban all websites that hire people to optimize their robots.txt, then a lot of the web isn't indexed and the search results suffer.
Or if we ban every creator on tik tok who posts a lot of text in a short video, then a lot of creators are going to be de-platformed and moved elsewhere.
The point of what I wrote is that 'humans are really god at following rules, so much so that they often are able to manipulate rules to their advantage, without breaking said rules.
That's why this is such a hard problem to counter.
Their only job is to stack order the internet. Down ranking a site to the 1,000,000th result is the same as banning them. They necessarily do this all the time (like for every search with more than 10 potential hits).
For bot detection, when was the golden age? I'd say that, rather than follow the above description, it started as non-existent and there was basically a smooth descent downward to where we are now.
There’s no comparing how things were in the 90s and early 00s to now.
Every social media company before Facebook faltered or began losing a good chunk of their user base fairly quickly. Until that stopped happening with Facebook and IG for over a decade now.
Reddit’s position has also been here for a decade now.
I can't find shit with DDG and the experience is like using Google from 20 years ago when it didn't have any of its bells and whistles.
I’m glad it works for you, it just reminds me how different are everyone’s expectations of good search results.
"pizza near me" - Google suggested a well rated place within 5 miles. DuckDuckGo suggested a pizza place 91 miles away??
"busted kids lip" - Google says gauze and a cold pack. DuckDuckGo says salt water??
"best monitor for mac" - Google says Dell Ultrasharp with 4.5 stars and 738 ratings on amazon. DuckDuckGo says BenQ 4 stars with 174 ratings on amazon?
how is this better?
It's true building a good search engine is an gigantic undertaking but if Google (or someone else entirely) is aware of the current issues, they may have ideas to tackle those problems and then we can all share a better internet.
Same for the argument with democracy and any other similar arguments, pointing out problems within our current institutions doesn't remotely mean that we want to abolish everything and start from scratch.
Has anyone considered the possibility that all this Machine learning and AI models is what made Google and YouTube searches worse?
The Amazon recommendations engine, the poster child of ML in 2014 isn't any good at recommending products that I actually want to buy either.
If prison food sucks why do prisoners eat it?
Maybe the real truth is "the top N% of everything sucks?"
http://web.archive.org/web/20070110035128/http://www.encyclo...
Edit: NSFW