Ask HN: Why isn't there a Google competitor emerging?
Despite all the posts about the declining quality of Google, we haven't seen any serious competitors arise over the last years. DDG is at about 2% market share, which is great but still very low.
Apart from Google's monopoly and the big technical challenges, what would a competitor need to defy Google?
331 comments
[ 18.5 ms ] story [ 683 ms ] threadHabits and ignorance. The casual web user is just fine with what Google has to offer.
And besides that: Google is convenient and everywhere. To get rid of it it requires not only the will but also the technological skill that the common user just does not have.
And that leads to the answer of your question: Create an alternative that is easy to use,ofcourse, that can be implemented by just "a click" and is has to be everywhere. On your phone, TV and computer.
I think that’s a contradiction. If it’s just fine for their needs, the they have all the relevant information they need. They know perfectly well finding another service would take considerable effort, might well lead to a worse experience, and there’s a high chance they’d switch back anyway. That’s an informed decision, not an ignorant one.
Also while the quality of search has declined it is nowhere near the point where casual users would be bothered enough to switch
That Google is still at 85% show that people really want to use Google, it is not just "whatever comes with the computer".
Google owns one of the major mobile platforms, where it is the default search engine. It pays competing Browsers like Firefox to be the default and is so tied in public perception that "to google" is a verb.
Also Google is fast and mostly reliable.
Also Google search integrates with other services like Maps for localized search so that one is torn back to them easily.
And if you were to get close to it they have tons of money to fight you.
The interesting aspect is that outside the broader search domain their approach often enough doesn't work. They didn't get social networking, they didn't get messaging (except mail) ... so they way to beat them likely is to find new segments and occupy that space (like Zuckerberg wants to do with "Metaverse," whatever that shall be)
A lot of it started happening when they took away the ability to search for exact terms, etc.
You can still do this. Put quotes around the terms.
But that's it, it was just "there" near the bottom. They put in all this fancy AI understanding and ML and NLP effort but when I go out of my way to tell the search that "this word right here is super important and critical", they just go ahead and append "page.includes(word)" as a filter to their super algorithm. Instead of using that signal to drive the search. No better than when sites used to stuff their html pages with keywords to trick search engines.
#grumpy
https://kagi.com/
How much will Kagi cost?
Kagi will be completely free during the beta-test period for all users. Once we officially launch, we are dedicated to providing the best possible search experience to our users at a reasonable price. We plan to offer entry level plans for as low as $10/month, unlimited plan at around $20-$30/mo as well as have bundles (to include Kagi email and other services), organization/team plans, family plans and annual payment discounts available.
We understand that for some user this may sound prohibitively expensive, but unfortunately we are not in the position to set the price point by consensus or market expectation, but by realistic cost of providing the service at a given level of quality in a way that potentially ensures sustainability and serving our customers long-term.
Beyond that, Google’s dominance isn’t going away due to their own “monopoly-like” positioning, their business relationships, the inertia of their massive public adoption, all of these applied within three or four other vital areas (ie: YouTube), their ability to pivot in response to anything novel that appeared to undermine their position, no apparent stories of Google executives hosting puppy kicking parties for the entire company to Satanic Panic everyone away from their products, the problem of promising companies being acquired because the owners (VC or founder or otherwise) are happy to be bought, the general “ick” factor of someone like Facebook attempting to enter the fray, some other things none of us have ever considered, random luck, and the initial conditions of the universe.
You have to have good results against the inconceivably vast amount of content out there, you have to be easy to use, you have to be free, and you have to be able to do all of this and more for the yeeeeeears it would take to wiggle into the space and expand while resisting the pressures above. Doable, wildly improbable.
However, I guess big tech companies have become part of the superpower games (USA vs. China etc.). Breaking up Google might just mean a Chinese company takes over. Can't trust the other governments to enforce similar market conditions.
So yeah, like you said, conditions of the universe :)
Well-crafted arguments showing where Google (or ilk) are anti-competitive are likely to gain some traction.
You can break up the phone systems because the child companies can provide similar levels of service. But how do you break up a single search algorithm?
In my mind it shouldn't make sense to found a company with the explicit goal of being purchased by one of the tech giants in a few years.
Many are never even really trying to get s sustainable business model and venture capital is fueling this machine.
One point of view is that it’s a more efficient way for the tech giants to develop new features. An internal team trying to do greenfield work will inevitably be slowed down by bureaucracy, where a startup can iterate more quickly without all the friction of things like performance reviews, HR exercises, and if I’m being cynical, pesky issues like user data protection frameworks.
It’s risky, but the payoff for founders is significantly larger than what an equivalent employee would get for leading an internal project.
One thing a competitor needs is a new and innovative technical algorithm.
Back in 1998, Google's PageRank was an innovative algorithm that calculated relevance based on counting back links instead of parsing the word counts in embedded HTML text like other search engines. This created a noticeable improvement in quality of results.
Nobody seems to have The Next Big Idea for a better search engine yet. Somebody did a Shown HN of a new search engine based on whitelisted curated domains such as reddit discussions. But there are many technical problems with that (e.g. Goodhart's Law & Hawthorne Effect creates bad feedback loop of gaming the reddit discussions which then poisons the search engine.)
Another technical idea of crowdsourced decentralized search index creates a very slow query engine which is a hard sell when web surfers are used to Google results appearing in less than 1 second.
DDG's idea of "privacy" is interesting, but being (mostly) based on Microsoft Bing's search engine doesn't actually create a quantum leap in better search results.
What's the next breakthrough idea that extracts the good signal from all the noise of a trillion web pages?
At least for me, I find myself adding “site:Reddit.com” all the time I need a good answer and not just a SEO-tricked ad page. Vector Search could be a technical solution for this. Although it’s fairly new and there is still a lot of research to do.
Seems like with enough actual humans I should never have to see another geeksforgeeks page come up in my search results.
Do the algorithms weight links based on visibility or relevance? A link at the top of a sorted content site HN should probably be worth more than one at the bottom, and one buried in invisible metadata somewhere should be worth 0.
But AFAIK Google no longer uses PageRank, or at least not the same version as we know it.
To me this is going to be manual ranking, but the biggest issue with manual ranking is not that you have to do it manually, but how do you know if you can trust users who rank pages. The Wikipedia system is a good start but not enough, and the second issue is how do you prevent its business model to kill its quality
However as for Google, I feel like even merely surfacing a range of relevant filter options can help their search engine, since it's what informed users are already doing by structuring a query when the first one produces crappy results. E.g. These results mostly contain links to Pinterest.com, would you like to repeat the search with these omitted? (or even just being able to collapse these site-by-site.)
Additionally I think modern search could be broken down into a range of sub-tools that exist beyond google or even online. One such example is searching our web history, perhaps browsers could produce a privacy-preserving way of storing information about each webpage and its images that the user can then search later (by either terms or images.)
IMO, all startups have a sweet-spot where they've figured out what the market needs, and they can be profitable, but they haven't yet sucked that bone dry to where they're forced to extend more tentacles to grow their revenue. For Google, for me, that was about the 2007-2012 time frame.
Any other competitor would have the same financial pressures eventually. Long term, it's impossible to "not be evil".
> They stop using it if the results are poor.
These are not laws of nature. There are so many reasons why most people would still use $X even if the performance of $X was underwhelming.
Here's another recent Show HN that demoed a new search engine:
Show HN: Goopt – Search Engine for a Procedural Simulation of the Web with GPT-3
https://news.ycombinator.com/item?id=30443738
I know this was offered as a bit of joke. But it prompted a really thought-provoking discussion and seemed to me to suggest a genuine paradigm shift in search or information organization. It is a novel way to extract a useful signal from all the noise. It brought to my mind "Dr. Know" from Spielberg's AI movie:
https://www.youtube.com/watch?v=x0QkgAuEPbk
Will the signal be a good one? It will need to get better and inevitably will. The thing that worries me: will it even matter?
Delicious sort of tried it but never executed well on the search side.
In a way it's a personalized PageRank algo with users linking to pages they like and to other users they follow. This always made more sense to me than a global authority ranking.
When looking for a babysitter you want recommendations from friends you trust, not some global ranking of popular babysitters.
Most long tail queries work well enough with Google and indeed most of the benefit of this approach would be in the head where ranking matters more. Furthermore nothing prohibits users from liking domains, not just individual URLs.
Create 3-5 different search algorithms, with names and understandable characteristics, then let me choose which one when searching. You may have some algorithm that prioritizes recency, or rapid changes, or more interaction, and others that prioritize longevity, citations/links, others that prioritize that type of information - prioritize regional or noteworthy events over historical info, etc. Have some algorithms that penalize spam/linkfarms more heavily than others. The git/so spam farms that come up when I search for an error message are simply useless, and would love ways to switch away and try other algorithms, without necessarily having to switch entire search systems.
Have algorithms that explicitly take in to account the search/click behaviour of others. Let me choose to have the search behaviour of friends/colleagues/famous-folks explicitly included or excluded in search results ranking.
Let people easily switch between multiple algorithms, and let 1000 gardens bloom (or something like that).
Also, determining better ways to 'search' non-text stuff. If virtual worlds become a thing, finding ways to 'search' those (whatever that might mean) will be big.
Image search still has a long way to go, IME.
The way it's likely seen by many tech companies is that the more decisions you have to offload to the user, the less advanced your system is. An old car with a lot of knobs and dials and levers is not more advanced than a self-driving car with a single control, select destination. Some people will still prefer the old car, but it will be a niche market.
"You're viewing through FOOBAR lens. Click here to view results from another lens".
Doesn't have to be that complicated.
You can add three more buttons on top of the tabs that are already there, but would their value for more advanced users be worth the extra friction/confusion for average users? And how would it scale if someone wants a fourth, fifth, or tenth "lens"?
Try images.yandex.com: it's vastly better than most others. It's not too good at identifying specific people but it will find prior exact uses of an image (like TinEye) but is especially very good at abstracting what's the abstract subject, color, pose, setting etc in a photo and show you similar.
[1] https://brave.com/static-assets/files/goggles.pdf
We now have a couple digital generations of folks who've grown up with 'web', and we've generated billions of 'documents'.
TIME ACCURATE QUERIES need to be mainstream.
"Let me search for FOOBAR and get results that would have been accurate/returned for Dec 11, 2006. Now search for FOOBAR and get results from Dec 11, 2014", etc.
The question is if such an approach would lure a lot of people away, I could see an entry angle might be a front end to google with an app that does that.
For example, someone comes up with a 'Finder' thing that's barely a comprehensive directory of the internet, and that, along with 'site search' aka for FB, Amazon, does better for the use case.
Most people are not 'searching' the web, they're often typing URLs or searchers for things they want answers to.
When I type a technical thing I'm really just searching 'stack exchange'.
On mobile, we use apps, which are different than pages.
Apple could come up with something that renders Google less than necessary.
Google will not be replaced directly.
Not really, because Google can do algorithms. For a new competitor to really succeed they have to start with something Google can't do. Perhaps:
- device/platform related. Google owns chrome & android but a competitor could emerge somewhere out of their reach, like only on iPhone, only on airpods, alexa, oculus, etc.
- being worthless to 99% of the market to make it 10X better to a 1% audience (e.g. developers, nurses, or even more niche like college athletes, jewish moms, etc)
- privacy/ad related
The implication of OP's question and perhaps the heart of the matter is whether Google is unfairly applying some impediment or anti-competitive behavior. My personal opinion is that they're not. I'm not a software guy so maybe I don't know the facts. I would very much like to be educated on that here, the same way I've tried to educate people here about display industry facts on the ground.
The absolute no 1 prerequisite for building a good search engine is having good benchmarks.
As a side note, I'm a publisher and Google Discover traffic alone has made me about $1k in the last week alone to just one single article by itself
https://developers.google.com/search/docs/advanced/mobile/go...
Some of the content on there is, however, fairly obvious clickbait, but there's also a super easy (working!) 'not interested' button in there.
You're going to need a quantum leap improvement over Google to unseat their positioning. It has to be very substantial to overcome all the various moats they have, not least of which is consumers being used to using Google, the brand awareness.
The next great search engine will emerge from a niche and conquer one segment after another from there. It won't be a massive general search engine that shows up one day (which is what the Google watchers have been waiting for forever - that new behemoth comprehensive competitor is never going to arrive fully formed). There's a decent possibility consumer Web search will be a later stage addition to said new niche competitor, consumer Web won't be its primary or initial target. They'll add on general consumer Web search as a "we might as well" offering once they conquer enough niches.
I love this because I want it to be true. However, iirc Google wanted to sell itself to Yahoo! for a million dollars and about five years later again for a billion dollars.
I know there are quite a few millionaires here but for me, five million dollars would change my life. I can’t imagine being able to turn it down.
Even if by some lucky circumstance Google doesn't become aware of the new kid conquering their beachhead, as they start down this expansion path Google will acquire them or throw resources at competing. A deliberate strategy of only going after segments that Google avoids (like social) almost inevitably attracts the attention of some other MANGAM.
Still, given enough dice rolls someone will eventually have a streak, but we'll be waiting a while for that to happen.
If microsoft quit search for some reason, DDG would go away with them.
By the very nature of solutions: a new company cannot beat google at general search. Google was and is search.
Like a tree that has grown up and shadows the entire land around it. There will be random small pockets of sunlight with smaller plants in it. But the big tree owns the area.
There can be a short term competition via extremely specialized search engines, but they will not rise to the same dominance. Google, in a way, is the entire idea of searching the web.
The next dominant generation has to be from a new paradigm, that makes web search obslete.
---
A relatable current example could be: youtube being dwarfed by tiktok. The nature of the new thing is such that it very naturally dethrones the old thing, without directly competing with the incumbent.
There will be specialized providers like vimeo, But at this point, youtube IS the idea of video on the internet.
---
Just note the context of this understanding: This understanding came from struggling to change the education system of my country. And then it seemed to apply everywhere I saw.
I'll round-off my reply with this quote:
“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”
– Buckminster Fuller
https://www.justice.gov/opa/pr/justice-department-sues-monop...
The only thing that would have a chance is if Apple developed a search engine and used it as the default instead of using Google. People are busy/ignorant and do not care and will eat whatever search results they are given.
But this might be a risk to Apple based on how many people use Gmail on their Apple phones.
It is not about quality anymore, it is only about market domination. The only thing that will bring about new search engines will be anti-trust legislation.
Google search itself is irreducible, at least in any productive way.
You can know there is a problem without knowing how to fix it. That should not stop you from talking about the problem.
Returning blue links is a thing of the past, as the Web of yesterday is long gone. Blue links always were about surfing i.e. following hyperlinks just for the sake of it since the main premise was most of them were of high quality and quickly proliferating.
All that is gone now and the links are a promotional thing how to get paid in one way or another. This is why Google results have been deteriorating, regardless of tens of trillions of archived pages on the Web. Google has lost the principal ranking signal years ago.
The next huge scale smart information system will be based on dense vectors (a few hundred dimensions) such as in AI but the key will be much bigger scale, of (tens of) billions of vectors. Contemporary AI works won datasets 4-5 orders of magnitude smaller, getting bogged down in gigantic transformer models such as GPT-3 with 175B+ parameters that take weeks and millions of dollars just to train. One might wonder what is innate knowledge of such a huge model, and it is not much as one can see for themselves as GPT-3 is now open (until Apr 1).
The future will be based on embeddings that are NOT contextualized i.e. no separate vectors for different senses in superpositions. Such systems will not be based on ads nor tracking as the resources required will be orders of magnitude less than what is currently required at Google.
If you're willing to pay for search there is an alternative emerging: Kagi. It's not a Google competitor, though. It's a niche product for people (like me) who are willing to pay a significant amount of money for access to a search engine whose creators make money by providing value to their users rather than by providing their users to advertisers.
IMO Kagi can only appeal to people who care about search enough to pay and don’t search enough to make it too expensive, which is probably a contradiction.
> We plan to offer entry level plans for as low as $10/month, *unlimited plan at around $20-$30/mo" as well as have bundles (to include Kagi email and other services), organization/team plans, family plans and annual payment discounts available.
I don't know where you're getting 5c per search from, but that unlimited plan would cap it at something I could live with and I doubt I'll need it.
I said a single search 'can cost' 5 cents, but in reality it depends on the search and how many filters you apply.
You can view the cost of your searches on your account. Each 'user search' can be multiple 'charged searches', which is why the cost can be >5c for some searches (e.g. if you search Dog, then click images and size -> large this is 3 searches charged to your account not one, and if you click through various options to refine your image search each click is chargable for instance).
If you click on the top result, find out it's not what you wanted so hit back, then click on the second result, this is charged as two searches if your browser prompts a reload - which depends on the page you visit (even though I would class it as a single search).
If you have a better business model, that does not involve building yet another ad-supported search engine, we are all ears!
I don't think most users would think this was 3 distinct searches, if it's just going through the steps you are required to take to get the single search you want.
I realize I am a sample size of one, but I run this exercise about twice a year, and sadly always go back to Google.