Developing search engines and personal assistants are the way to go. Too much information these days to digest , what a better job for a computer that helps us with the literacy aspects of it.
"The next Google can’t just be an input box that spits out links"
I think one strong contributing factor to Google's success is its simplicity. All the listed competitors add a lot of complexity imho. While all the customization buttons, knobs, lenses, meta crawling, code generation features etc might add some value for the advanced and technically skilled user, it provides rather little value for the average user who just wants to look up a cooking recipe.
So maybe when searching for "the next Google" the interesting question is not "what search features can be added", but "how can search even be simpler than using Google".
I think they each capture different markets. Google is simple because for most people it is “good enough”. Google isn’t going to spend a ton of time developing new features for the (large) minority that find it inadequate.
Google also has a few good things going for them:
- it’s the standard when it comes to search and is setup by default on just about every platform
- People use the brand name itself when talking about looking stuff up (no one says “just duck it”)
- They control the ad market, the standards for what a “optimized” site is, and the web standards themselves.
Actually, just taking the ads out of the experience can make for a simpler and better search experience. I think the google founders knew this too (https://www.reddit.com/r/degoogle/comments/rzr2n3/the_founde...). They just couldn't hold back the avalanche of revenue that search ads yields.
I work for Neeva, and this is a big part of why I left Google to join Neeva. There has to be a better experience, and it doesn't start from another business that works just like Google. It has to be a different kind of business. Neeva does not make money from showing you ads, so it can provide a different search experience... a simpler search experience, like the original google even, but it can go further...
With the Neeva app for example as you start typing in the URL bar, it will take your input as search suggestions (just as any other browser + search engine would) but instead of just showing you completed search suggestion, Neeva will show you the results from running those searches inline. The idea being that maybe those results will be helpful to you and make it so you don't even need to go to the search results page. You can just take the result right there from the URL bar suggestions drop down. Saves you time. Simpler.
Stuff like that. There's a swim lane of innovation and ideas on how searching and browsing can be better that is just really hard for Google to build, even though many of these ideas are thought of inside the walls of Google. They just can't ship them if they are stuck being beholden to their search ads model.
Another great example... ever wonder why Google isn't working to make it so Chrome doesn't have a million tabs at the top of your browser? It gets to the point where it is hard to get back to what you were doing. Me, I just end up closing the tabs, declaring tab bankruptcy. Google is okay with that because it means I have to search again. The Chrome team wants to fix this but it is hard to do so as it would result in people searching less often!
Again, just means there is opportunity for a simpler better experience to be had and Google won't be the ones creating it.
To be fair to Google, they do have the I’m feeling lucky button, which will take you directly to Amazon (example from the article) rather than showing you links.
Exactly. Notice how "I'm feeling lucky" is only on the home page and not part of the search experience when using Chrome or any modern browser where you search from the URL bar? Wonder if that is intention? Not a wonder at all.
The "I'm feeling lucky" button would never be added to Google if it didn't already exist. It was grandfathered in.
The problem with so many customization options is that it creates friction to users. We need something that just works out of the box. Just like Google did when it launched.
I mostly replaced my use of Google Scholar with Elicit. You ask questions, and Elicit gives answers, with citations. Elicit is powered by GPT-3 and it is amazing. Go try it.
The next Google should have less AI and be more deterministic. At least it should have a mode where it searches exactly what I specify instead of searching what it thinks I want. Also give me options to filter out certain domains. Again, let me tell the search engine what I want instead of the engine telling me what I may want.
I feel a lot of modern software is becoming very authoritarian. “We know better than you what you need”.
Google used to let you block domains a long time ago. I'm not sure why that was removed, but I have a feeling it had something to do with giving the expectation that you can block ads from a specific domain.
>But right after that, we’re greeted by an “Interesting Finds” section, which has a fun blog post by Derek Sivers, an article filled with stories of Steve Jobs in Japan, and some other cool things you can’t find on Google.
Exactly. Give me relevant links to my term, with no SEO spam. Don't correct my term, especially when I'm searching verbatim (which I almost always want). It'd also be nice to be able to downvote certain results.
The problem is that for every $1 someone is willing to spend to not have spammy results, a spammer is willing to pay $10 for you to see it anyway. And the more "trustworthy" a platform grows, then the going price for manipulating it will keep rising until they hire the right MBA who decides to squander the company's reputation for a quick buck. Seems to be a recurring theme at least.
Making spam unprofitable would be a good first step. Rather than trying to detect whether something is spam or not, just target how spam sites are funded: ads, analytics, affiliate links, etc and use those as a negative ranking signal.
You'll still get spam if that's all that matches your query, but now all it takes is for someone to make a page matching the query without the aforementioned items to outrank the spam results. You wouldn't need to append "site:reddit.com" to your queries because the (mostly) non-commercial Reddit results would automatically outrank all the blogspam and listicles.
If ads were downranked it would make a lot of spam/clickbait/listicles unprofitable overnight as they'd rank low enough that the costs of creating & maintaining the spam site/content would outgrow the returns from ad impressions.
> Making spam unprofitable would be a good first step. Rather than trying to detect whether something is spam or not, just target how spam sites are funded: ads, analytics, affiliate links, etc and use those as a negative ranking signal.
This would be a really interesting experiment: A search engine that ranks websites by the amount of Ads and other spam that they contain.
I think it would probably be worthless. SEO means money spent to the search engine company, and why improve UX when you could buy your seventh super yacht?
This is one of the reasons I love working for You.com.
SEO is killing Google and it needs to be addressed before it kills the internet. Having the opportunity to build an app that solves a search, instead of perpetuating a spam system, has been really rewarding... and we're just getting started.
It's crazy how quickly I got quality results for "java ed25519 bouncy castle" on both You and Kagi. I literally spent all day on Monday using Goog and DDG trying to find implementation examples of Ed25519 and AES-256-GCM in Java, trying every variation of keyword, verbatim quotes, and site scope I could think of, and ended up using GH gist search instead to find what I was looking for. The results on Goog/DDG were literally all SEO spam sites copying content from SO or the Bouncy Castle docs that I had already read.
I've been trying out Kagi as my default search engine, but will give You a try next.
>It'd also be nice to be able to downvote certain results.
When I'm researching stuff, I go thru a whole page of google result, and 'open in new tab'...90% of the time it's crap, and I really wish I could signal google 'this was useless'.
You want the information that you'll eventually get from visiting links, not the links themselves.
In fact often you don't even want the information, you just want to solve a problem. I don't know about you, but I don't like learning all about air conditioners and spending time finding the product available in my area with the highest quality/price ratio that fits into my budget. I just want the best air conditioner for me.
To go a step further - you probably don't even care about having the best air conditioner. You probably just want to reliably feel cool when it's hot at a reasonable cost.
Never mind the creators of those sites. Do they really want the Reader's Digest version of their pages scraped and presented to a user without getting the click?
I would prefer it returned "knowledge" instead of links but I hear you. Still, I think the fundamental flaw of all of this including Google is everyone looks for info in documents, while what you really need is a queryable knowledge graph where documents are linked to. Google made a half assed attempt at it but never took off, I'm looking forward to what that space could look like in the future.
Your answer reminded me of the Ford quote: "If I had asked people what they wanted, they would have said faster horses."
I don't mean that in a derogatory sense, I just strongly believe that in 20 years we'll look back at our Googling years and wonder how we managed to find anything at all. Who know what the next pattern of information retrieval will be, but I personally think it'll be an even bigger jump than the jump from reference desks and encyclopedias to Google.
I don’t think Ford ever said that, and I don’t think that quote is a good argument for not listening to users. Cars existed pre-Henry-Ford. They were just expensive. Many people could have correctly identified the need for a more affordable car. Ford’s innovation was actually doing it.
Also the Ford company’s dogged resistance to listening to customer wants (“Any colour you want so long as it is black”) hurt Ford as soon as they had to compete on anything but price, with GM finding success with “a car for every purse and purpose”. From 1921 to 1927 Ford went from controlling 65% of the American car market to just 15%.
All that to say: “innovator vision that flies contrary to common sentiment about what people want” is often overrated and can be a trap. That’s not to say that the customer is always right… but they’re not always wrong either.
putting aside the (idiotic) negative framing of third-world countries, that are called developing countries nowadays anyway, they also represent 3/4 of the World's population.
I'm longing for categorized YouTube site... When doing home project I can navigate with tabs: home -> security -> outside cam installation -> softfit mounted -> how to strip softfit / how to run cable / how to drill and patch cables hole etc.
Could be Wikipedia-style cultivated if enough people know and care.
> No, most people have never given a single thought to this.
Probably because they're stuck in a local maximum with no idea how much better it can be. They may not even be aware that there are sites out there that have quality non-commercial content if all they've been given for the last decade is spam to the point where spam has been normalized.
Also important for anyone actually thinking of taking Google on, very few of the features listed are things Google can't easily do, too. Attacking their strengths is crazy. You better have something both crazy good and hard to replicate by someone with more money than god.
Whatever replaces Google will be doing something that Google can't without causing them other problems. The first thing that comes to mind is make them choose traffic vs. advertisers (I don't know, if I had an idea of how to, I would not be writing this), but they're big enough that other wedges could start chipping away at their margins.
I think this comment is a bit strange in the present, considering search engines like duckduckgo, which is basically Bing promoted with a "we don't track" advertising campaign (also hashbangs are pretty cool). DDG is not at google numbers, I know, but you don't need google numbers to make money.
I don't think privacy is a very special angle to advertise from either, promising to remove amazon-affiliate blog-spam from results for example, would be a major feature in this space as far as I'm concerned. Being able to edit searches, and potentially gain some intuition for how the search space is set up, might be a much more significant feature, depending on how people take to it. It might flop but atm I'm excited to check it out
Actually, you are spot on. One simple feature of the Neeva app is that it shows inline search results as you type into the URL bar. This is because we aren't trying to show you ads, so we don't need you to visit the search results page (where Google and others show you those ads). We just show you the results straight away in the suggest experience. Now, this isn't going to show you everything you care about and you can still click to see the search results page. It is just handy to be able to quickly get to where you are trying to go and especially if it is likely to match what you are looking for (e.g., a wikipedia link). This is something Google cannot bring itself to do because it would be cost way too much in terms of lost ads revenue. There are other examples like this where Google and other ad-supported search engines just can't innovate, can't change the search experience. The current way of searching is too lucrative and there is too much business inertia around it. That's why Neeva is interesting and why I left Google to join and help :)
But that is exactly how Google searches worked on desktop platforms for more than half a decade (Instant Search), not some kind of a new idea. Given how long they kept that feature on, it seems pretty obvious that it can't have been the kind of revenue killer you suggest. If you can serve and display search results for a given possibly partial query, you can obviously serve ads too.
I was talking about mobile. As for desktop, Instant Search was serving up full page results instantly, which included ads. That's a different thing altogether, and of course, in the case of Instant Search there was plenty of room for both sponsored results as well as real results. On mobile there isn't.
I've long wondered about a search engine that doesn't index a page that has AdSense or similar code. There'd be a lot of collateral damage, but it would knock out a lot of annoying made-for-adsense sites at least. Basic thinking, but along the lines of what you suggested.
It's free while they're in beta. There's a "waitlist" but put your email on it and you'll get an invite within a week. Give it a try.
I've been using it for a couple weeks now on my work laptop and for programming-related searches its been great so far. And the usual annoyances that show up at the top of google and DDG don't show up on Kagi (geeks4geeks, etc).
It's really good in some ways, and very lacking (for me) in others. The search is fine. Good enough that I would switch. However, if I was out and about and quickly needed directions to Walmart, my normal flow with Google is
Pull out phone -> Safari -> type "Walmart" -> click on the map -> Maps app opens and starts guiding me
When I switched to Kagi, the flow went like
Pull out phone -> Safari -> type "Walmart" -> top result is Walmart.com... no address to nearest Walmart to be found -> Close Safari -> Open Maps app -> search for Walmart
And it got so annoying to have those extra steps. I know I can change my workflow and get used to it. But it wasn't just directions. It was other basic searches. Like if I needed a phone number for a local business. It drove me crazy. I really hope Kagi gets better at those sort of things. I want them to succeed. But it was just too much friction for me.
To be perfectly honest, I cannot even remember making that change, but also couldn't find any info on whether Firefox or some extension may have done this. I will leave it disabled, however, because its purpose seems to be analytics, and opting out of (at least some of) them this way seems like a good idea.
Since disabling it this way removes the `sendBeacon` function from existence, you should just be prepared to properly handle its absence.
Is it always about analytics? Or is it ocassionaly also a useful API for other tasks?
I know little about front end JS dev, but the docs say it’s just a way to send an asynchronous request (no response) with a guarantee that the the request will be sent out, even if the page starts unloading. That’s apparently not a guarantee with the two other to send requests - XmlHttpRequest and fetch api.
I agree this disrupts my muscle memory, but if we're honest, that particular workflow isn't necessarily optimal. It's just what Google has trained us to do. If you're seeking directions to Walmart, the most efficient (and one might argue intuitive) method is to open your maps app directly, and type in "Walmart".
I am a Kagi beta user, and I intend to pay when they start charging. I have wanted a paid ad/tracker-free Google for a decade now. I want to be the customer, not the product
The cost per search query is incredibly small. You could probably have <1% premium users. If your search engine is used by a billion people a day, 10 million paying users are still enough to pay a few thousand people developing your product (depending on your location).
I believe products can work that way, offer premium service and features for the very few that need it and a basic service to anyone else. In the end, the free tier is cheaper than what you'd have to spend on marketing otherwise.
I guess it depends on whether you consider "The Next Google" to imply that it becomes a the dominant company in the space, used by every "average user", or if it's enough to be a niche solution for highly technical users who prefer it to Google.
Another important feature of Kagi that I'm paying close attention to is: they are currently privately bootstrapped as far as funding goes.
To me the fact that Kagi is not currently VC funded is huge for me as far as adoption. Every customer facing VC funded startup I've worked at inevitably starts to institute increasingly anti-user practices while grinning and talking about "customer first!"
I know it's a huge ask, but if Kagi remains privately funded/non-VC I'll happily pay the moment I can.
I've only been using the service for a short while now but have been enjoying it a lot. The ability to blacklist domains has already dramatically improved my search.
Really? Or are you the one I should have refrained from feeding.
But if you must know:
First you need to collect a lot of content from the internet. From many different sites. With very different types of code structure. Broken html. More often than not behind some SPA JS code. Behind robots.txt files and bot protection efforts.
So the first problem to solve would be building a crawler at scale. That is able to crawl anything your users might want to visit but don't know of yet.
Then storage and retrieval. You need to store and update all this content your crawler collected. You need to enrich it with meta data and organize it for efficient retrieval. So that you can surface it to your users when they use your search engine. Indexing, structure, build g connections between content pieces. A lot of interesting things to think about.
Then there is the front end. Make it easy to search, to refine. Surface relevant content for search queries.
OH maybe I forgot, but you probably need to do a bit of engineering to make your system understand the users' search intent.
This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Bigger than that. I would applaud you if done with orders of magnitude lower than Google. Anyone would.
All of this is a long series of solvable problems. I should know, I've dabbled in solving most of them. This is why I suggest actually taking a stab at it before you dismiss it as impossible.
There are some problems that aren't as big as they seem. Parts of an SPA can't be reliably linked to anyway even if you find interesting text there, so you can just leave them out of the index.
Likewise, there isn't as great of a need to keep a fresh index as it may seem. The odds of a document changing is proportional to how frequently it changes. This is a bit of a paradox, where even if you crawl really aggressively, the most frequently changing documents will still always be out of date. Most documents are relatively stable over time. You can actually use how often you see changes to a document or website to modulate how often you crawl it.
The bad HTML is quite manageable. You really just need to flatten the document to get at the visible text. Even with really broken formatting, that's manageable.
The storage demands are also not as bad as you might think (most documents are tiny, sub 10 Kb), there are ways to lessen the blow on top of that. Both text and indexes can compress extremely well. Since you're paying for disk access by the block, you might as well cram more stuff into a block.
Most of the crawling concerns, in general, can be gotten around by starting off with Common Crawl (even if I do my own crawling, which also is finnicky but manageable).
> This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Right, so shouldn't the question be how to find the documents that are even candidates for being search results? Most documents are not ever going to be relevant to any query ever. Get rid of that noise and your hardware goes a lot longer.
I'm running a search engine on consumer hardware out of my living room that can index 100 million documents. Go a bit higher budget than a consumer PC, and you've got 5 billion. That goes a long way.
> Get rid of that noise and your hardware goes a lot longer.
What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
> I'm running a search engine on consumer hardware out of my living room that can index 100 million documents.
That's extremely cool. I would love to know more. To me an impressive feat already.
I think I was editing the comment while you were replying. Sorry about that. I was just adding to it though, didn't really rug pull on your response so I think it's fine.
> What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
Now this is a proper difficult problem with (probably) fairly subjective answers. I do however think it's something that warrants serious investigation. It's probably a decent candidate for a machine learning model combined with some manual tweaking for sites similar to wikipedia or github that have absurd amounts of parallel historical content.
Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
> That's extremely cool. I would love to know more. To me an impressive feat already.
Yeah it's at <https://search.marginalia.nu/>. I've built all the software myself from scratch in Java[1], and I'm doing my own crawling and indexing. The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
I do use a MariaDB database for some metadata, but I think it will have to go as its hardware demands is becoming a serious bottleneck.
[1] Despite using Java, I should say regarding the index. This is approaches sunk cost at this point. Building a search engine index is not something Java is at all suitable for, its limited low-level I/O capabilities is incredibly handicapping.
> I [...] didn't really rug pull on your response so I think it's fine.
No you didn't. All good. And I learned a lot from the extended answer. So I am thankful for the explanation.
> Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
I can totally understand the feeling. There are quite a few things that I'd like to go deeper into either at work or in private. But alas time.
> Now this is a proper difficult problem with (probably) fairly subjective answers.
I agree. And I don't have answers ready. A lot boils down to preference. Personally, for example I prefer written content over video. Except in a few areas were I like (some) explanatory videos. To me it comes down to the question of how easy I can skim the content when I am looking for an answer.
On the other hand - for deep immersion into a topic I use multiple media formats.
In terms of web search I sadly nowadays need to sift through a lot of seo-fied content that is there either to build a (personal) brand or to attract clicks for advertising revenue/affiliate revenue.
So in principle I agree with you on the noise problem. Still I also believe that there are real great gems to be found in the long tail. When I still feel like I came late to the party, but when I started out in the web in '97 there were so many lovely, quirky sites. So many places that people had put a lot of time, energy and thought into. And sites so packed full of information that I came away not only with more knowledge, but in awe that somebody would give this knowledge away for free.
There also were quite a number of horrible sites (my first ones probably included). So there was a noise vs. signal problem back then. Maybe not to the extent today, though.
> The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
Call me impressed. Sounds absolutely cool.
So even with a raid setup for redundancy this is doable.
May I ask how you decide to add me content? Do you follow links? Do you use other search engines' results as a starting point?
I could probably shoot many more questions, but don't want to be a nuisance.
> May I ask how you decide to add me content? Do you follow links? Do you use other search engines' results as a starting point?
I initially did basically a DFS-walk originating at a few websites I liked, with some filtering criteria that deprioritized websites that didn't look too interesting. Now that I have a fairly comprehensive mapping of the space I want to index, I use a few factors like frequent outbound links from highly ranking domains to inform which new sites to index.
> I could probably shoot many more questions, but don't want to be a nuisance.
Why? I don't change my reply based on the author. I reply to a statement to the best of my knowledge regardless of the author behind it.
And I learned already a lot in this thread after the explanations unfolded.
The initial statement sounded exactly like the armchair "experts" one so often encounters. Actually this was for a long time the first time that there is a person with substantial experience in the problem space behind such a statement.
> The initial statement sounded exactly like the armchair "experts" one so often encounters.
Maybe - but [marginalia_nu](https://news.ycombinator.com/user?id=marginalia_nu) isn't an armchair expert - they've actually implemented theor own publically available search engine - which is linked in their profile.
I didn't say they are. Only that the initial comment sounded like that. And in the thread above we discussed a bit about their achievements. I really liked it and learned a lot.
The reverse index is 180 Gb, on an SSD. I do think using SSDs are a major part of why this is possible on consumer hardware. I'd need a lot of spinning rust to get the sub-100ms response times I can get it to when the index is warmed up.
Should be said I do wear through this SSD at a pretty alarming rate. I'm at 193 TBW on this disk since I started using it as an index less than a year ago.
I do have a bunch of mechanical drives I use for archiving and as intermediate working areas as well, but the index itself is on an SSD.
Thanks - I'd be keen to try this at some point, if anything just for personal usage. I've got more than enough hardware CPU & RAM-wise, if all it takes is getting a few TBs worth of solid-state storage it seems like a no-brainer.
> - who pays for the service (ads? users pay? Average user will never use a paid service if a free one is available)
- how to resist attacks against the algorithm (Google has been fighting spam for decades)
The solution for o both of these might actually be a paid service. If you have a paid service, there is a possibility of it being profitable with much fewer users. As an example, let’s say you have 1,000,000 users at $10/month, that is a $10,000,000/month which might be enough to run the service and provide a comfortable profit.
With regards to the spam issue, the fact that you have a small user base would be to your advantage. Because there are so many Google users, it is in websites’ economic interests to spend money to try to game the algorithms. With much fewer users, your paid search users may not be worth it for the sites to spend money trying to game your algorithms.
It will still pay to put ads in to paying customer's feeds. It's more or less inevitable if the customers tolerate it. If you think that's impossible I'd point to how streaming services are now serving up ever more adds.
The spam issue can trivially be addressed by implementing actual penalties for rule-breakers. If it takes a long time to acquire a good reputation & ranking on the search engine, you're unlikely to risk it by doing something nasty in fear of your domain, keyword or brand name being banned for a long time.
Raising the cost of spam would be a good first step.
At the moment, spamming Google seems to be trivial with no long-term penalties if you get caught doing something nasty.
A simple rule (manually enforced on a case-by-case basis) that would ban your brand/domain for a year if you get caught breaking the rules would get Pinterest into compliance from day 1 for example.
Using ads/analytics/affiliate links as a negative ranking signal would make a lot of blogspam/listicles/clickbait disappear if their only funding method immediately makes them rank much lower below where they are no longer profitable.
This would be easily exploitable by a competitor. For example, search engines (used to) rank back links - that is other domains pointing to your domain. Some bad actors took advantage of this by creating rings of sites that voted each other up. Google responded by punishing the behavior. Then, competitors started taking advantage of this punishment by creating a network of sites that backlinked to a competitor, so they would get punished instead.
This isn’t a hypothetical example - Google actually includes in their webmaster tools a “disavow links” capability so sites can avoid getting punished for bad actors trying to make them look bad. But you can imagine if the penalties were even more severe other folks may get caught up in an unforgiving dragnet with no judge or jury and no way to appeal.
My main point is that people will find ways to game the system, and usually sharp edges (“harsh punishments”) on any system will be taken advantage of by actors, and unfairly penalize others.
Agreed, I'm not saying this is the end-game or that it will be perfect. But a simple rule (that's actually enforced) saying that you are forbidden to serve a different experience to the Google bot vs a normal visitor would take care of Pinterest for example, and they're not even doing that despite it being a major complaint especially in tech-circles where Googlers no doubt lurk.
Average users may not pay, but specialized users may pay and pay more than enough to subsidize some sort of free tier.
Not to mention, if free search engines keep devolving into an endless sea of spam, people may have no choice but to start paying. There's plenty of things out there people pay for not necessarily by choice but because there's nothing else out there that would accomplish the task at hand.
Adding on to this, customization is nice but customization is not why DuckDuckGo isn't as good as Google. The reason nothing is as good as Google is because Google indexes way more content than every other service that I'm aware of
Lately Google has gotten so bad that I've even occasionally brought up Bing (which I've often referred to as the Zune of search engines) and gotten better results. It's looking more and more to me like Google has stopped trying to improve things and are simply milking their dominance for all it's worth.
Search is the wrong way to look at it. It needs to answer questions, like an oracle.
Anyway, Google is getting less relevant because the technology is getting better than "good enough", and any additional tech that Google adds is not really all that useful. It's like PCs. You don't need a faster one because your old one can do word processing just fine.
This depends on user preference and query. Some users want answers to a question, other users genuinely want to find websites.
If I search for "what's the weather now" I probably want to get an answer "like an oracle" as you say. But if I search for "riaa vs napster" I don't want an "oracle answer", I want a list of several websites so that I can learn more about the case. I'd like a search result for court documents, but also commentary from news websites and blogs, and maybe wikipedia. I want to open those tabs and come out the other way with more information, but there was no "oracle answer" to be provided.
An oracle is how Google sees itself. However, their quest to be an oracle has come at the expense of losing their edge at searching the web. So now there is an opening for another service to be better at search than Google.
> DuckDuckGo and Bing are not true alternatives – they’re just worse versions of Google.
I disagree. I use Bing for about half of my searching because Google simply struggles with things like exact phrase matching in many cases. Where Google falls off, Bing tends to succeed. Though Bing also sometimes lags behind, where Google tends to do better, hence my split usage.
> Glad to disagree, DuckDuckGo is good enough if not great.
For anything that isn’t the least bit obscure or technical. (Same with just about every search engine)
IIRC, someone on hn said that search engines are only good for things you already know are there, not for finding new information or different perspectives.
Happy to answer any questions you all might have about Neeva. I left Google to join Neeva about a year ago. Got inspired by the opportunity to make a better product. AMA :)
In my view Google could possibly be the next Google. They're leading in AI right now and if they can get the magic we've seen in some of their papers into products like Search or Google Assistant it would be major moment.
People aren't complaining about Google because of a lack of AI, but because of product decisions they're intentionally making, presumably that increase their profits, but many of us would be overjoyed if they simply rolled back to their 2012 algorithm.
Pathways was the first time when Google got to a level where they could collapse multiple searches into one. They needed working chain-of-thought prompting to make deep learning useful for researching solutions for problems.
> Everybody has different preferences of how they want a search engine to look and feel.
I will never ever ever ever spend time going into the settings of a search engine to customize my search experience. Not when Google does a good enough job for most of my tasks.
My problem with google is that it often tries to figure out my search intent and then return results accordingly. the attempt might be commendable but for many types of searches they fail miserably. when they misread my search intent, the results are completely awful.
and to make matters worse, 95% or more of the web isn't even being shown to users and so I keep getting the same stupid search results: which is fine for many cases but not all cases. sometimes I need more than just another post by NBC.
I will say, google search is very good at technical searches, that's for sure. But, for example, they totally suck if you're looking information or doing research on financial matters or economics.
>I will say, google search is very good at technical searches, that's for sure.
Unless your technical search, for example, contains jargon with a similar spelling to or other sense synonymous with a common term that Google will helpfully also consider to match your query. I think it's actually rather mediocre for technical searches now that they've gone so deep into stemming and such and lessened the ability to use operators to refine queries. And, of course, the lack of support for symbols makes many technical queries extremely difficult.
Google won because they started early and had a right algorithm. They then had the scale to grow with the internet. It would be very hard to start a new general-purpose search engine now (capital, monopoly, tech)
Nobody seems to have The Next Big Idea for a better search engine yet.
I think the bigger reason why search isn’t getting much better is that incremental technical improvements are being offset by Google trading their dominance for greater shareholder value. A safe business can take away or buy-and-bury features that are better for the customer but worse for the business’s top line. Things like the Power Search API, or search that integrates with other platforms that hurt Alphabet owned businesses.
Traditionally big innovation breaks some socio-political constraint on market participation. I don’t see how AI will do that yet.
NLP generative models that you can interrogate with natural language prompts, and which generate coherent responses together with citations for sources, is definitely the future of search.
I don't know anything about AI or the relationship between market participation and innovation. All I know is that LLMs are a useful tool that is already displacing search in many parts of my life.
I see. These sound like two different products, Google Search vs Google Scholar. I can imagine AI + natural language processing is dramatically changing search performance of dense-detailed text.
I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end. I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't. There's a huge cost associated with having to make choices, and one feature of successful modern apps is that they're frictionless. That's why TikTok is so successful. There's no login, no user chosen social graph, everything's abstracted away.
And that's by the way why Google is still successful as well. Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else. The only way to beat that is to make it even better while not making it more complicated which is very hard to do.
This is the thing. I'm the power user in my circle. People comes to me for suggestions about pretty much anything involving tech. Sometimes just because they see me with different stuff.
So if I'm not the average user I may look for other options, as other people like me may do. If I find such options reasonable for the average user, that will be my recommendation to them.
Tiktok optimizes for momentary engagement and fast-paced social content. It turns out that's a niche that people want filled, and the rapid bouncing of ideas between users has created some very fun content which sometimes escapes containment so I can watch it. But not everything is going for the same goal, and honestly I wouldn't be surprised if more than a few players could really get in on that space. For other use cases, customization may well win out over a totally frictionless experience. A search engine is a tool, and benefits from more options much more than Tiktok does.
Tech power users (and power users in general I would say) might have more money to throw at the problem. "Nicheness" isn't necessarily a bad thing if your niche is profitable. I heard somewhere that power-grid-scale transformers are have insanely long lead times so the industry most be pretty niche (when's the last time you needed one of those?) and yet I think we can all agree that the equipment is valuable and I bet those manufacturers are making bank.
I agree, none of these are the next google. Except perhaps YouWrite which taken to the limit is asking an AI for the answer rather than searching the internet.
> a simple box where you put a question in and it gives you answers
Google is very good at this, but this is exactly what I _don't_ want in the "Next Google". I want a _search engine_ for the web, not an answers engine that tries to know what I want better than I do.
Search the web. Give me links to websites. This seems obvious to me, but everyone is trying to be like Google.
I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
The death of the web is because most people don't want what you want. They don't mind walled gardens, so long as they are easy to use and have the content and connections that they want to see.
The audience of HN is extremely skewed towards preferring systems that allow tinkering but that's not what the market wants.
People don't mind walled gardens because for a while they've been good enough if not better than the previous status-quo. However, those walled gardens are decaying such that there might actually be demand for something better if it existed.
> However, those walled gardens are decaying such that there might actually be demand for something better if it existed.
Some walled gardens, like facebook, are decaying. Others, like tiktok, are vibrant, still fresh and new.
Walled gardens are a natural result of the pursuit of capital. If you burn VC money, but create no moat (it turns out the walls of a walled garden are also a moat, weird huh), then as soon as you introduce a clever algorithm that introduces 50% more ads to extract profit, your users will all leave.
As such, a business is incentivized to build these walls and moats.
How do we avoid this?
Well, we do have examples. Mastodon and other open source projects eschew walled gardens in favor of free software ideals. Web3 embraces a certain "decentralized" vibe which lends itself in this direction. Universities, and other public-ish institutes like DARPA, created the original internet and many of its technologies.
Unfortunately, open source projects will struggle to advertise or find users. They do not have the initial capital to get as much momentum as the VC-funded alternatives. Web3 seems surely doomed to end up also building walled gardens for the crypto-anarcho-vibe is only skin-deep, and many a regular business is now highly involved.
This leaves government entities. The government is the one group that is both well funded enough, and has motive to create protocols which prioritize the user's freedom over profit (after all, the users will pay taxes regardless of how high the walls are).
It seems to be out of fashion these days for the government to actually do anything though, so perhaps there is no more chance of that than of a free software project managing the same.
What the market has produced is not ipso facto what the people want. The market is simultaneously optimizing many things. Walled gardens are much better explained by companies benefitting from not having to allow their competitors access to their customers than by "being what the people want".
This. People don't realize that the early web was elitist. Now, the entire population is online. And, as you said, most people simply don't care about the stuff we care about.
That's also why "Google's search results are soo bad." They're not. For the bulk of Google's visitors, they're good enough.
Desiring choice isn’t elitist. Early web adopters were passionate and willing to put in more work. Nothing wrong with that, and nothing wrong with liking simple default settings either.
Wondering if, by “elitist”, GP meant more like “out of reach to many laypeople because of learning curve.” Is there a good single word for that? “Difficult” and “complex” aren’t quite right.
Anyway - you’re right, nothing at all wrong with wanting choice. I think the point being made here though is that “layperson gravity” / mass market appeals / lowest common denominator is going to mean that tuneable web search will be a niche product, forever. Even if we’d both like that niche.
Not trying to be awkward, but even back in the early days of Netscape, it wasn't like you had to recall any arcane commands to use most search engines.
Sure, they all had limitations then, such as not necessarily knowing similar word senses to create more nuanced searches, but that was a fairly level playing field.
Is it simply that it was initially a bit obscure and not everyone had found out about it? That's not really a barrier in terms of difficulty - as soon as people got into the "in crowd" they could use it just like the rest of them.
When I started all modem calls required remembering 'atdt'
When netscape came out if you were lucky to have ppp access setting it up on windows 3.1 was difficult but once you were up as long as no one picked up the phone you were fine.
But even this was too difficult for the average person who lived in an aol world where the internet was limited to aol.
there are so many options, many complex, but many too are simple. One of my personal favorites is using github pages, you can host a plaintext html document for free in about 10 minutes (including downloading a text editor and github desktop)
Cost was one. Hardware wasn't cheap and it would seem to have no practical use except to satisfy one's curiosity. A PC for the kids cost about half of our family's monthly gross income. A modem would be 10%. And the phone bill caused tourette's like symptoms more than once.
>Er, isn’t the whole point of Google tracking you and knowing your mothers blood type so they can give you better search results tailored to you ?
AFAICT, quality personalized search results isn't the goal for Google here.
I think the point of all that is generating revenue via advertising sales.
And while providing high quality search results might once upon a time have been a goal, both as a goal in itself and a tool to drive user adoption/engagement, that's no longer necessary as they have a (relatively) captive audience and a (relatively) captive customer base (advertisers). As such, quality search results are no longer all that important.
I think this is exactly why Google's search has become trash. They don't need it. Even when their goals changed from "making searching the internet better" to "Making money and collecting data on everyone" they still depended on search to see what people were doing both online and offline.
Now they have millions of cell phones collecting data and the GPS location of everyone offline, they run extremely popular DNS servers to see what websites people go to, most websites (including educational and government websites) include google's trackers so on most websites every page loaded will ping at least one of google servers. They've got people uploading their personal and work documents to their cloud. They are swimming in data collected from sources outside of web search. Between that and the lack of actual competition it's no surprise they aren't investing in making searching the internet better for people. In some ways they profit from search results being terrible. If anything beyond the first few results on the first page is filled with spam and irrelevant websites the top few results to any search become even more valuable.
so they can give you better search results tailored to you
That depends on whether or not you agree with Google's definition of "better". What is better for Google is a set of results that make you most likely to click on an advert. Failing that, they want you to click a link, see the site, and either click an ad there or quickly return to Google where you'll click on an ad. The worst outcome for Google is that you'll click a non-paid search result, like it, and stay on that website.
This is the fundamental problem with search engines - if they work well and give you exactly what you're searching for first time then they won't make any money. A lot of what Google does is subtly trying to give you results that look great but really aren't.
But to be honest the internet is still the internet. The web still exists. Any lamentation at the loss of the "old" internet is that you don't have more angry uncles spewing political rhetoric on your motorcycle forum.
Do you want the unwashed masses in your specialist forums? I certainly don't. Seems to me things are working pretty well. I do worry about the next generation but to be honest, the same was said about me.
Instead of complaining about the state of the web just be the change. Host your site, run your forums, live your life. Stop worrying about how other people should live their lives. Take some youngsters under your wing and show them how a keyboard works instead of a touch screen.
> Take some youngsters under your wing and show them how a keyboard works instead of a touch screen.
This x1000.
The only way to ensure that future people can continue to maintain things is to teach those who are interested.
While I think we need to scale back the opaqueness of tech to the average user[0], there will always be some who just know more.
[0] By which I don't mean "more configuration options", because the average user doesn't want those.
I mean make it easy to access the source - and just as importantly make it easy to use changes to said source, so those who think, hm, I want this, can learn to fix it themselves, or find someone who can.
Wonderfully put. I would add that we need to see things for their scale and proportion. Yes, the simple walled web is bigger than ever. So is the hacker community, and this website (so far as I can tell), Reddit (for all its faults), etc.
FYI: Wiki says: <<Eternal September or the September that never ended is Usenet slang for a period beginning around 1993[2] when Internet service providers began offering Usenet access to many new users. The flood of new users overwhelmed the existing culture for online forums and the ability to enforce existing norms. AOL followed with their Usenet gateway service in March 1994, leading to a constant stream of new users. Hence, from the early Usenet point of view, the influx of new users in September 1993 never ended.>>
> Do you want the unwashed masses in your specialist forums?
Having the choice between "unwashed masses" and "curated to suit political agenda" I would firmly choose the former.
Before I get attacked on political agenda angle, please understand that it takes trained professional actively trying to remain objective to have impartial moderation. Many professional discussion moderators fail here and you just cannot expect neither unpaid volunteers (doing things out of passion which is not impartial in the first place) nor paid employees enforcing company policy (again very much partial) to remain objective.
The goal of moderation should be to allow for multiple angles of thought to emerge and not get drowned. Give me the tools to see those non-mainstream positions.
The unwashed masses are often filthy and malicious (on the internet).
Moderation, of the governance and oversight meaning, is unforunately very necessary to keeping any pool from becoming a cesspit. By some combination of technical, volunteer, and participant effort (membership fee, or service requirement), areas can be kept decent.
Without this, decent and would-be participants are run off by spam, scams, trolls and harrassment, political rants, religious rants, etc.
Hence what happened to Usenet. I was active on comp.lang.c comp.lang.c.moderated comp.sys.3b1 comp.sys.3b2 back in the mid 1990s, by 1998, it had become a total hot mess. The moderated groups imploded, or shutdown, when the moderators just gave up. In particular, comp.lang.c.moderated was a fountain of knowledge on C language implementation specifics, then a few really pedantic asshats joined, and their MO was purely slandering and shouting. They had agendas, and they had no interest in sharing knowledge. I think a couple of them thought they could affect the evolution of the language, from outside ANSI, to suit their own personal opinions. Nasty little dictators. I completely dropped off usenet around 1998. In retrospect, I hung on for to long, hoping for a return to rational behavior.
Admninistration web sites are now forcing you to use google(blink/geeko) or apple(webkit) based browsers.
Asking nicely to restore interoperability with noscript/basic (x)html browsers does not seem to work, namely keeping the door open to alternatives requiring a reasonable effort of development, and not the army of devs of big tech.
> For the bulk of Google's visitors, they're good enough
I'm not sure that's true. Just to pick an example, basically any type of product search now leads to auto-generated spam/borderline-spam websites that scrape reviews from Amazon, which are themselves often fraudulent.
I guess you could say that's "good enough" but only in the sense that someone being scammed at some point consents to something that makes them a victim of the scam.
For many searches, you'll see "reddit" autocompleted to the end of the search because Google's organic results are not good enough and people are trying to weed out the spam somehow, albeit in an imperfect way that restricts results to one site.
DuckDuckGo is horrendous for spam and malware distribution. I have stopped recommending it to non-technical people because they can't recognize the obviously evil/bad-pattern sites that are on the first page of many DDG searches I do
User manuals for stereo systems, specs for old hardware. Anything that can be generically referenced, maybe a bit harder to find. Someone can throw up a webpage that SEO's on "manual" or "user guide" and when I type in "Sony DTH-345 user guide" I'll see results like "manualsonline.xyz" that are clearly bogus.
I can't lab it up now, but that is the gist. And DDG has it while Google rarely has the same level of it.
That's my understanding. And while I appreciate that Bing is the "baseline", I would be interested in what it would take for them to do additional cleanup on results. Blacklisting domains should be a no-brainer. "Safe search" for suspicious sites, not adult theme.
Thats the advantage Facebook has, people trust each other if they know them or a recommendation comes from a friend of a friend, but you cant really see who is recommending something on Amazon, Reddit and other sites.
What exactly is Google supposed to do here? They're returning what's available. Beyond that it goes from search engine to active curation and recommendation - and they are actively moving in that direction by replacing results with "answers", which I find to be much worse.
Nothing, really. It's the "what's available" that's the problem. Quality paid services can't gain market share because free is a anticompetitive. Consumers will almost always choose it if it's not complete garbage. This throws all the metrics off for search because quality paid services become a statistical anomaly.
Have you read any of Google's advice to site owners? It amounts to "provide your users with what they're looking for, and do it with a fast and easy to use site". If you think SEO is an easy hack to get any site to the position 1 for any search, you're just ignorant of the landscape. All the "SEO" crap you see about keyword density or whatever, is just an industry trying to sound different and like they've got the key to position 1.
> They could start penalizing sites that show different things to googlebot and a real browser again.
Google is ridiculously good at this. Would be keen to see examples of a site that does this in a meaningfully negative way that ranks well for searches that matter.
I don't fully agree with all of your other points but I can see why you'd mention them.
I've been using Kagi search and I find the same stuff there. I can block them from my personal results as I encounter the garbage, but it's there by default.
And the people trying to get their garbage spam sites to rank high are wise to the common tricks people use to attempt to get better results. Just today I accidentally clicked on a result that from the title looked like it would lead me to reddit. I should have paid attention to the URL because instead I was sent through a bunch of redirects to some terrible spam site.
Part of it is, that everyone already has heard about Google. Trying to get another search engine that people trust and have heard about is hard, not to mention it has to work decently.
Worse than that are the SEO results that don't give you what you want at all.
For products, one of the worst offenders is Target. Yes, Target. They show up near the top of about 60% of my searches, including for things they simply do not carry. But hey, they carry a similar product that you might be interested in! (not)
SEO results make Google money, but they also make the results suck.
Anyone I watch older than 50 years old is constantly frustrated by google, they just don't know what's good, what's not. I think they use it because it's the "norm" and that is the only reason.
And as far as everyone else.. I have no idea to be quite honest. I have a fairly big handful of non-techie friends who think google sucks but is good enough so there is no real incentive to change.
With any habit people don't change it unless they have a huge reason to. No one stops eating double cheeseburgers until their doctor literally tells them they are going to die. I think that's what we're up against here.
Hey, now wait a second there, you young whipper-snapper. I'm sitting here reading HN while I install Debian on an XCP-ng VM, and I hear I can't understand whether Google results are good or not? There are plenty of us over 50 that actually lived the early internet and watched it first-hand transform into what it is today.
This is a great reply. Real question: Why hasn't the same happened to HN? To be fair, there are some arguments that play out over and over again: SF crime & homelessness, Bay Area housing, US taxes, central banks "printing money", office vs work-from-home, feeling forced to adopt liberal social views / stances at a tech company (Black Lives Matter, LGBTIQ+), etc.
And somehow, there is still enough good, new content to keep the conversations fresh. Some of the best are when someone shares a personal story, then there are tens or hundreds of follow-up posts -- generally supportive or inquisitive.
> Anyone I watch older than 50 years old is constantly frustrated by google, they just don't know what's good, what's not.
Yes, and now imagine these same people, most of whom never bothered with basic google search options (eg. excluding entire domains, mandatory keywords, ...) presented with a search engine with a plethora of knobs and levers to tune it to their liking.
not us yahoo / Altavista / or whatever else was around at the time users. Google was so clearly better right out of the gate that I distinctly remember having conversations conveying the shock and awe at the quality of the results. Before Google everything was mediocre at best.
I’d say the for the bulk of Google’s visitors the results aren’t merely good enough, they’re insanely great.
Now, that said, the G monopoly is horrible and needs to die.
It isn't elitism. It's competence. A system designed by competent users for competent users are going to be different from one made for general masses. There's nothing elitist about that.
> This. People don't realize that the early web was elitist. Now, the entire population is online.
That's an excellent demographic point.
A counter-argument is that the move from port 80 to port 443 is the elitist drift. Instead of innocently putting information out there and sharing in an egalitarian fashion, we now have certificates and gatekeepers and lions and tigers and bears, oh my!
In an age of free SSL/TLS certificates, and built-in support in pretty much anything, is this really true? Making your content available via port 443 really isn't impeding anything at all.
I think you’re conflating “tinkering” with simply desiring a different feature set. In my case I actually want “less” from Google in terms of number of features. I don’t want to tinker either, but we naturally reach for toggles as a way to tell the system we want different (not necessarily more) features.
Most of my search results at this point look like a spam ridden inbox from the mid-2000s.
I think you’ve touched on an interesting point here with respect to connection. Content naturally lives inside a walled garden and is often created there whether by users or professional studios. Users expect and dont care about this. However _connection_ is naturally between things and is a source of user friction when things dont connect well together.
Yep. Imagining nontechnical users would endure paths of greater resistance to avoid shady privacy and openness practices is magical thinking. Privacy and openness are important, but having tools designed to maximally reduce the overhead of solving your problems is more important.
I know I get on people's nerves on HN by harping on user needs and FOSS usability and interface design and such, but I think we as a group need to start taking user needs much more seriously. Open-source alternatives will always be alternatives until without taking usability and the overall experience seriously.
Many, if not most FOSS software developers choose some commercial tools— count how many MacOS and Windows laptops you see at OSCON or FOSDEM. Consider how different the cost/benefit analysis would be for people without the most basic requisite knowledge to reason about software problems, let alone troubleshoot, or throw in a PR to address them directly. Commercial companies don't get these seamless experiences by magic— it involves research, design, development and testing to deliberately remove friction and stumbling blocks rather than assuming your use case is universal, or that docs are a suitable replacement for fixing usability problems. There's nothing stopping any open source project from doing any of it. Volunteering isn't unique to coding— people volunteer to cook and build houses and clean up trash, too. But developers run FOSS and developers need to deliberately incorporate those other perspectives into their projects to get the benefit.
Imagine a commenter in a walled garden complaining about the walled garden's audience. "Facebook's audience is extremely skewed toward..., but that's not what...." How long before some Facebook reader asks, "Then why are you using it?"
If one dislikes systems that allow tinkering then why read and comment on HN. I do not understand.
Apparently "the market" dislikes systems that allow tinkering. Presumably "the market" refers to some people who said they dislike systems that allow tinkering. Either that or the commenter is inferring "the market" dislikes systems that allow tinkering even though no person has actually made a statement to that effect.
Maybe HN is not the ideal forum for discussions of "the market" (due to lack of interest and/or understanding), but maybe HN is a decent forum for discussing systems that allow tinkering.
This is absolutely true; however there are riches in niches.
The % of internet users who want X probably is lower, but I’d gander the absolute number of people who would want something like this is much bigger than it was a decade ago.
This community forgets that because we’re told by investors, the media we read, etc. to go after the biggest markets possible. “Hunt for elephants, not field mice,” they say.
Don’t forget that a niche market on the web can still be massive and small fortunes can be made building products and services for them. You might not even need to have investors on your cap table to bring products to market to sell to those niches if you know what you’re doing, but shhhh… don’t tell them I told you that.
There are no more websites worth linking to anymore... If you filter out all SEO spam there is barely few webpages left...
Google is desperately trying to hide that fact. Most of the web 1.0 can nowadays fit into small town telephone directory... You do not need mulitibilion dolar business to run web directory...
Try search.marginalia.nu (especially https://search.marginalia.nu/explore/random but try some searches for git commands or history too, just remember it is a search engine, not a conversation partner so only include words that should be in the article you search for) and come back to me afterwards.
I thought like you that if even Google couldn't find anything it was not there, but after discovering marginalia I now know it is just Google that has become unusably bad.
For day to day searching I now use Kagi and for me it is easily worth 10 or maybe 20 dollars a month since it "just works" unlike Google and has a larger index than Marginalia.
For now though Marginalia gets the money since Kagi is still in beta.
This is just retrieving articles when I search, not actual websites. Interesting if you're looking for article related to search keywords I guess, but genuinely unusable for actually finding something specific.
Gave it another try with "ssh scp". Google's first result explains me how to use scp (ssh provides a hint about the context), which was what i would be looking for. Marganilla... not so much it seems
The second hit links to a man page of scp. It is a formal description of the syntax, not what I was looking for... I'd rather have a short intro and a few examples of typical usage instead, am I being pedantic? If i wanted a formal description i would google 'scp cli', 'scp options' or even 'man scp'. Also, on a tangent, to be honest -- I find searching simply for 'scp' not a very clever approach. How did Marginalia guess it would be the cli tool and not one of the other acronyms?
Yeah that's not the sort of search engine this is. If you search for SCP, it will show you documents where that term is relevant, using domain ranking as a tiebreaker. It's quite intentionally not trying to read your mind.
I think it showed the man page first because that domain is highly ranked.
Just searched for "linux users" in marginalia and google. Google's first answer seemed spot on. (users command usage); Marginalia provided me with in comparison _marginal_ results.
Maybe it is because google knows me better then i am aware of. I really don't notice google results getting worse, while i read so several times in HN comments...
Yeah it's not a search engine for answering questions, but for finding documents. You'll get along with it better if you see it as something like grep for the web. This is something I'm very intentionally trying to accomplish, as it's something I feel Google has gotten worse at.
The best example of this is having a Google Home device and just being like, "Hey Google, how many ounces are in a cup?" as you're standing in the kitchen with your hands covered in flour.
Google still searches forums pretty decently - I think what you are describing are two separate phenomena
1. Yes Google search has gone to shit - even putting stuff in quotes now does not do an exact search (there is another checkbox you ALSO need to use for that). It tries to be too smart even when no user is logged in.
2. Giant mega forums like Reddit have really taken over. Instead of a dedicated forum people just go to subreddits. Personally I think it is good and bad and I still try to actively participate in both.
The "we answer your question" thing could be useful, but:
1. It should be in addition to (not instead of) the web search results.
2. The answers can't be wrong. Google often gives such a low quality answer that I no longer want to use it for this, besides asking about the weather and sunset/sunrise.
> I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
I think you have the cause and effect reversed. (Though I disagree with the meme that Google's results have gotten worse. For me, it's more useful than it's ever been. I often find the information I need without even needing to click on a result.)
A lot of search results pointing to independent forums are about people asking the same question I have only to be told to use Google to find the answer.
"Word for a bad scientist" gives me results as if I had searched for "Mad scientists fun stories", and it wasn't even offered as a correction. Google can't even accept that I'm typing the word I want anymore, I must surely want something else.
> this is exactly what I _don't_ want in the "Next Google".
You don't want that, I don't want that, and I am sure many people here on HN and similar gathering places for powerusers agree with us.
But "the next google" doesn't have to replace google for us, it would have to replace google for the average consumer (AC). And the AC likes the "simple box". The AC doesn't want to fiddle with customization options. The AC is used to walled gardens, advertising, "Apps" that are just wrappers around webpages.
So any "next google" will have to compete exactly at the "simple box" game to get the attention of the masses.
>I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
You are wrong; Open Web died or almost died because of walled gardens like Facebook, Instagram and Twitter. Billions of people who hang out on Facebook, Instagram and Twitter would hang out instead on Open Web e.g. websites, blogs, forums etc. if walled gardens didn't exist.
Google is anything but a simple box that gives you answers now. It hasn't been that for a very long time. I wish it was.
It is a box that gives you a screenfull of ads, some spam copycat sites I wish I could remove, and a lot of clutter. As many people on here have said before, it only gets away with it because it owns the web browser through Chrome, Android and its Apple deal for Safari.
>it only gets away with it because it owns the web browser through Chrome, Android and its Apple deal for Safari.
From someone who tried the switch to duckduckgo and uses ddg as the default engine: I can't remember the last query I typed without adding "!g". Google doesn't get my queries because the service is shoved down my throat, it gets them because the alternatives I tried are worse wrt the total scope of my queries.
Dude why do you have to fill these threads about all the spam on search with yet more spam promoting your crappy Bing-based search engine. Just let the conversation happen without all the self promotion ok? Jeez Louise this is annoying!
I looked for "Rewe" to see wether I get informations about local supermarkets, got a panel-view that is expensive to parse (eyes have to move in multiple directions, content is not clearly focussed), and was rewarded by getting no information about the local markets. Panel view only, forever, seriously, ONLY makes sense if you already know all the elements that will be displayed. Two giant rows of icons are terrible UX. Please make your default view easily-digestible. I enter a query, get a set of results. That set needs to be represented in a way that the brain actually wants to operate on. Nobody wants to operate on a set by digesting a table of unknown contents. The correct UX is a list.
I didn't bother checking customization-options, the ux was hostile and the results did not give me what I want. Pass, sorry.
Just tried to search "t-sne" was pleasantly surprised the first result [1] was what I found on DDG and a good result. Compared to Google which couldn't even find it on the first page. Most of Google's result are not exactly useful, and the only useful article is not my favorite in terms of quality. Order being.
1. Wiki article
2. Github repo
3. API documentation
4. Introduction with Python Code(Not my fav quality article)
5. Guide in R
6. Research article
I can't quantify what is better. But Google didn't give me what I wanted but DDG and you.com did. So congrats. My only issue is seeing Medium articles above the web results. While Medium might have the answer in this scenario the top web result was correct but de-prioritized which was incorrect from my prespective.
All you have to do to improve on Google at this point is to do less, make it less bad, i.e. a process of removals, not additions. Just do the same thing, but without all the shitty extra stuff. But then what's the business model? (Since that is in fact most of the shitty stuff. Oh sure there's still the SEO spam, and you're in that arms-race, like it or not, even if you're not a successful search engine, so you do the best you can with that.)
Speaking of doing less, I would love to see the web be more hierarchical or semantic (but not necessarily "the semantic web" as it's currently conceived). Google itself is what made the world reorganize itself. A world where that kind of search exists will reorganize itself around search, maybe not always for the better.
Concrete example of going from a hierarchical/semantic world to a search-based one: Instead of finding your socks in the sock area, which is inside your clothes area, which is inside your "do private things" area, let's say now every sock has a trackable chip in it similar to an AirTag and you just say "Alexa where's the nearest pair of socks?"
Pros:
No effort spent on putting your socks in the sock place. Just throw them anywhere.
Instant access to socks.
Cons:
Big Tech, with all its limitations and machinations, now mediates and controls the relationship between you and your socks.
You succumbed to the temptation to slack off, and now there are socks everywhere. The Roomba doesn't even work right.
Those 4 points can be solved by ad blockers or other browser extensions. The real problem is that the results below the ads are also ads, seo spam and clickbaity stuff.
>I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
People want to accomplish stuff, this means they need tools to do stuff and if a tool can be customized to do the stuff faster or better people want the customization. At my job we have paying users that requests features that indeed are work related(not moving shit around). I know GNOME-minded people will disagree and they prefer to bend their work to fit a guru-s vision.,
Now my turn to ask, why do people like you think there is a generic and basic solution that works at the same time for the casual user and for the user that has a lot of tasks to accomplish? Is there some theorem that shows this, like "The GNOME theoreme of product design, keep removing features until the shit convergence to the local minimum where you find the minimum product and the minimum set of users possible.
Because most users would rather not have to customize anything. It's funny that your example is gnome, as most users are not tinkerers and would never use Linux on the desktop unless it were made so simple that (again) they would not have to customize anything.
Obviously there are exceptions, and some tools are so advanced that it's necessary to be able to customize them. No argument from me there. But for most tools, and most users, there's just no hunger for customization. Almost nobody wants to have to manage an array of options to do a web search.
I agree, there are users that don't use customization for X but they use it for Y, so you get the idiotic philosophy that removes customization from X and Y.
So you get GNOME fanboys that love that 10 features they personally don't use are removed but when the ones they use is removed their brain finally realize that not all people use the exact same options, the exact same workflows etc.
What is even more shitty is when soem feature is removed like the System Tray and first they pretend they do not understand what you mean when you say this feature is very important, then after someone wastes his time to explain a n=th time again how working people use the System Tray features at work or at home the GNOME dude finally admits the issue and offers some workarounds that are not equivalent but are "possible to do but with lot more work".
On short, some people do work on the computer, some people use search engines for work to find relevant stuff, this people do not ask features like "please use the exact same padding everywhere because I am OCD" or "please make those buttons/corners or edges smoother so I don't cut my tongue when I lick my screen", this people want the customization to do a task.
I am not sure why simple people have a SEGFAULT if they randomly end up in the Advanced section of a settings section, what do they expect when they open Advanced? Google main page has a small link for Advanced search see
https://www.google.com/advanced_search?hl=ro&authuser=0 how many GNOME users got hurt by this link existing?
MacOS is the perfect example: it's both easy to use and the preferred choice by many professionals.
"Hold on", you say, "professionals want options, like user-expandable RAM etc". No, that's the misconception about the concept of a "professional". Unless you are a hardware engineer, tinkering with your notebook's internals is the absolute opposite of professionalism, its either a completely misguided waste of time and money, or a perfectly fine hobby.
Real professionals get work done. Customizing their workspace is something they feel ashamed to do, because it's procrastination at best.
What? Professionals customize their workspace to make it more efficient all the time.
Just that exactly is the workspace changes a lot, the hardware internals are hardly ever a consideration for any professional, it's easier to just buy something that works well from the start (and hardware costs peanuts).
The biggest problem with GNOME, and with Mozilla, and with almost everyone who's commenting on their choices, is that all of them are shuffling deck chairs around on the Titanic.
The Titanic didn't sink because of the arrangement of deck chairs, and Mozilla didn't sink because of any features they did or didn't provide (and GNOME didn't fail to achieve significant market share in the first place because of features either). The actual problem doesn't have anything to do with the stuff on the deck at all.
It's the one-two combination of vendor-lock in and bottomless marketing budgets. Since most of the value of the Windows platform and the Web is the immense amount of stuff that's built on top of it, there's a huge lock-in effect that prevents you from even reaching parity, much less exceeding it. And in order to overcome the marketing budget and pure inertia, you need to be ten times better, not just on par.
If GNOME becomes as usable as Windows, it won't have anything to do with what they actually do in the desktop environment itself one way or the other, whether it's continuing on the road they take now, or reverting everything back to the way GNOME 2 was, it's totally irrelevant. GNOME becoming usable will be entirely because of Valve investing in Wine, combined with a whole bunch of other apps moving to Electron and shipping Linux versions because heck why not?
Unfortunately, while they are probably already on par with Windows, they aren't ten times better than the Mac:
* The Mac has a bottomless marketing budget. Good luck competing with that, GNOME.
* They've shown a lot more restraint than Microsoft has, probably because macOS is considered a niche product to round out their catalog rather than being their one and only operating system like Windows is for Microsoft. They have even reversed course on a few anti-features, like adding back USB-A ports to the Macbook Pro even though it made the laptop slightly thicker. And unlike Windows RT, they didn't lock down the ARM Macs.
* Those tectonic shifts I mentioned that made Linux usable? They also make the Mac usable, because Wine is open source and Electron is basically its own operating system. Anything truly good that GNOME does, Apple can copy it just like Chrome copied all the really good stuff from Firefox.
The issue in open source is with projects with not a strong leader ship , then you get some wanna be designer copying Apple because they read some book and now he thinks that shit needs to look and feel different. Then you have developers that want to work on new cool stuff and not maintain existing code, so every few years you get a full reset but because of inexperience or incompetence the new version is buggy for a few years, it gets fixed but then the developers are bored and want to rewrite it using some new ideas/tech.
What would work IMO is someone with money paying the developers and the designer but force them do do customer support, you don't play with the new shit until most tickets are resolved and customers waiting for response are satisfied. maybe a paid support would help too.
Did you reply to the wrong comment? Because nothing you said has anything to do with any of what I wrote. It's not even a counterargument. It just reiterates the original point, which I don't entirely disagree with, but don't think has anything to do with GNOME's lack of market success.
Market success has almost nothing to do with product quality. Well, okay, it does, but only in the sense that you need to not actually be a total fraud. You can get away with dismal quality as long as your marketing is good [1]. In formal terms, software development is a loser's game [2].
This implies that the churn you're complaining about has nothing to do with market success. You might not like it, but that doesn't mean the failure of Linux on the desktop is actually caused by it.
There's a certain type of person who loves buttons and knobs. They're also the same sort of person who might decide to make a new search engine. But yeah, more knobs is definitely not something the market is asking for.
Most people don't want to tinker if they can avoid it, but many will come to appreciate the power of the advanced tab if and when they need it. These startups should take the "people are lazy" line of thinking to heart and make customization profiles easy to share, whether by direct link posted to Slack or a public customization "store" a la Chrome Web Store.
I agree with this, however it could be used as a platform for others to provide their own config and you can piggyback off the work of others.
It's kind of like ad-blockers where you don't have to maintain a list of domains to block. Others do that for you. And then people could create hosted version of their simple box with all the infra taken care of.
More likely, it exploits human's cognitive weaknesses successfully with a simple way.
It learns how how people get their dopamine dose. And there is no going back. You need more and more, more extreme content. More polarization. All you need to do is to open app and get that dose. Is it the same for search engine?
And people make more crazy stuff to get views. How this ends? Not well, probably.
I don’t think it is fair when we frame products as being objectively and consciously nefarious in this way. Conjuring images of executives rubbing their hands together, giddy with enjoyment that the war is leading to more exciting content.
These are firms that are meeting a legitimate need- and that’s the need to feel connected. Tik-tok provides that, and very effectively. Plenty of people get genuine enjoyment out of their product, and meaningful connections do happen thanks to Tik-tok
Maybe it is not fair, but that is what happens and eventually it is acknowledged by executives, which leads for new design decisions based on that on TikTok and other platforms to get more money and users.
I don't think they fulfill some gap of the need of feeling of connected in a real way. More like a bandage. We have seen the development of Facebook, Snapchat, Instagram. Their audience is fading on countries who have used them longer time, what went wrong? How is TikTok so good that they try to adapt it on their platforms as well? No way to connect?
Short video clips which might or might not lead for a real conversation. They might offer escape from reality in your lunch break at work.
I understand the perspective of "feeling connected". It brings people together with similar mindsets on entertaining way. Or at least people who seems to enjoy similar things.
On the contrary, is it different than some oldschool cults or religions? Cults which are using psychology writings as base for feeling mutual understanding of themselves. Or religions which share same ideologies and use it as a solution for their problems?
Technology is advancing, is TikTok a modern solution for finding your role and place in the world when it does not make sense and you feel you are alone with your thoughts? Maybe it is, maybe it then fills some gap.
I agree that TikTok is providing entertainment (well, that is what dopamine usually is). It is easier to hook people on short videos which are done by global audience versus Netflix where there is a limited amount of material and they cost a lot to make, when audience on TikTok is mostly making them free and you just pick suitable ones with your algorithms for showing the other audience.
However, there are many problems in this. How it can be abused and how it creates people living on their own bubble, like people on some extreme Facebook groups. When a narrative includes only content that boosts your own thoughts, a reality can be lost. We need some research on this matter, but for some reason social media companies are doing their best to prevent that.
Someone people also really get addicted on the entertainment and cannot stop using it. Well, same thing can apply also for alcohol, but is addiction risk closer to opiates for example?
I don't really believe that executives are thinking for the best of the people, so optimizing platforms to hook users is a quite dangerous play.
>You need more and more, more extreme content. More polarization.
This seems a little sensational. Many people (myself and most of my friends included) don't see _any_ "extreme" content or polarization. A quick scroll through my feed is largely nothing but magnet fishing, frog tracking, cats, DIY projects, and geologists talking about rocks. It's enjoyable and arguably a dopamine dose on-demand, but not necessarily a road down more "extreme content and polarization".
I agree that it is a little sensational, as it happens mostly when you like only limited groups of things. It gets harder for algorithms to polarize if you "mess up" with the algorithm and like many different kind of things.
To clarify, I don't mean with "extreme" necessarily a bad things, just content which gives you "extreme" emotions.
>It learns how how people get their dopamine dose. And there is no going back. You need more and more, more extreme content. More polarization.
What kind of TikToks are you watching? Tiktok sees I like watching Japanese videos that compare American and Japanese culture so it knows to show me more of that. Trying to say that showing me videos I like is them just giving me a done of dopamine is such a weird way to phrase it. Should they be instead just constantly show me videos they think I would dislike?
For Kagi, at least, there's a very well integrated search customization method that they didn't bother to show here. For any search result, you can add a ranking adjustment for the site it came from. This is directly in the results, so it's very accessible, and quite easy. One of the choices is 'pin', which is fantastic for technical work: 'sqlite.org' is now boosted over everything else, for me, and it's exactly what I want. I could just as easily take it out, if it becomes a problem.
I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end.
I tend to agree. It's an attempt to deal with bad design by putting in switches, options, and knobs to tweak. That's because 1) it's easier than focus groups, A/B testing, and taking video of users using the thing, 2) design takes some artistic talent, and 3) it lets the programmer blame the users for the problem.
This is a vice of open source people. It's why Linux on the desktop has never taken off. "Just edit /etc/conf/foo/bar/prefs.txt" is not a design. Nor is "On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS."
I'm critical of Google's search, but this is not the way to fix it.
Consider just few more search options, alongside "News", such as "Scholar" and "Noncommercial", for when you're overwhelmed by crap. And "Popular", for when you want the crap. Don't add another interacting dimension of tweaking.
At https://you.com we believe in choice but not force it. It will just work out of the box, but as folks shop or get really into something like coding - we have heard from many users that they like or dislike certain sources or apps.
Like w3school - it's in every search engine but some folks hate it so they can downvote it.
I personally benefitted a lot from the ability to like the reddit app once and then see more real reddit results?
There's a book I love to send people: https://www.amazon.com/dp/B001REFRZG/ref=dp-kindle-redirect (old school. no kindle. just hardcover. :) ) "Something Really New" How innovation works. First question in the book is: you need to come up with a new idea for a faucet company. Customer research says: Users want lots of variety in their faucets. Everyone then immediately comes up with the same exact ideas: faucets that are easily customizable. Faucets that have skins. Etc.
When really innovative stuff is just about removing steps. If a process has 10 steps, remove as many as you can, and now you have something truly innovative on your hands.
I feel like Google did exactly that. Pre Google steps: search for XYZ, wait, check first link, second link, spam, spam, check next page. Post Google: search for XYZ, get XYZ.
eh... except google is now basically just the pre-google product: Search for XYZ, wait, skip promoted link, skip second promoted link, third promoted link is actually the direct competitor to what I fucking searched, click result that was on top ten years ago, but is now almost below the fold.
Or worse - Search for exact term: get a page full of "Missing X - must include X" links hidden in tiny text below a result, click "Must include X" get the SAME FUCKING RESULTS again, click tools, click the dropdown, select verbatim, finally see decent results
Of course blog and SEO spam is such a problem on Google now that it looks more similar to your Pre Google steps.
I don't know how to easily fix that though. Simply "crawling the entire internet" is still not a simple problem, let alone doing something more useful with the result than google can. Ahrefs is an interesting business but not what people mean when they say the next google. "Machine learning" but I think google is all over this already (and has been for years).
Google does a bad job at getting user feedback about results while reddit does better so people search reddit, maybe a hybrid is an opportunity.
I've been using Kagi for a little while. The customisation isn't really necessary - the defaults work fine - but it's quite nice to have when you want it. I've banned Pinterest from the search results.
Google has these options, but these are solved by AI. So with that you come in a catch22: people want personalised content, but rather not have their data given away.
If you open YouTube without login, you get all kind of rubbish so you want to login. Probably this is by design: you want what google wants: results based on your data. Both happy.
Now if there was a privacy friendly way of doing this, I am all for using that. I just don’t see how, and I don’t see who wants not to gain a profit if you would have that data.
So the next google probably is another google.
If Google has these options, why can't I search for any image and have it not return any Pinterest results? If I search with -site:pinterest.com, I get Pinterest's million alternate tlds, if I just search for -pinterest, Google decides, in their infinite wisdom, that I didn't actually mean that and ignores it.
> Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else.
And that's great if it works! The problem is that once it fails (and, at least in my use-case, it does so quite often), working with it becomes an absolute pain.
I, too, would prefer an omniscient box perfectly answering my questions. But it clearly doesn't exist. And a box with screws to adjust so that I can eventually find what I'm looking for is the second best thing.
It never fails to amaze me how many people, apparently triggered by these omens, come out and say this as if there was never such a thing as a default configuration. "But you MUST configure!" Um, no.
What counts as customization? I might agree that most people don't care to change font sizes or colors or even themes but they might want to be able to tell Google to never return results from a particular site. Is hiding results from a site customization? Is it the sort of customization that would overwhelm a user if they saw it as an option?
> I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end.
It doesn't need much--a single button with "Remove this site from future results".
That would solve 99% of the problem because it would destroy SEO optimization since whole swathes of people would have different criteria due to the blocked websites. It would also tell Google which websites suck if they didn't already know.
Alas, it would also destroy their ad revenue because once a site got overly scummy people would start delisting it.
I would go one step further and question the premises that a) the next google must necessarily be dominant/monopolistic and b) that said dominance must necessarily ride on top of free user economics.
What TikTok shows to me is that users want specific things when they go to a site: for TikTok, they want fresh entertainment, plain and simple. I go to HN when I want tech discussion. I go to reddit when I want aggregation of niche topics. I go to costco.com when I'm looking to shop.
Google is frankly a horrible experience for ecommerce, it simply cannot compete w/ the likes of Amazon or any retail store website, really. Being a search portal, it's fundamentally incompatible with the concept of evolving through permanence of a hivemind; every new search is like reseting a would-be community to zero all over again, so you cannot gradually build up a collective commons on Google like you can on HN. Youtube has recently gotten pretty bad with cycling fresh content on the front page. For stack overflow sort of stuff, any other search engine does more or less the same. For trivia, Siri/Alexa are fine substitutes. The Google search properties have become mediocre on average; they're not particularly great at any particular thing.
Are many of these domains walled gardens? Yes, think of things like Doordash/Uber Eats grocery catalogs, a ton of brick and mortar shops are choosing to integrate with these delivery apps, and these catalogs are completely invisible to Google. News SERPs on Google are often garbage since they just link to paywalled content half the time, might as well just get a subscription from the actual news outlets. Etc.
IMHO, the next Google is already here, and it's everybody else wisening up to the simple business fact that they need to own the top of their funnels.
Spot on. Back in the days of web portals, millions was spent on making them customizable. I can't find a link, but I believe back in the early 2000s reading that less than 5% of people actually customized their portal.
Couldn't agree more. For the average person outside of our bubble, google "just works". They don't give a fuck about customization or the occasional ads. There are still plenty of people who click on the sponsored links without even knowing, or caring, that they are ads.
There should be a "make me lucky" button - to tell you what you ought to be searching for, since it's got enough data to know you better than you know yourself :)
I agree; choices and tinkering add complexity to a search engine, and search engines need to be simple.
That aside, a search engine needs to be really focused on privacy. Customising it implies that the user needs to log in and remain logged in. I'd much rather no do that with a search engine, and I don't want its behaviour to react to my identity either.
> I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
It's often said that people don't want more choices, they want to be confident in the choices they make. It's the strength in simple product lineups like good, better, best. Googles lone search box is good example. Apple's product line also tends to be a good example at times.
i think more customization can help you get power users to spread the word, as long as it doesn't get in the way of someone who just wants to do/read/understand as little as possible to get results
>I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
I think it really depends on the product.
If it's something like Photoshop or Vim which are used by professionals for multiple hours a day and productivity is to be maximised, customisation is not just a nice-to-have but a necessity. Power users appreciate it too.
You're absolutely right in 90% of the cases, though I'm not sure which box web search would fall into.
I still love one of the design stories of Steve Jobs in Isaacson's biography. I don't remember the exact phrasing, but essentially someone at Apple was showing Jobs the new DVD burner software the company wanted to distribute. It likely had a ton of options to it (like other dvd writer software of the time). Jobs' response was roughly "Hmm, ok. Here's what we're going to do. There's going to be a box, and you drag the files you want on the dvd into the box. Then we're going to have a button that says "Burn DVD" and when you press it, it burns the DVD"
I haven't used Google's search for a long time. The sticky ones for me now are gmail and YouTube. gmail because it is a lock-in, once you have the email address, I don't think you can ever change. Since I am a Chinese now I have to think about if Google stops serving me in the future, but I am deeply invested in my gmail account :( The other one is YouTube, where it can work without login.
Google's simple box is not attractive when others can provide the same quality in search results.
>I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end. I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
I agree, BUT I would like to share my experience moving from Google to Kagi. Google serves me primarily ads, whether it labels them ads or pretends the results aren't ads, they're mostly ads. I see the same horrible domains pop up frequently, and there's nothing I can do about it. Kagi gives me the ability to remove these domains from my search, and it is INCREDIBLE. How many times has Pinterest barged into your search results?
Now, one could argue that these are domains which no one wants in their search feed. So perhaps this could be solved by being less beholden to shareholders and an advertising model. However, what about domains which most people want, but I don't? For example, I was finding increasingly polarising political content coming from MSNBC. I didn't like their "human" angle on news, and I didn't appreciate their extremely polarising commentary. So I want to remove all MSNBC results from my search feed. I understand I am probably in the minority. Perhaps my political views don't align with the typical customer. Could a well designed search provider anticipate this need of mine without giving me some kind of method to tell the provider that I don't want to see MSNBC? I'm not convinced so.
So I think the answer here is both. Design a search engine well, and give users the ability to customise it.
This! This is why I love Apple products too. It abstracts away many of the things that 99% of the people (general public, not HN community) don't need or use.
It's better to focus on the core basics and perfecting it than to add many features and try to support and maintain them especially if they are features that only 1% of the users use.
Right. Their premise is “There’s no average human”. IMHO, it is ridiculous.
The majority of people are just average every day normal ones, and they don't give a flying fuck about customizations.
This is not a binary "the people who don't want" / "the people who do".
The old Google was able to do an exact search using Boolean expressions. Man, I miss that all the time....
It's not rocket science to provide a user interface for this. A combination of GUI configuration, keywords, and specific URLs for these cases would go a long way here without bothering the regular user in the slightest. Yes, it's probably slower than in earlier times, pure search volumina and its handling by AI weighting etc. considered. But there is also a "don't want" on the part of Google and the other players.
Exactly this. Scrolling through the customisation feature list, I just see "more work", "more work", "more work" - and it's not super clear how doing all this extra work is going to help me. And I'm in the target audience for a "technical" search engine; this is a total non-starter for the 99.999% of the world we call "normal people".
The useful ideas in here is the "just my data" search, linking into external providers across siloes. Of course, Microsoft, Google & Apple already have this as long as all of your data is held in their ecosystem.
> We need new thinking to create something much better than what came before. In the last few years, different groups of people came to the same conclusion, and started working on the next generation of [foo]. For this new generation, [bar] is necessary, and [baz] are not an option. But that’s where the commonalities end. Beyond that, they’ve all [zoz] in very different directions.
This pitch voice makes me want to gouge my fucking eyes out.
560 comments
[ 13.3 ms ] story [ 307 ms ] thread"The next Google can’t just be an input box that spits out links. We need new thinking to create something much better than what came before."
Could not agree more.
Great roundup of some exciting search alternatives. Thanks for sharing.
I think one strong contributing factor to Google's success is its simplicity. All the listed competitors add a lot of complexity imho. While all the customization buttons, knobs, lenses, meta crawling, code generation features etc might add some value for the advanced and technically skilled user, it provides rather little value for the average user who just wants to look up a cooking recipe.
So maybe when searching for "the next Google" the interesting question is not "what search features can be added", but "how can search even be simpler than using Google".
Google also has a few good things going for them:
- it’s the standard when it comes to search and is setup by default on just about every platform
- People use the brand name itself when talking about looking stuff up (no one says “just duck it”)
- They control the ad market, the standards for what a “optimized” site is, and the web standards themselves.
I work for Neeva, and this is a big part of why I left Google to join Neeva. There has to be a better experience, and it doesn't start from another business that works just like Google. It has to be a different kind of business. Neeva does not make money from showing you ads, so it can provide a different search experience... a simpler search experience, like the original google even, but it can go further...
With the Neeva app for example as you start typing in the URL bar, it will take your input as search suggestions (just as any other browser + search engine would) but instead of just showing you completed search suggestion, Neeva will show you the results from running those searches inline. The idea being that maybe those results will be helpful to you and make it so you don't even need to go to the search results page. You can just take the result right there from the URL bar suggestions drop down. Saves you time. Simpler.
Stuff like that. There's a swim lane of innovation and ideas on how searching and browsing can be better that is just really hard for Google to build, even though many of these ideas are thought of inside the walls of Google. They just can't ship them if they are stuck being beholden to their search ads model.
Another great example... ever wonder why Google isn't working to make it so Chrome doesn't have a million tabs at the top of your browser? It gets to the point where it is hard to get back to what you were doing. Me, I just end up closing the tabs, declaring tab bankruptcy. Google is okay with that because it means I have to search again. The Chrome team wants to fix this but it is hard to do so as it would result in people searching less often!
Again, just means there is opportunity for a simpler better experience to be had and Google won't be the ones creating it.
The "I'm feeling lucky" button would never be added to Google if it didn't already exist. It was grandfathered in.
https://elicit.org/
I feel a lot of modern software is becoming very authoritarian. “We know better than you what you need”.
You can definitely find these on Google.
An input box that spits out links is _exactly_ what I want.
I think achieving this would be worth billions, at least?
We really tend to underestimate the amount of spam on the internet.
Google does badly here: it overcorrects and you get only super safe results. Yandex? Unless you get lucky, lots of spam.
You'll still get spam if that's all that matches your query, but now all it takes is for someone to make a page matching the query without the aforementioned items to outrank the spam results. You wouldn't need to append "site:reddit.com" to your queries because the (mostly) non-commercial Reddit results would automatically outrank all the blogspam and listicles.
If ads were downranked it would make a lot of spam/clickbait/listicles unprofitable overnight as they'd rank low enough that the costs of creating & maintaining the spam site/content would outgrow the returns from ad impressions.
This would be a really interesting experiment: A search engine that ranks websites by the amount of Ads and other spam that they contain.
I'd pay /mo for a good one. Why not? Currently trying Kagi.
SEO is killing Google and it needs to be addressed before it kills the internet. Having the opportunity to build an app that solves a search, instead of perpetuating a spam system, has been really rewarding... and we're just getting started.
I've been trying out Kagi as my default search engine, but will give You a try next.
When I'm researching stuff, I go thru a whole page of google result, and 'open in new tab'...90% of the time it's crap, and I really wish I could signal google 'this was useless'.
I currently recommend Kagi.com, it can block but also pin selected domains at the top of the results list.
In fact often you don't even want the information, you just want to solve a problem. I don't know about you, but I don't like learning all about air conditioners and spending time finding the product available in my area with the highest quality/price ratio that fits into my budget. I just want the best air conditioner for me.
Providing links is the only thing that scales, since you can delegate the aggregation work to specialists and you van verify the source.
I don't mean that in a derogatory sense, I just strongly believe that in 20 years we'll look back at our Googling years and wonder how we managed to find anything at all. Who know what the next pattern of information retrieval will be, but I personally think it'll be an even bigger jump than the jump from reference desks and encyclopedias to Google.
Edit: okay, maybe not 20 years. Decades, perhaps.
https://horses.extension.org/what-is-the-carbon-footprint-of...
> The digestive process of horses produces far less methane than the digestive system of cattle and sheep. (...)
Also the Ford company’s dogged resistance to listening to customer wants (“Any colour you want so long as it is black”) hurt Ford as soon as they had to compete on anything but price, with GM finding success with “a car for every purse and purpose”. From 1921 to 1927 Ford went from controlling 65% of the American car market to just 15%.
All that to say: “innovator vision that flies contrary to common sentiment about what people want” is often overrated and can be a trap. That’s not to say that the customer is always right… but they’re not always wrong either.
This sounds like it was written by the infamous Dropbox commenter.
No, most people have never given a single thought to this.
https://gs.statcounter.com/os-market-share/mobile/united-sta...
Android is 42% of the US market.
Doesn't seem so bad as a market.
It might plenty to create a nice business, but a bit absurd to call it the next Google.
Could be Wikipedia-style cultivated if enough people know and care.
Probably because they're stuck in a local maximum with no idea how much better it can be. They may not even be aware that there are sites out there that have quality non-commercial content if all they've been given for the last decade is spam to the point where spam has been normalized.
https://images.app.goo.gl/G1K7J8gZ135x98mz7
- who pays for the service (ads? users pay? Average user will never use a paid service if a free one is available)
- how to resist attacks against the algorithm (Google has been fighting spam for decades)
- how to personalize without invading privacy, e.g. Google had an option to search through your email in Google search...it's gone now, I wonder why?
Whatever replaces Google will be doing something that Google can't without causing them other problems. The first thing that comes to mind is make them choose traffic vs. advertisers (I don't know, if I had an idea of how to, I would not be writing this), but they're big enough that other wedges could start chipping away at their margins.
The article is titled "The Next Google." I was responding to that, not "A Profitable Also-Ran".
I've been using it for a couple weeks now on my work laptop and for programming-related searches its been great so far. And the usual annoyances that show up at the top of google and DDG don't show up on Kagi (geeks4geeks, etc).
Pull out phone -> Safari -> type "Walmart" -> click on the map -> Maps app opens and starts guiding me
When I switched to Kagi, the flow went like
Pull out phone -> Safari -> type "Walmart" -> top result is Walmart.com... no address to nearest Walmart to be found -> Close Safari -> Open Maps app -> search for Walmart
And it got so annoying to have those extra steps. I know I can change my workflow and get used to it. But it wasn't just directions. It was other basic searches. Like if I needed a phone number for a local business. It drove me crazy. I really hope Kagi gets better at those sort of things. I want them to succeed. But it was just too much friction for me.
or https://you.com/search?q=pyspark%20filter%20array%20element
Since disabling it this way removes the `sendBeacon` function from existence, you should just be prepared to properly handle its absence.
I know little about front end JS dev, but the docs say it’s just a way to send an asynchronous request (no response) with a guarantee that the the request will be sent out, even if the page starts unloading. That’s apparently not a guarantee with the two other to send requests - XmlHttpRequest and fetch api.
I believe products can work that way, offer premium service and features for the very few that need it and a basic service to anyone else. In the end, the free tier is cheaper than what you'd have to spend on marketing otherwise.
Chances are there will be both.
To me the fact that Kagi is not currently VC funded is huge for me as far as adoption. Every customer facing VC funded startup I've worked at inevitably starts to institute increasingly anti-user practices while grinning and talking about "customer first!"
I know it's a huge ask, but if Kagi remains privately funded/non-VC I'll happily pay the moment I can.
I've only been using the service for a short while now but have been enjoying it a lot. The ability to blacklist domains has already dramatically improved my search.
I'm afraid what happens to 1Password since they get big funding recently.
Overall it seems very rare anyone ever considers this an engineering problem. Really, what's stopping you from running a search engine?
But if you must know:
First you need to collect a lot of content from the internet. From many different sites. With very different types of code structure. Broken html. More often than not behind some SPA JS code. Behind robots.txt files and bot protection efforts.
So the first problem to solve would be building a crawler at scale. That is able to crawl anything your users might want to visit but don't know of yet.
Then storage and retrieval. You need to store and update all this content your crawler collected. You need to enrich it with meta data and organize it for efficient retrieval. So that you can surface it to your users when they use your search engine. Indexing, structure, build g connections between content pieces. A lot of interesting things to think about.
Then there is the front end. Make it easy to search, to refine. Surface relevant content for search queries.
OH maybe I forgot, but you probably need to do a bit of engineering to make your system understand the users' search intent.
This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Bigger than that. I would applaud you if done with orders of magnitude lower than Google. Anyone would.
There are some problems that aren't as big as they seem. Parts of an SPA can't be reliably linked to anyway even if you find interesting text there, so you can just leave them out of the index.
Likewise, there isn't as great of a need to keep a fresh index as it may seem. The odds of a document changing is proportional to how frequently it changes. This is a bit of a paradox, where even if you crawl really aggressively, the most frequently changing documents will still always be out of date. Most documents are relatively stable over time. You can actually use how often you see changes to a document or website to modulate how often you crawl it.
The bad HTML is quite manageable. You really just need to flatten the document to get at the visible text. Even with really broken formatting, that's manageable.
The storage demands are also not as bad as you might think (most documents are tiny, sub 10 Kb), there are ways to lessen the blow on top of that. Both text and indexes can compress extremely well. Since you're paying for disk access by the block, you might as well cram more stuff into a block.
Most of the crawling concerns, in general, can be gotten around by starting off with Common Crawl (even if I do my own crawling, which also is finnicky but manageable).
> This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Right, so shouldn't the question be how to find the documents that are even candidates for being search results? Most documents are not ever going to be relevant to any query ever. Get rid of that noise and your hardware goes a lot longer.
I'm running a search engine on consumer hardware out of my living room that can index 100 million documents. Go a bit higher budget than a consumer PC, and you've got 5 billion. That goes a long way.
What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
> I'm running a search engine on consumer hardware out of my living room that can index 100 million documents.
That's extremely cool. I would love to know more. To me an impressive feat already.
> What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
Now this is a proper difficult problem with (probably) fairly subjective answers. I do however think it's something that warrants serious investigation. It's probably a decent candidate for a machine learning model combined with some manual tweaking for sites similar to wikipedia or github that have absurd amounts of parallel historical content.
Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
> That's extremely cool. I would love to know more. To me an impressive feat already.
Yeah it's at <https://search.marginalia.nu/>. I've built all the software myself from scratch in Java[1], and I'm doing my own crawling and indexing. The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
I do use a MariaDB database for some metadata, but I think it will have to go as its hardware demands is becoming a serious bottleneck.
[1] Despite using Java, I should say regarding the index. This is approaches sunk cost at this point. Building a search engine index is not something Java is at all suitable for, its limited low-level I/O capabilities is incredibly handicapping.
Worth pointing out that Lucene/Solr, the biggest open source player, is also Java!
* You can only allocate on-heap arrays of 2 billion items.
* On-heap arrays have a massive size overhead in terms of GC book-keeping.
* You can only allocate off-heap memory map 2 Gb at a time.
* This also goes for memory mapped areas.
* You have no control over the lifecycle of mapped memory and off-heap memory. They get cleared if and when the GC feels like it.
* You have no madvise capabilities
* The language barely acknowledges unsigned types
No you didn't. All good. And I learned a lot from the extended answer. So I am thankful for the explanation.
> Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
I can totally understand the feeling. There are quite a few things that I'd like to go deeper into either at work or in private. But alas time.
> Now this is a proper difficult problem with (probably) fairly subjective answers.
I agree. And I don't have answers ready. A lot boils down to preference. Personally, for example I prefer written content over video. Except in a few areas were I like (some) explanatory videos. To me it comes down to the question of how easy I can skim the content when I am looking for an answer.
On the other hand - for deep immersion into a topic I use multiple media formats.
In terms of web search I sadly nowadays need to sift through a lot of seo-fied content that is there either to build a (personal) brand or to attract clicks for advertising revenue/affiliate revenue.
So in principle I agree with you on the noise problem. Still I also believe that there are real great gems to be found in the long tail. When I still feel like I came late to the party, but when I started out in the web in '97 there were so many lovely, quirky sites. So many places that people had put a lot of time, energy and thought into. And sites so packed full of information that I came away not only with more knowledge, but in awe that somebody would give this knowledge away for free.
There also were quite a number of horrible sites (my first ones probably included). So there was a noise vs. signal problem back then. Maybe not to the extent today, though.
> The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
Call me impressed. Sounds absolutely cool.
So even with a raid setup for redundancy this is doable.
May I ask how you decide to add me content? Do you follow links? Do you use other search engines' results as a starting point?
I could probably shoot many more questions, but don't want to be a nuisance.
Thanks for your time already.
I initially did basically a DFS-walk originating at a few websites I liked, with some filtering criteria that deprioritized websites that didn't look too interesting. Now that I have a fairly comprehensive mapping of the space I want to index, I use a few factors like frequent outbound links from highly ranking domains to inform which new sites to index.
> I could probably shoot many more questions, but don't want to be a nuisance.
No worry, I love to talk about this stuff.
And I learned already a lot in this thread after the explanations unfolded.
The initial statement sounded exactly like the armchair "experts" one so often encounters. Actually this was for a long time the first time that there is a person with substantial experience in the problem space behind such a statement.
Maybe - but [marginalia_nu](https://news.ycombinator.com/user?id=marginalia_nu) isn't an armchair expert - they've actually implemented theor own publically available search engine - which is linked in their profile.
Should be said I do wear through this SSD at a pretty alarming rate. I'm at 193 TBW on this disk since I started using it as an index less than a year ago.
I do have a bunch of mechanical drives I use for archiving and as intermediate working areas as well, but the index itself is on an SSD.
The solution for o both of these might actually be a paid service. If you have a paid service, there is a possibility of it being profitable with much fewer users. As an example, let’s say you have 1,000,000 users at $10/month, that is a $10,000,000/month which might be enough to run the service and provide a comfortable profit.
With regards to the spam issue, the fact that you have a small user base would be to your advantage. Because there are so many Google users, it is in websites’ economic interests to spend money to try to game the algorithms. With much fewer users, your paid search users may not be worth it for the sites to spend money trying to game your algorithms.
From what I'm seeing, if you could create a bot free eco system, people will pay for it.
The question is "can you make it bot free". This is gonna be the next trillion dollar company.
At the moment, spamming Google seems to be trivial with no long-term penalties if you get caught doing something nasty.
A simple rule (manually enforced on a case-by-case basis) that would ban your brand/domain for a year if you get caught breaking the rules would get Pinterest into compliance from day 1 for example.
Using ads/analytics/affiliate links as a negative ranking signal would make a lot of blogspam/listicles/clickbait disappear if their only funding method immediately makes them rank much lower below where they are no longer profitable.
This isn’t a hypothetical example - Google actually includes in their webmaster tools a “disavow links” capability so sites can avoid getting punished for bad actors trying to make them look bad. But you can imagine if the penalties were even more severe other folks may get caught up in an unforgiving dragnet with no judge or jury and no way to appeal.
My main point is that people will find ways to game the system, and usually sharp edges (“harsh punishments”) on any system will be taken advantage of by actors, and unfairly penalize others.
Not to mention, if free search engines keep devolving into an endless sea of spam, people may have no choice but to start paying. There's plenty of things out there people pay for not necessarily by choice but because there's nothing else out there that would accomplish the task at hand.
If I can, I'm happy to pay, so others doesn't need to. I don't understand why I'm in minority and most of the people thinking only about themselves.
I'm happy to pay because for many years I also had to use free services, paid by others (or ads, but ads I'm blocking however I can).
That said, I do like the feature of ignoring entire domains. Google used to allow that
Anyway, Google is getting less relevant because the technology is getting better than "good enough", and any additional tech that Google adds is not really all that useful. It's like PCs. You don't need a faster one because your old one can do word processing just fine.
If I search for "what's the weather now" I probably want to get an answer "like an oracle" as you say. But if I search for "riaa vs napster" I don't want an "oracle answer", I want a list of several websites so that I can learn more about the case. I'd like a search result for court documents, but also commentary from news websites and blogs, and maybe wikipedia. I want to open those tabs and come out the other way with more information, but there was no "oracle answer" to be provided.
I disagree. I use Bing for about half of my searching because Google simply struggles with things like exact phrase matching in many cases. Where Google falls off, Bing tends to succeed. Though Bing also sometimes lags behind, where Google tends to do better, hence my split usage.
Glad to disagree, DuckDuckGo is good enough if not great.
For anything that isn’t the least bit obscure or technical. (Same with just about every search engine)
IIRC, someone on hn said that search engines are only good for things you already know are there, not for finding new information or different perspectives.
And I use it in two languages. Not bad.
For other results I find myself reaching to a bookmarked searx instance. I thought it just agreggates results but somehow it gives me different stuff.
And you also get magnet links :)
It's sad what today's web has become. Each year you see it degrade and there's no real fix I'm afraid.
- does my current location matter for the search?
- am I trying to spend money (shopping), or trying to get information (facts)?
Nobody uses yahoo search today...
I will never ever ever ever spend time going into the settings of a search engine to customize my search experience. Not when Google does a good enough job for most of my tasks.
and to make matters worse, 95% or more of the web isn't even being shown to users and so I keep getting the same stupid search results: which is fine for many cases but not all cases. sometimes I need more than just another post by NBC.
I will say, google search is very good at technical searches, that's for sure. But, for example, they totally suck if you're looking information or doing research on financial matters or economics.
Unless your technical search, for example, contains jargon with a similar spelling to or other sense synonymous with a common term that Google will helpfully also consider to match your query. I think it's actually rather mediocre for technical searches now that they've gone so deep into stemming and such and lessened the ability to use operators to refine queries. And, of course, the lack of support for symbols makes many technical queries extremely difficult.
Nobody seems to have The Next Big Idea for a better search engine yet.
I think the bigger reason why search isn’t getting much better is that incremental technical improvements are being offset by Google trading their dominance for greater shareholder value. A safe business can take away or buy-and-bury features that are better for the customer but worse for the business’s top line. Things like the Power Search API, or search that integrates with other platforms that hurt Alphabet owned businesses.
Traditionally big innovation breaks some socio-political constraint on market participation. I don’t see how AI will do that yet.
I don't know anything about AI or the relationship between market participation and innovation. All I know is that LLMs are a useful tool that is already displacing search in many parts of my life.
And that's by the way why Google is still successful as well. Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else. The only way to beat that is to make it even better while not making it more complicated which is very hard to do.
This is the thing. I'm the power user in my circle. People comes to me for suggestions about pretty much anything involving tech. Sometimes just because they see me with different stuff.
So if I'm not the average user I may look for other options, as other people like me may do. If I find such options reasonable for the average user, that will be my recommendation to them.
Google is very good at this, but this is exactly what I _don't_ want in the "Next Google". I want a _search engine_ for the web, not an answers engine that tries to know what I want better than I do.
Search the web. Give me links to websites. This seems obvious to me, but everyone is trying to be like Google.
I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
The audience of HN is extremely skewed towards preferring systems that allow tinkering but that's not what the market wants.
Some walled gardens, like facebook, are decaying. Others, like tiktok, are vibrant, still fresh and new.
Walled gardens are a natural result of the pursuit of capital. If you burn VC money, but create no moat (it turns out the walls of a walled garden are also a moat, weird huh), then as soon as you introduce a clever algorithm that introduces 50% more ads to extract profit, your users will all leave.
As such, a business is incentivized to build these walls and moats.
How do we avoid this?
Well, we do have examples. Mastodon and other open source projects eschew walled gardens in favor of free software ideals. Web3 embraces a certain "decentralized" vibe which lends itself in this direction. Universities, and other public-ish institutes like DARPA, created the original internet and many of its technologies.
Unfortunately, open source projects will struggle to advertise or find users. They do not have the initial capital to get as much momentum as the VC-funded alternatives. Web3 seems surely doomed to end up also building walled gardens for the crypto-anarcho-vibe is only skin-deep, and many a regular business is now highly involved.
This leaves government entities. The government is the one group that is both well funded enough, and has motive to create protocols which prioritize the user's freedom over profit (after all, the users will pay taxes regardless of how high the walls are).
It seems to be out of fashion these days for the government to actually do anything though, so perhaps there is no more chance of that than of a free software project managing the same.
That's also why "Google's search results are soo bad." They're not. For the bulk of Google's visitors, they're good enough.
Anyway - you’re right, nothing at all wrong with wanting choice. I think the point being made here though is that “layperson gravity” / mass market appeals / lowest common denominator is going to mean that tuneable web search will be a niche product, forever. Even if we’d both like that niche.
"Elitist" feels pejorative. Also a not-inaccurate way to describe a system which presents high barriers to entry.
Not trying to be awkward, but even back in the early days of Netscape, it wasn't like you had to recall any arcane commands to use most search engines.
Sure, they all had limitations then, such as not necessarily knowing similar word senses to create more nuanced searches, but that was a fairly level playing field.
Is it simply that it was initially a bit obscure and not everyone had found out about it? That's not really a barrier in terms of difficulty - as soon as people got into the "in crowd" they could use it just like the rest of them.
When netscape came out if you were lucky to have ppp access setting it up on windows 3.1 was difficult but once you were up as long as no one picked up the phone you were fine.
But even this was too difficult for the average person who lived in an aol world where the internet was limited to aol.
Why is publishing documents so complex now?
my site is all html, why not.
Cost was one. Hardware wasn't cheap and it would seem to have no practical use except to satisfy one's curiosity. A PC for the kids cost about half of our family's monthly gross income. A modem would be 10%. And the phone bill caused tourette's like symptoms more than once.
AFAICT, quality personalized search results isn't the goal for Google here.
I think the point of all that is generating revenue via advertising sales.
And while providing high quality search results might once upon a time have been a goal, both as a goal in itself and a tool to drive user adoption/engagement, that's no longer necessary as they have a (relatively) captive audience and a (relatively) captive customer base (advertisers). As such, quality search results are no longer all that important.
I'm not a Googler, IMHO, YMMV, etc.
Now they have millions of cell phones collecting data and the GPS location of everyone offline, they run extremely popular DNS servers to see what websites people go to, most websites (including educational and government websites) include google's trackers so on most websites every page loaded will ping at least one of google servers. They've got people uploading their personal and work documents to their cloud. They are swimming in data collected from sources outside of web search. Between that and the lack of actual competition it's no surprise they aren't investing in making searching the internet better for people. In some ways they profit from search results being terrible. If anything beyond the first few results on the first page is filled with spam and irrelevant websites the top few results to any search become even more valuable.
That depends on whether or not you agree with Google's definition of "better". What is better for Google is a set of results that make you most likely to click on an advert. Failing that, they want you to click a link, see the site, and either click an ad there or quickly return to Google where you'll click on an ad. The worst outcome for Google is that you'll click a non-paid search result, like it, and stay on that website.
This is the fundamental problem with search engines - if they work well and give you exactly what you're searching for first time then they won't make any money. A lot of what Google does is subtly trying to give you results that look great but really aren't.
But to be honest the internet is still the internet. The web still exists. Any lamentation at the loss of the "old" internet is that you don't have more angry uncles spewing political rhetoric on your motorcycle forum.
Do you want the unwashed masses in your specialist forums? I certainly don't. Seems to me things are working pretty well. I do worry about the next generation but to be honest, the same was said about me.
Instead of complaining about the state of the web just be the change. Host your site, run your forums, live your life. Stop worrying about how other people should live their lives. Take some youngsters under your wing and show them how a keyboard works instead of a touch screen.
This x1000.
The only way to ensure that future people can continue to maintain things is to teach those who are interested.
While I think we need to scale back the opaqueness of tech to the average user[0], there will always be some who just know more.
[0] By which I don't mean "more configuration options", because the average user doesn't want those.
I mean make it easy to access the source - and just as importantly make it easy to use changes to said source, so those who think, hm, I want this, can learn to fix it themselves, or find someone who can.
Ref: https://en.wikipedia.org/wiki/Eternal_September
Having the choice between "unwashed masses" and "curated to suit political agenda" I would firmly choose the former.
Before I get attacked on political agenda angle, please understand that it takes trained professional actively trying to remain objective to have impartial moderation. Many professional discussion moderators fail here and you just cannot expect neither unpaid volunteers (doing things out of passion which is not impartial in the first place) nor paid employees enforcing company policy (again very much partial) to remain objective.
The goal of moderation should be to allow for multiple angles of thought to emerge and not get drowned. Give me the tools to see those non-mainstream positions.
The unwashed masses are often filthy and malicious (on the internet).
Moderation, of the governance and oversight meaning, is unforunately very necessary to keeping any pool from becoming a cesspit. By some combination of technical, volunteer, and participant effort (membership fee, or service requirement), areas can be kept decent.
Without this, decent and would-be participants are run off by spam, scams, trolls and harrassment, political rants, religious rants, etc.
Admninistration web sites are now forcing you to use google(blink/geeko) or apple(webkit) based browsers. Asking nicely to restore interoperability with noscript/basic (x)html browsers does not seem to work, namely keeping the door open to alternatives requiring a reasonable effort of development, and not the army of devs of big tech.
This requires regulation I guess.
I'm not sure that's true. Just to pick an example, basically any type of product search now leads to auto-generated spam/borderline-spam websites that scrape reviews from Amazon, which are themselves often fraudulent.
I guess you could say that's "good enough" but only in the sense that someone being scammed at some point consents to something that makes them a victim of the scam.
For many searches, you'll see "reddit" autocompleted to the end of the search because Google's organic results are not good enough and people are trying to weed out the spam somehow, albeit in an imperfect way that restricts results to one site.
I guess you could call that good enough.. I call it morally irresponsible.
I can't lab it up now, but that is the gist. And DDG has it while Google rarely has the same level of it.
They could start penalizing sites that show different things to googlebot and a real browser again.
The could do a better job penalizing sites that just copy content from other sites.
They could penalize ad-laden sites.
They could stop prioritizing youtube results over text content.
They could allow you to blacklist sites again.
They could do so many things.
None of this is curation, put priorization of search result is exactly what a search engine does.
Have you read any of Google's advice to site owners? It amounts to "provide your users with what they're looking for, and do it with a fast and easy to use site". If you think SEO is an easy hack to get any site to the position 1 for any search, you're just ignorant of the landscape. All the "SEO" crap you see about keyword density or whatever, is just an industry trying to sound different and like they've got the key to position 1.
> They could start penalizing sites that show different things to googlebot and a real browser again.
Google is ridiculously good at this. Would be keen to see examples of a site that does this in a meaningfully negative way that ranks well for searches that matter.
I don't fully agree with all of your other points but I can see why you'd mention them.
And the people trying to get their garbage spam sites to rank high are wise to the common tricks people use to attempt to get better results. Just today I accidentally clicked on a result that from the title looked like it would lead me to reddit. I should have paid attention to the URL because instead I was sent through a bunch of redirects to some terrible spam site.
For products, one of the worst offenders is Target. Yes, Target. They show up near the top of about 60% of my searches, including for things they simply do not carry. But hey, they carry a similar product that you might be interested in! (not)
SEO results make Google money, but they also make the results suck.
Anyone I watch older than 50 years old is constantly frustrated by google, they just don't know what's good, what's not. I think they use it because it's the "norm" and that is the only reason.
And as far as everyone else.. I have no idea to be quite honest. I have a fairly big handful of non-techie friends who think google sucks but is good enough so there is no real incentive to change.
With any habit people don't change it unless they have a huge reason to. No one stops eating double cheeseburgers until their doctor literally tells them they are going to die. I think that's what we're up against here.
Hey, now wait a second there, you young whipper-snapper. I'm sitting here reading HN while I install Debian on an XCP-ng VM, and I hear I can't understand whether Google results are good or not? There are plenty of us over 50 that actually lived the early internet and watched it first-hand transform into what it is today.
Plus the topics were very Linux, DRM focused. Digg had newer more interesting tech topics
And somehow, there is still enough good, new content to keep the conversations fresh. Some of the best are when someone shares a personal story, then there are tens or hundreds of follow-up posts -- generally supportive or inquisitive.
I've had a number of replies get hammered, unfairly in my view.
Yet the signal-to-noise ratio benefits overall if we eat a little Type I error now and then.
Yes, and now imagine these same people, most of whom never bothered with basic google search options (eg. excluding entire domains, mandatory keywords, ...) presented with a search engine with a plethora of knobs and levers to tune it to their liking.
How well will they work with that?
I’m sure that’s what Yahoo! used to think about Google before Google replaced it.
I’d say the for the bulk of Google’s visitors the results aren’t merely good enough, they’re insanely great.
Now, that said, the G monopoly is horrible and needs to die.
What pushed Google over the top for me was the sheer size of their index...
That's an excellent demographic point.
A counter-argument is that the move from port 80 to port 443 is the elitist drift. Instead of innocently putting information out there and sharing in an egalitarian fashion, we now have certificates and gatekeepers and lions and tigers and bears, oh my!
Most of my search results at this point look like a spam ridden inbox from the mid-2000s.
Google is a drug dealer selling to the digitally crack addicted. The death of cocaine was meth, and the death of the web was google.
I know I get on people's nerves on HN by harping on user needs and FOSS usability and interface design and such, but I think we as a group need to start taking user needs much more seriously. Open-source alternatives will always be alternatives until without taking usability and the overall experience seriously.
Many, if not most FOSS software developers choose some commercial tools— count how many MacOS and Windows laptops you see at OSCON or FOSDEM. Consider how different the cost/benefit analysis would be for people without the most basic requisite knowledge to reason about software problems, let alone troubleshoot, or throw in a PR to address them directly. Commercial companies don't get these seamless experiences by magic— it involves research, design, development and testing to deliberately remove friction and stumbling blocks rather than assuming your use case is universal, or that docs are a suitable replacement for fixing usability problems. There's nothing stopping any open source project from doing any of it. Volunteering isn't unique to coding— people volunteer to cook and build houses and clean up trash, too. But developers run FOSS and developers need to deliberately incorporate those other perspectives into their projects to get the benefit.
If one dislikes systems that allow tinkering then why read and comment on HN. I do not understand.
Maybe HN is not the ideal forum for discussions of "the market" (due to lack of interest and/or understanding), but maybe HN is a decent forum for discussing systems that allow tinkering.
This is absolutely true; however there are riches in niches.
The % of internet users who want X probably is lower, but I’d gander the absolute number of people who would want something like this is much bigger than it was a decade ago.
This community forgets that because we’re told by investors, the media we read, etc. to go after the biggest markets possible. “Hunt for elephants, not field mice,” they say.
Don’t forget that a niche market on the web can still be massive and small fortunes can be made building products and services for them. You might not even need to have investors on your cap table to bring products to market to sell to those niches if you know what you’re doing, but shhhh… don’t tell them I told you that.
There are no more websites worth linking to anymore... If you filter out all SEO spam there is barely few webpages left...
Google is desperately trying to hide that fact. Most of the web 1.0 can nowadays fit into small town telephone directory... You do not need mulitibilion dolar business to run web directory...
I thought like you that if even Google couldn't find anything it was not there, but after discovering marginalia I now know it is just Google that has become unusably bad.
For day to day searching I now use Kagi and for me it is easily worth 10 or maybe 20 dollars a month since it "just works" unlike Google and has a larger index than Marginalia.
For now though Marginalia gets the money since Kagi is still in beta.
This is just retrieving articles when I search, not actual websites. Interesting if you're looking for article related to search keywords I guess, but genuinely unusable for actually finding something specific.
I think it showed the man page first because that domain is highly ranked.
You’re in the small minority. You’re a power user that thinks like a software engineer, and likes to deal with data in lists.
Most users are completely happy that Google tries to answer their questions and (usually) provides the right answer front and center.
1. Yes Google search has gone to shit - even putting stuff in quotes now does not do an exact search (there is another checkbox you ALSO need to use for that). It tries to be too smart even when no user is logged in.
2. Giant mega forums like Reddit have really taken over. Instead of a dedicated forum people just go to subreddits. Personally I think it is good and bad and I still try to actively participate in both.
The "we answer your question" thing could be useful, but:
1. It should be in addition to (not instead of) the web search results.
2. The answers can't be wrong. Google often gives such a low quality answer that I no longer want to use it for this, besides asking about the weather and sunset/sunrise.
I think you have the cause and effect reversed. (Though I disagree with the meme that Google's results have gotten worse. For me, it's more useful than it's ever been. I often find the information I need without even needing to click on a result.)
You don't want that, I don't want that, and I am sure many people here on HN and similar gathering places for powerusers agree with us.
But "the next google" doesn't have to replace google for us, it would have to replace google for the average consumer (AC). And the AC likes the "simple box". The AC doesn't want to fiddle with customization options. The AC is used to walled gardens, advertising, "Apps" that are just wrappers around webpages.
So any "next google" will have to compete exactly at the "simple box" game to get the attention of the masses.
You are wrong; Open Web died or almost died because of walled gardens like Facebook, Instagram and Twitter. Billions of people who hang out on Facebook, Instagram and Twitter would hang out instead on Open Web e.g. websites, blogs, forums etc. if walled gardens didn't exist.
I kind of miss forums being the place where most discussion took place.
It is a box that gives you a screenfull of ads, some spam copycat sites I wish I could remove, and a lot of clutter. As many people on here have said before, it only gets away with it because it owns the web browser through Chrome, Android and its Apple deal for Safari.
From someone who tried the switch to duckduckgo and uses ddg as the default engine: I can't remember the last query I typed without adding "!g". Google doesn't get my queries because the service is shoved down my throat, it gets them because the alternatives I tried are worse wrt the total scope of my queries.
I didn't bother checking customization-options, the ux was hostile and the results did not give me what I want. Pass, sorry.
1. Wiki article 2. Github repo 3. API documentation 4. Introduction with Python Code(Not my fav quality article) 5. Guide in R 6. Research article
I can't quantify what is better. But Google didn't give me what I wanted but DDG and you.com did. So congrats. My only issue is seeing Medium articles above the web results. While Medium might have the answer in this scenario the top web result was correct but de-prioritized which was incorrect from my prespective.
UI is nice and fast, different too. Not sure if I love yet, but will keep trying. Thanks for the link. [1] https://towardsdatascience.com/t-sne-clearly-explained-d84c5...
Speaking of doing less, I would love to see the web be more hierarchical or semantic (but not necessarily "the semantic web" as it's currently conceived). Google itself is what made the world reorganize itself. A world where that kind of search exists will reorganize itself around search, maybe not always for the better.
Concrete example of going from a hierarchical/semantic world to a search-based one: Instead of finding your socks in the sock area, which is inside your clothes area, which is inside your "do private things" area, let's say now every sock has a trackable chip in it similar to an AirTag and you just say "Alexa where's the nearest pair of socks?"
Pros:
No effort spent on putting your socks in the sock place. Just throw them anywhere.
Instant access to socks.
Cons:
Big Tech, with all its limitations and machinations, now mediates and controls the relationship between you and your socks.
You succumbed to the temptation to slack off, and now there are socks everywhere. The Roomba doesn't even work right.
1. Much clearer delineations between ads and organic search results.
2. For anything remotely commercial it wasn't the case that literally the first 4 or 5 results were ads, pushing organic results below the fold.
3. No AMP carousel
4. There wasn't a vomitous amount of those "tidbit" sections - there were only a couple and they were usually helpful.
i know what you're saying, but this isn't exactly a triviality
People want to accomplish stuff, this means they need tools to do stuff and if a tool can be customized to do the stuff faster or better people want the customization. At my job we have paying users that requests features that indeed are work related(not moving shit around). I know GNOME-minded people will disagree and they prefer to bend their work to fit a guru-s vision.,
Now my turn to ask, why do people like you think there is a generic and basic solution that works at the same time for the casual user and for the user that has a lot of tasks to accomplish? Is there some theorem that shows this, like "The GNOME theoreme of product design, keep removing features until the shit convergence to the local minimum where you find the minimum product and the minimum set of users possible.
Obviously there are exceptions, and some tools are so advanced that it's necessary to be able to customize them. No argument from me there. But for most tools, and most users, there's just no hunger for customization. Almost nobody wants to have to manage an array of options to do a web search.
What is even more shitty is when soem feature is removed like the System Tray and first they pretend they do not understand what you mean when you say this feature is very important, then after someone wastes his time to explain a n=th time again how working people use the System Tray features at work or at home the GNOME dude finally admits the issue and offers some workarounds that are not equivalent but are "possible to do but with lot more work".
On short, some people do work on the computer, some people use search engines for work to find relevant stuff, this people do not ask features like "please use the exact same padding everywhere because I am OCD" or "please make those buttons/corners or edges smoother so I don't cut my tongue when I lick my screen", this people want the customization to do a task.
I am not sure why simple people have a SEGFAULT if they randomly end up in the Advanced section of a settings section, what do they expect when they open Advanced? Google main page has a small link for Advanced search see https://www.google.com/advanced_search?hl=ro&authuser=0 how many GNOME users got hurt by this link existing?
"Hold on", you say, "professionals want options, like user-expandable RAM etc". No, that's the misconception about the concept of a "professional". Unless you are a hardware engineer, tinkering with your notebook's internals is the absolute opposite of professionalism, its either a completely misguided waste of time and money, or a perfectly fine hobby.
Real professionals get work done. Customizing their workspace is something they feel ashamed to do, because it's procrastination at best.
Just that exactly is the workspace changes a lot, the hardware internals are hardly ever a consideration for any professional, it's easier to just buy something that works well from the start (and hardware costs peanuts).
The Titanic didn't sink because of the arrangement of deck chairs, and Mozilla didn't sink because of any features they did or didn't provide (and GNOME didn't fail to achieve significant market share in the first place because of features either). The actual problem doesn't have anything to do with the stuff on the deck at all.
It's the one-two combination of vendor-lock in and bottomless marketing budgets. Since most of the value of the Windows platform and the Web is the immense amount of stuff that's built on top of it, there's a huge lock-in effect that prevents you from even reaching parity, much less exceeding it. And in order to overcome the marketing budget and pure inertia, you need to be ten times better, not just on par.
If GNOME becomes as usable as Windows, it won't have anything to do with what they actually do in the desktop environment itself one way or the other, whether it's continuing on the road they take now, or reverting everything back to the way GNOME 2 was, it's totally irrelevant. GNOME becoming usable will be entirely because of Valve investing in Wine, combined with a whole bunch of other apps moving to Electron and shipping Linux versions because heck why not?
Unfortunately, while they are probably already on par with Windows, they aren't ten times better than the Mac:
* The Mac has a bottomless marketing budget. Good luck competing with that, GNOME.
* They've shown a lot more restraint than Microsoft has, probably because macOS is considered a niche product to round out their catalog rather than being their one and only operating system like Windows is for Microsoft. They have even reversed course on a few anti-features, like adding back USB-A ports to the Macbook Pro even though it made the laptop slightly thicker. And unlike Windows RT, they didn't lock down the ARM Macs.
* Those tectonic shifts I mentioned that made Linux usable? They also make the Mac usable, because Wine is open source and Electron is basically its own operating system. Anything truly good that GNOME does, Apple can copy it just like Chrome copied all the really good stuff from Firefox.
What would work IMO is someone with money paying the developers and the designer but force them do do customer support, you don't play with the new shit until most tickets are resolved and customers waiting for response are satisfied. maybe a paid support would help too.
Market success has almost nothing to do with product quality. Well, okay, it does, but only in the sense that you need to not actually be a total fraud. You can get away with dismal quality as long as your marketing is good [1]. In formal terms, software development is a loser's game [2].
[1]: https://danluu.com/nothing-works/
[2]: https://news.ycombinator.com/item?id=26660679
This implies that the churn you're complaining about has nothing to do with market success. You might not like it, but that doesn't mean the failure of Linux on the desktop is actually caused by it.
Too many products equate frictionless == featureless these days and there should be more power when the user needs it.
That's why we built a search extension that improves upon Google specifically in ways that it's weak https://news.ycombinator.com/item?id=30725933 http://abhinavsharma.com/blog/google-alternatives
Turns out the market doesn't know what it wants until market participants see what is being offered.
It's kind of like ad-blockers where you don't have to maintain a list of domains to block. Others do that for you. And then people could create hosted version of their simple box with all the infra taken care of.
It is not a dead end if you're building a tool.
More likely, it exploits human's cognitive weaknesses successfully with a simple way. It learns how how people get their dopamine dose. And there is no going back. You need more and more, more extreme content. More polarization. All you need to do is to open app and get that dose. Is it the same for search engine?
And people make more crazy stuff to get views. How this ends? Not well, probably.
These are firms that are meeting a legitimate need- and that’s the need to feel connected. Tik-tok provides that, and very effectively. Plenty of people get genuine enjoyment out of their product, and meaningful connections do happen thanks to Tik-tok
I don't think they fulfill some gap of the need of feeling of connected in a real way. More like a bandage. We have seen the development of Facebook, Snapchat, Instagram. Their audience is fading on countries who have used them longer time, what went wrong? How is TikTok so good that they try to adapt it on their platforms as well? No way to connect?
Short video clips which might or might not lead for a real conversation. They might offer escape from reality in your lunch break at work.
I understand the perspective of "feeling connected". It brings people together with similar mindsets on entertaining way. Or at least people who seems to enjoy similar things.
On the contrary, is it different than some oldschool cults or religions? Cults which are using psychology writings as base for feeling mutual understanding of themselves. Or religions which share same ideologies and use it as a solution for their problems?
Technology is advancing, is TikTok a modern solution for finding your role and place in the world when it does not make sense and you feel you are alone with your thoughts? Maybe it is, maybe it then fills some gap.
I agree that TikTok is providing entertainment (well, that is what dopamine usually is). It is easier to hook people on short videos which are done by global audience versus Netflix where there is a limited amount of material and they cost a lot to make, when audience on TikTok is mostly making them free and you just pick suitable ones with your algorithms for showing the other audience.
However, there are many problems in this. How it can be abused and how it creates people living on their own bubble, like people on some extreme Facebook groups. When a narrative includes only content that boosts your own thoughts, a reality can be lost. We need some research on this matter, but for some reason social media companies are doing their best to prevent that.
Someone people also really get addicted on the entertainment and cannot stop using it. Well, same thing can apply also for alcohol, but is addiction risk closer to opiates for example?
I don't really believe that executives are thinking for the best of the people, so optimizing platforms to hook users is a quite dangerous play.
This seems a little sensational. Many people (myself and most of my friends included) don't see _any_ "extreme" content or polarization. A quick scroll through my feed is largely nothing but magnet fishing, frog tracking, cats, DIY projects, and geologists talking about rocks. It's enjoyable and arguably a dopamine dose on-demand, but not necessarily a road down more "extreme content and polarization".
To clarify, I don't mean with "extreme" necessarily a bad things, just content which gives you "extreme" emotions.
What kind of TikToks are you watching? Tiktok sees I like watching Japanese videos that compare American and Japanese culture so it knows to show me more of that. Trying to say that showing me videos I like is them just giving me a done of dopamine is such a weird way to phrase it. Should they be instead just constantly show me videos they think I would dislike?
I tend to agree. It's an attempt to deal with bad design by putting in switches, options, and knobs to tweak. That's because 1) it's easier than focus groups, A/B testing, and taking video of users using the thing, 2) design takes some artistic talent, and 3) it lets the programmer blame the users for the problem.
This is a vice of open source people. It's why Linux on the desktop has never taken off. "Just edit /etc/conf/foo/bar/prefs.txt" is not a design. Nor is "On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS."
I'm critical of Google's search, but this is not the way to fix it.
Consider just few more search options, alongside "News", such as "Scholar" and "Noncommercial", for when you're overwhelmed by crap. And "Popular", for when you want the crap. Don't add another interacting dimension of tweaking.
I personally benefitted a lot from the ability to like the reddit app once and then see more real reddit results?
When really innovative stuff is just about removing steps. If a process has 10 steps, remove as many as you can, and now you have something truly innovative on your hands.
I feel like Google did exactly that. Pre Google steps: search for XYZ, wait, check first link, second link, spam, spam, check next page. Post Google: search for XYZ, get XYZ.
Or worse - Search for exact term: get a page full of "Missing X - must include X" links hidden in tiny text below a result, click "Must include X" get the SAME FUCKING RESULTS again, click tools, click the dropdown, select verbatim, finally see decent results
I don't know how to easily fix that though. Simply "crawling the entire internet" is still not a simple problem, let alone doing something more useful with the result than google can. Ahrefs is an interesting business but not what people mean when they say the next google. "Machine learning" but I think google is all over this already (and has been for years).
Google does a bad job at getting user feedback about results while reddit does better so people search reddit, maybe a hybrid is an opportunity.
Crazy from a privacy perspective, but potentially very seductive in terms of convenience.
And that's great if it works! The problem is that once it fails (and, at least in my use-case, it does so quite often), working with it becomes an absolute pain.
I, too, would prefer an omniscient box perfectly answering my questions. But it clearly doesn't exist. And a box with screws to adjust so that I can eventually find what I'm looking for is the second best thing.
Google has preferences also.
It doesn't need much--a single button with "Remove this site from future results".
That would solve 99% of the problem because it would destroy SEO optimization since whole swathes of people would have different criteria due to the blocked websites. It would also tell Google which websites suck if they didn't already know.
Alas, it would also destroy their ad revenue because once a site got overly scummy people would start delisting it.
What TikTok shows to me is that users want specific things when they go to a site: for TikTok, they want fresh entertainment, plain and simple. I go to HN when I want tech discussion. I go to reddit when I want aggregation of niche topics. I go to costco.com when I'm looking to shop.
Google is frankly a horrible experience for ecommerce, it simply cannot compete w/ the likes of Amazon or any retail store website, really. Being a search portal, it's fundamentally incompatible with the concept of evolving through permanence of a hivemind; every new search is like reseting a would-be community to zero all over again, so you cannot gradually build up a collective commons on Google like you can on HN. Youtube has recently gotten pretty bad with cycling fresh content on the front page. For stack overflow sort of stuff, any other search engine does more or less the same. For trivia, Siri/Alexa are fine substitutes. The Google search properties have become mediocre on average; they're not particularly great at any particular thing.
Are many of these domains walled gardens? Yes, think of things like Doordash/Uber Eats grocery catalogs, a ton of brick and mortar shops are choosing to integrate with these delivery apps, and these catalogs are completely invisible to Google. News SERPs on Google are often garbage since they just link to paywalled content half the time, might as well just get a subscription from the actual news outlets. Etc.
IMHO, the next Google is already here, and it's everybody else wisening up to the simple business fact that they need to own the top of their funnels.
That aside, a search engine needs to be really focused on privacy. Customising it implies that the user needs to log in and remain logged in. I'd much rather no do that with a search engine, and I don't want its behaviour to react to my identity either.
It's often said that people don't want more choices, they want to be confident in the choices they make. It's the strength in simple product lineups like good, better, best. Googles lone search box is good example. Apple's product line also tends to be a good example at times.
I think it really depends on the product.
If it's something like Photoshop or Vim which are used by professionals for multiple hours a day and productivity is to be maximised, customisation is not just a nice-to-have but a necessity. Power users appreciate it too.
You're absolutely right in 90% of the cases, though I'm not sure which box web search would fall into.
Google's simple box is not attractive when others can provide the same quality in search results.
site:reddit.com {search_query} site:twitter.com {search_query}
I hope to have this as default at some point because some of the indexed results by default are just terrible.
You are so on point with being opinionated though...
I agree, BUT I would like to share my experience moving from Google to Kagi. Google serves me primarily ads, whether it labels them ads or pretends the results aren't ads, they're mostly ads. I see the same horrible domains pop up frequently, and there's nothing I can do about it. Kagi gives me the ability to remove these domains from my search, and it is INCREDIBLE. How many times has Pinterest barged into your search results?
Now, one could argue that these are domains which no one wants in their search feed. So perhaps this could be solved by being less beholden to shareholders and an advertising model. However, what about domains which most people want, but I don't? For example, I was finding increasingly polarising political content coming from MSNBC. I didn't like their "human" angle on news, and I didn't appreciate their extremely polarising commentary. So I want to remove all MSNBC results from my search feed. I understand I am probably in the minority. Perhaps my political views don't align with the typical customer. Could a well designed search provider anticipate this need of mine without giving me some kind of method to tell the provider that I don't want to see MSNBC? I'm not convinced so.
So I think the answer here is both. Design a search engine well, and give users the ability to customise it.
It's better to focus on the core basics and perfecting it than to add many features and try to support and maintain them especially if they are features that only 1% of the users use.
The old Google was able to do an exact search using Boolean expressions. Man, I miss that all the time....
It's not rocket science to provide a user interface for this. A combination of GUI configuration, keywords, and specific URLs for these cases would go a long way here without bothering the regular user in the slightest. Yes, it's probably slower than in earlier times, pure search volumina and its handling by AI weighting etc. considered. But there is also a "don't want" on the part of Google and the other players.
The useful ideas in here is the "just my data" search, linking into external providers across siloes. Of course, Microsoft, Google & Apple already have this as long as all of your data is held in their ecosystem.
This pitch voice makes me want to gouge my fucking eyes out.