My favorite was LinkExchange. An interesting little success story too [0]. We should also bring back the term "webmaster" so when people ask what you do, respond, "I'm a webmaster".
I do wonder how unique web searches really are. It'd definitely take some time to achieve a critical mass but I swear most of the stuff I want to look up I could do with an index although sites come and go so the larger your index gets the more upkeep it costs.
I think the biggest issue is live news but I mean then just pay like 12 people to watch news/sports networks and you won't be that far behind. (Or just have an section of the index for live information and let users to go nfl/espn/oan/cnn).
It’s about time. Google results are so full of promoted and sponsored content that you must use tricks to get past it. Even so most of the best things are no longer findable via Google even if you know what you are searching for!. I’ve been using Kagi for a few weeks now and really like the results. I signed up for a paid account to support their progress.
Because they only allow 50 searches per month in their free tier. You can check how the UI looks using the "Example search results" area in the home page.
I find myself still using Google for searching the web a lot, however, maybe 2% is the classical website search.
It’s mostly to find an image, a video, a location on maps, etc.
I wonder if Google could turn things around by rethinking the search experience. They’re still the fastest first stop on your journey to find something on the web. But the behavior has changed dramatically that a change in their UX alone could really make a big impact. Maybe moreso than just an algorithm change.
I once redesigned the search experience of a large global ecommerce company. The UX changes alone grew their revenue quite a substantially and reduced customer complaints.
How would you really blackbox a search ranking, though? A website owner can always just search for themselves, and see how their ranking changes.
When there’s so much money at stake, people will go to great lengths to reverse-engineer the things that put them higher in the page, no matter how you try to hide the levers that make the rankings work.
Ranking has many many signals, which are mixed in non-linear ways, and are dampened and vary over time. You only get low-resolution low-frequency sampling of the result.
I mean blackbox in the sense that I can't explain how it works or look under the hood and understand it. Many machine learning models you want to be explainable for UX or debugging (e.g. Netflix's "because you watched X"). This is a rare case where it's better if you can't figure out how it produced a result.
Instead, you want to throw queries and user behavior into a blackbox algo and have it tell you a result then give it feedback on whether the result was good (did the user come back and ask the same question? did they click the top result and leave or did they have to come back and click through many more?). I think this is kind of how google works now, though results are frequently meh. Millions of backlinks will get you noticed but your ranking will just keep dropping if users don't appear to find your content useful (e.g. they hit the back button a lot and keep searching).
>(did the user come back and ask the same question? did they click the top result and leave or did they have to come back and click through many more?).
What stops me from writing a bot to simulate that behaviour for my website (or websites of my competitors) ?
One option is to create a 'slow search engine' that only re-ranks pages once a year, doing the computation during the year and then updating the ranks all at once. You could evaluate the ranking before releasing it and patch obvious exploits, or even patch out exploits after the yearly release using data from before the release. This should slow the info leak to a crawl
Google has a big bag of tricks to gaslight webmasters, they even have patents that describe some of them.
One form of personalization that they applied early on is move results you click on a lot up in your results you can’t trust the serps you see at all.
They also inject a lot of arbitrariness and randomness to make it impossible to make small changes to your site to incrementally improve it the way you would improve an advertising campaign. One reason the web seems so frozen is that if you have a successful site and make major changes to the layout, titles, link structure, etc. you will possibly trigger the ‘chaos monkey’ and wreck your ranking and it may never recover.
PageRank is gameable but harder to game than some of the competing algorithms (such as a straight link count.)
The real consequence PageRank had was that it got web pages to stop linking to each other…. Google gaslighted people into removing the competition they could have had navigating from one page to another through links. (e.g. web directories)
Google didn’t gaslight the web: Web directories died out because they go stale, fast, and require far more human curation than is economically viable. I remember Yahoo effectively deprecating their directory pages long before I heard of Google, back when I was using Dogpile and CompuServe.
I never saw a web directory that took a serious approach to using automation to curate the directory. It's probably more feasible than people think because it's a matter of classifying links up or down (relevant or not) which is a much more ontologically and mathematically tractable problem than learning a ranking function or trying to classify things into one of N>2 categories.
I think the problem may more have been the lack of a sustainable revenue model. Getting volunteers to curate the directory is particularly destructive because the people who most want to volunteer either (1) want to promote something or (2) want to get paid so somebody else can promote something.
No, I think a modern directory would be a set of topics in an ontology with links. If I had to seed one I would suck all the external links out of Wikipedia, impose some organizing structures (probably overlapping trees or dags) and then build a set of classifiers for nested topic relevance, spaminess, etc.
There are certain sites which have a landing page for every topic in some set of topics, you could add a lot of links quickly if you built rules for importing links from particular sites. Adding 10 sites a day with 1000 links each would be very possible, in 100 days you could build out a million links.
Exactly how do you think Google (and Bing, and ...) work? They do start from known indexes, in particular wikipedia. Hell, Google even has internal papers where they claim they improved search quality by "wikipedia's" as a unit.
Actually Microsoft bought a company called Powerset which did information from Wikipedia to build a "semantic as in semantic web" index, that technology became the heart of Bing.
Google was caught flat footed and wound up buying and killing Freebase in order to catch up. They lied and said they rejected "semantic web" approaches despite hiring one of the leaders of the Cyc project as their head of research.
Still there is a difference between exposing that kind of database through a full text index vs exposing it through a browsing interface.
It's not quite the same as the web directories of old, but I find the links and content contained in the "Awesome [topic]" lists on GitHub to be a useful resource. The curation aspect is managed via pull requests just as any other open source project, although this doesn't make it immune from going stale.
I'd say rather that it wasn't designed for modern adversarial scenarios. Google won the search engine wars because PageRank did better in the late 90s adversarial environment, where every page was packed with white-on-white keywords. It was significantly harder to create a counterfeit influence graph than to keyword spam with hidden text elements.
That all seems pretty quaint these days, but even worms and viruses at the time were versions of "my goodness, who would ever write a script that emails itself to your address book, or copies itself across the network to all the other unsecured PCs with passwordless full hard drive access?"
Was researching Ad spend as the ad-tech industry is falling apart.
https://imgur.com/gallery/1Aw3t4i
This is not a search result anymore. This is an outrage.
Honestly not sure if search engines are the future at all, at least for commercialisable content.
More and more I find myself searching through moderated social media like Reddit or Facebook Groups/Marketplace for product recommendations or local services.
Gave Kagi a go, but it seemed to be even worse than Google for what I tried.
i think general purpose search engines will always have a place, but they're ultimately a tool you use for finding information, not necessarily a tool you should be expecting to make decisions for you.
so many of the complaints about google search results seem to be along the lines of "i asked google what X i should purchase, and they served me an ad", whether that's a first-party ad on google or search results that are external ads. If you're only attempting to use a search engine to find facts, it's a much more satisfactory experience.
search results for opinions are and have always been trash - they don't serve you the best opinion, or the most trustworthy, or the most well-reasoned. they serve you the opinion most relevant to your search terms. and that's probably not what you want.
Isn't FB marketplace just another search engine? It seems unlikely that the algorithm would be able to purely guess what you currently (actually) need.
And moderated groups don't sound like a solution to the 'I have a concrete question that I want an answer for right now' problem either (unless you're thinking of massive live chats with many lurkers, which sounds more like the web of the past, tbh).
There are a lot of public articles stating that PageRank is no longer in top signals any of the mainstream search engines for more than a decade now. The patent already expired years ago and no one cared. It's surprising CEO of a search engine company just woke up to this. The future of search is AI model-as-index infrastructure.
I've always wondered why Google doesn't have some way to provide an explicit signal from the users like favorite/bookmark or explicitly like/dislike a search result.
It's often pretty fustrating to try and find something you know you've seen before but don't remember quite the name of or exact keywords.
Google added some "plus" button when Google+ was pretending to be a thing. It made no sense though as how would someone know it's a good result before they visit it?
Brave browser uses "attention" as the main factor. It's currently used for their paying the content creators and charging for ads, but with their new search engine [0] it may be a factor there too.
What a completely disconnected-from-reality article.
There are so many ways to use Google to obtain the information you're looking for, and so many people do it literally on a minute-to-minute basis that it's flat absurd to call that ineffective.
It's trivially easy to learn something by typing a question into Google; maybe when writing an article try gut checking it against obvious observed reality first.
I simply cannot fathom a noble reason why the author would decide to publish this.
Edit: Oh wait I found it; the author is Vladimir Prelovac, CEO, Kagi Inc. What does Kagi Inc. do? I'll leave that as an exercise to the reader, but I bet you guess it in one try...
Search peaked 10 years ago. There's lots of regression in Google Search. For example, you used to be able to search images by specific dimensions. You can do this today on Yandex. The quality of results and promotion of sponsored results, the scraping and displaying of site contents so users visit less (how is this legal btw?) Not respecting verbatim searches.
That's not really what this article is about though, it's about how PageRank doesn't work anymore because advertising is at odds with returning valuable results, which is complete nonsense for a gigantic subset of the value that Google offers.
Also it's written by a CEO of a competing search company. It's literally the very SEO he's railing against...
About '08. Spam took over right after that. Used to go in waves of better/worse as they played cat-n-mouse with spammers, but they seemed to kinda give up right around then.
For me, almost everything that's not coding related, and that's after all the SO clones etc have been removed from the results, is SEO "optimized" at best and SEO spam at worst. Most websites that show up on Google follow the same recipe of headlines, quotes, stock images and repetitive content and annoying introduction paragraphs that place them at the top of the list.
> Edit: Oh wait I found it; the author is Vladimir Prelovac, CEO, Kagi Inc. What does Kagi Inc. do? I'll leave that as an exercise to the reader, but I bet you guess it in one try...
Point taken. But it's not as though this was concealed. The top line of the page is "Tales from Kagi." The piece is hosted on blog.kagi.com. The author's signature at the end of the piece reads "CEO, Kagi Inc." The final line of the post is "I hope you join us [i.e., Kagi] on this journey."
Writing blog articles like this and posting them to social media to create crosslinks is a very common SEO strategy to "trick" PageRank into promoting your site higher in search results.
It's the whole purpose behind all of those random blog articles on corporate sites; to improve PageRank ranking.
Google is a horrible search engine, you routinely have to hack it to get searches not linking to the centralised web. The first 5 results are ads, followed by Youtube links or a Quora/Reddit on the topic.
If you used the web 20 years ago you know what a good search engine was like.
I started using Kagi and Orion a while back and it has changed how I used the web. Both the products Kagi offers has only increased my productivity and removed the internet junk that I am used to seeing on the web. A note on Orion, the ability to use extensions from both Firefox and Chrome store is why it is admirable, the cherry is Mac integration and its similarity to safari with the tree-view tabs.
There was a recent bug fixed, specifically on YouTube, due to ad-blocking causing a constant loop fetching and reloading an icon or something to that end, I believe - it was fixed in a recent release.
I suspect that was what you were seeing, and if so, updating should have addressed it.
I've been using Kagi the last few months. I've never had a reason to complain about its search results - they seem to work plenty well enough day to day that I don't really 'notice' them. Like most, I now search reflexively, as an extension of the mind, so I only 'notice' search when it's bad.
What really excites me is that is that I'm paying them. That sounds odd, but seriously. It's incredibly refreshing to know that the company providing my search results has an incentive to make things better for _me_ and not a legion of advertisers. With Google I can't help thinking about every keypress being logged to optimize sales pitches at me. I just don't feel that with Kagi, because I'm paying them.
Sure, they might be logging every keypress (I don't actually think they are, but you never can tell) but even if they were, I could be reasonably certain they were doing it to retain my subscription, which probably means making my search better, not selling me other stuff.
It's a priveleged position to be in, and the economic argument isn't watertight, but in the "search as a brain extension" space it still _feels_ premium, because it creates trust. And that frees up brain space for other things - like where the hell was that article I was looking for?
If they have an auto-complete feature, the keypresses WILL be seen in not even in a smuggled way. They'll be directly in the URL to the search function!
You could probably hide that from anything but a deep dive by chucking the data through a websocket. Unless you see and recognise the initial socket connection you may miss it entirely.
The economic argument not being watertight is the problem. This is why tech companies keep pivoting to ads, they just didn't make enough money otherwise.
Afaik [1], there's about 5 employees and the revenue only covers server expenses while they're still trying to get more headcount. Not an expert on bootstrapping but I'm pretty sure you don't want to expand faster than your revenue does otherwise you stop being able to make all the decisions for the company.
They mention that users can invest in them through SAFEs. This only makes sense if they plan an exit, either by selling or through an IPO, or am I missing something?
I think it implies that they intend to take VC money at some point in the future, like a series A, and they’re trying to avoid VC money now while they can get better terms. I don’t know if that necessarily means they plan an exit, or if they’d be happy just being profitable, but I do think it implies they plan to grow significantly.
EDIT: Their website says that they don’t plan to take VC money, so I have no idea what a SAFE means in this case. The point of a SAFE is that the investor is guaranteed the best terms when a priced round of investment happens.
If a company is doing sustainably well, supposedly you could have a mutually agreeable mechanism to convert SAFEs to equity (at cap for example) and treat shaleholders with dividends.
The problem is that ad revenue so often becomes the escape hatch from an economically hard problem.
Need more money? Economic model not quite working? Companies face this all the time. Some solve it innovatively, survive, and show everyone else the way forward. Others fold or pivot.
But ad revenue short-circuits the hard process of economic innovation by tempting tech companies to take an easy way out, even when the problem is solvable without it.
(All of this wouldn't really be a problem if the ads business model weren't inherently adversarial against users.)
The sad truth is on average advertisers will pay more for your attention than you’ll pay not to have ads. So people continually complain about YouTube ads but ignore their option to pay to have them removed.
With YouTube premium you can listen to videos on the iPhone with the app in second plane or even with the phone locked.
Yeah, I should use a hacked rooted Android yadda yadda. But I just spent a week hacking and hawking for my life in an ICU, and I certainly didn’t want sanity-saving podcasts to shut down whilst on an hour-long stay at the breathing machine. Life is short.
I pay for YouTube too. Between that and Kagi search, it's ~$20 per month. Which seems crazy in today's environment of 'everything free'. But seriously, I use these both all day for both work and play, but I only watch an hour of Netflix a day. So which is really more valuable?
I agree that on the whole, people will continue to go for the free option, but I'm just glad that some options are appearing to pay, which rebalances the incentives a bit.
I think another factor is the paradox that the user who has enough money to be willing to pay to not see another ad is exactly the most valuable user to advertisers— they've pre-qualified themselves on multiple axes.
So there is always going to be an enormous moral hazard here with the ad-driven-freemium model— always a temptation to run an occasional high-value ad to the paying users, or to disguise the ad as a "recommendation" or position it instead as a "sponsorship", or whatever else. Other than users reacting swiftly and decisively against this kind of thing, it's hard to know how else to keep companies away from it.
One of the problems is price point. They are charging $10/month. While increasing the price will certainly decrease the number of users, a much higher price point could select for "power users" that will churn less and pay substantially more. In the long run that could lead to greater revenue.
Of course this depends on how the supply demand curve looks like for their specific business.
This showed up at an internal Twitter meeting this week. Mr. Musk proposed making Twitter ad free for 8$ accounts before being politely informed that 8$ accounts actually produce 50$ per month in ad revenue.
Because it's a pattern that manifests everywhere and can be extrapolated.
Though in this case, it's likely that Twitter has repeatedly run internal studies about what a paid-tier would look like, and each time concluded exactly what the GP is passing on: that it would be a net loss because the most-valuable-to-advertiser accounts would be the first ones to pay.
Even just psychologically, it would be a major blow to their sales pitch, going from "pay $X to put your name in front of the top movers and shakers in the world" to "pay $Y to put your name in front of all the Twitter users too cheap to cough up $8/mo to not have to see you." Not hard to see that $Y is going to be less than $X... probably a lot less.
The reason they advertise is because they can. It doesn't matter how much money they're making now, they can always make more by advertising. Paying money just makes your attention more valuable. The only way to stop them is to make it impossible or illegal to advertise. Blocking ads everywhere and making no exceptions.
When google started, they were having difficulty staying solvent until they added ads, even though they had a better product than the other search engines. Its a space that's ill-suited to market solutions.
"Enough money" is usually defined by venture capitalists who want ALL THE MONEY and not by normal investors who want a modest predictable return. Patreon, for example, could have been a modestly profitable business which owned itself, but instead they keep flailing and spending lots of other people's money to satisfy the next batch of investors that one day they will be as rich as Google.
Google's revenue is over 280B annualized from last quarter's numbers. Let's just assume that every person on earth is using Google at an equal rate. That's $35 worth of revenue per user per year. There is no way that you can come close to that without doing ads; the median global income is less than 4000 international dollars per year. So naturally, there is a strong pressure for companies to pivot to ads once they run out of other ideas to grow the business.
* one does not need $200-300B ARR to run a search engine; and merely making a ton of money does not mean that the way that one has been making that money will continue forever
* using Kagi, one realizes how much the web has been perverted, in my opinion, by the obsession with the ad model; there are many nice, novel things that Kagi could introduce which would reduce ad revenue for a company like Google, but would be useful and novel for Kagi without loss in revenue (on the contrary — could help); ie google is incentivized to keep people on the site through various irritating means; a search engine that isn’t distracting and gives me what I want — I would be more than glad to pay for (and already do pay Kagi $10/month)
Other thoughts:
* Supposing $10/month (honestly I would consider $20) with 100M paying users; that is $1B MRR which is not bad at all and is more than plenty for a meaningfully sized team with meaningful salaries.
* Just like how lowered energy prices during and after the Industrial Revolution made manufacturing at scale feasible, the incredible amount of high quality OSS and infrastructure these days is making it increasingly feasible to do things Google did 20 years ago — something unthinkable for even the best engineers of that era. Not to mention the relative ease of collecting capital with payment services (let alone VCs, etc.) today — even with a looming recession factored in.
what I find strange is that selling ads makes more money than selling service. Because at the end the people watching the ads are buying the stuff advertised so those who adverstise have more money for advertising….
So for it to be rentable people have to buy way more than the ad revenue google makes. If spending for google ads is 1% of bussines budget and budget has to be at least Revenue……..with 280 billion revenue for google it would be 280.000.000.000.000 that we as humans have to spend for stuff that was advertised on google so someone pays google for ads so they can run google search etc ….
35 dollar is maybe ok… …. Maybe better than spending 3500$ on … I dont know … stuff is good also ..
If you show an ad for a car that costs $100k to 1000 users and a single user buys it (0.1%) you still sold a car for $100k. Let's say it costs $90k to make it. You could pay $5 per ad displayed and still made a profit. My point is, the people who actually buy the products are actually paying for the other people who can't afford it. So, the model doesn't rely on everyone buying a keyring from a car dealership, but on most people not buying anything, and a couple buying a car.
In a sense, that model is better at distributing income, because everyone gets a service and the people who can afford pay for everything. If everything was behind paywalls, people who can afford would have all the nice services and people who can't, will have nothing.
What's interesting is that "targeted ads" is a bit of a misnomer. It's more of an ads ranking, where the user is shown the ads they are most likely to buy. However, from the point of view of the car dealership, when they think targeted ads, they would like their ads to be shown only to people who will likely buy the car. No company would want their ads shown to someone with no disposable income, yet, they will be shown some ad.
Advertising is kind of a backdoor price discrimination in that way. If you can afford to spend $10,000,000, people will advertise wealth management services to you. If you can afford to spend $1, McDonalds will advertise their dollar menu to you.
Looking at the global median income and coloring it with the mean revenue per user paints an incorrect picture. Advertisers are not paying even $35/yr for the attention of people making the global median wage.
US attention is the most valuable because it is the richest large nation and the largest rich nation.
European attention comes a distant second. Together they have 40% of the global GDP, but likely something like 60-70% of the consumption of non-essential goods in nominal terms. Accordingly, average advertiser cost per clock differs ~10x between the cheapest and most expensive groups of countries.
> ... once they run out of other ideas to grow the business.
Yep, but the main problem here is the perverse pressure to grow indefinitely.
And that comes from it being acceptable for public companies to pay no dividends and instead rely on share price increases to deliver a return on investment.
Something that slows down this growth spiral would fix things partially. Require profitable companies to pay a % of the profit as dividends to their shareholders?
While I am very willing to pay money for a user-centric search engine the requirement to do so directly conflicts with the greatest must-have feature that I need from one: Anonymity.
Now the issues is not just privacy but making sure it does NOT optimize search results based on prior searches or other facts it knows about me.
Now the big problem is that one would think delivering the exact search results that I am looking for is a good thing. No. It is super dangerous. It can causes me to live in a bubble, blinding me of possible opposing opinions. It is feeding my ignorance.
So there is a goal conflict here. I don't know how to solve it. I wish there would be something like Wikipedia for search engines but even the wiki models has its problems and biases.
I like the idea. Maybe a 'Search for me' button instead of the current 'I'm feeling lucky'. Or two columns or tabs for personalised and general results. Sounds easy, but I am not aware of any search engines that offer this, unfortunately.
On the other hand, a lot of the search results bias stems from the search query itself. "Does X cause Y" for example will show results that echo the search, and not show opposite results. Simply because you did not ask the opposite or a more open/unbiased query like "X Y".
That doesn't work unless the server sends all 1 million results (say) to the client up-front. Otherwise, if you prefer results from example.com but the server only sends example.com results on page 3, your client has no way of showing you those results on page 1.
I think I've been misunderstood, in my mind the client sends all the details about themselves to the server, the server builds up the search results and send them back.
Yes, the server has access to the information the client has, but it doesn't have any way to confirm that's true. It also could avoid storing that information all together if the client sends it again every time
I think if you can assume some level of trust in your search provider (i.e. through payment - let's get to that in a minute), without which you should probably be looking for a new search engine, then you can trust the search engine to not optimise results for you. If they tell you they don't, then do, then you likely have deeper-rooted issues with the search engine you use.
On the point about payments, there are some potential approaches to handling this, but they tend to be a bit complex and technical. I"m not sur ethis is really a technical problem though (or at least one where a technical solution is what's needed).
A challenge of taking traditional payments is that the identity of the payer is inherently "linked" (to some extent) to the transaction. You could maybe use a privacy.com type intermediary or similar.
What you could do however, is buy a "kagi voucher" or similar, which would be an RSA-blind-signed token that you generated client-side in JS. You'd send it to the server during the payment transaction, and be given back a signed version of the token, which you unblind client-side, then give a backup copy to the user to save, and set it as a cookie to authenticate payment has been made.
That should, in theory, at least un-link the payment from the user. Ultimately though, a bad actor can still trivially correlate your searches (and payment, if you used same IP or browser fingerprint), and all your searches are linked by a common cookie.
So you decide to create some kind of token fountain, which you can auth to with a blind-signed token, and get 50 "search passes" from. Those could be unique per-search, and just be "bearer tokens" you can use. You'd have to trust that this token fountain doesn't record which search passes were issued to which "blind token".
Ultimately, it feels like this problem then becomes "do you trust your search engine?" - I'm not sure this is a problem that technology needs to solve, as ultimately your search provider is seeing your queries and IP, and most users won't realistically change this. A provider that isn't incentized to act against your interests (i.e. something like Kagi which isn't advertising to you or selling data) seems to address this for most user scenarios.
I don't think there is much cause for worry that Kagi themselves might use the data for anything bad BUT data leaks do happen and more importantly Kagi will need to play ball with whatever government that exists in the markets they want to have users in, be it the USA but also China, Russia or whatever.
We already know that Google is in fact cooperating with these entities. Not only by giving out user data but also by actively censoring certain things. So there is a precedent.
It sounds like you use the internet way different than me. I can't remember the last time I searched something where the end result wasn't mostly objective. Documentation, directions, tour information, movie times, programming questions, patch notes. Maybe seek contrary views out in the world. The internet is a terrible place to form a world view and to really communicate in general.
Agree, however we are already in trap of algorithms. Look how few years of using social networks spread ideologies that would be laughable back then, ie "there is more than two genders" or "earth is flat".
It can causes me to live in a bubble, blinding me of possible opposing opinions. It is feeding my ignorance.
This made internet completely unusable for discovery. A big shiny window into another tiny room. Absolute BS.
Recently I discussed it here on HN and was advised to like, dislike, subscribe and so on on youtube (things I rarely have done on my old account) to break out of this bubble. Well, it doesn’t work. It just suggests the content I immediately dislike or ban entire channels and it all settles to the same echo chamber as before.
As a result, I often find myself closing youtube tab because there is not only nothing to watch, but nothing to scroll through even. On a platform with billions of videos, at least thousands of which I might like to watch if given a chance. I find this incredibly stupid.
Same for google search, really. It turned from “I’ll find you that if it exists” into “Look, if you change your query to ‘cool products if at all barely relevant to …’, I have literally billions of results for that”.
I'm with you; I'm a paid Kagi user for a while now, and unlike various attempts at switching to duck duck go (et al), Kagi is actually fully working for me. I do not switch back to Google more than once a month, and each time I've gotten "worse" results so far.
Do you stay under 30 searches a day? On their pricing plan it seems users searched a lot more in their beta program than anticipated. 240 searches cost them around 3 dollars.
>It's incredibly refreshing to know that the company providing my search results has an incentive to make things better for _me_ and not a legion of advertisers
Cable television and netflix have made it quite clear that payment does not mean “no ads”, it just means “no ads, yet”.
It still might be a better alternative in the short term, but the moment growth starts to peak, companies follow where the incentives lead them.
Having the actual user pay is a necessary but not sufficient condition for a user centered product or service. It doesn’t guarantee the company won’t double dip but without a direct economic model nothing is even possible but ads and surveillance.
It strikes me that a company like Kagi should be able to craft a legally enforcable agreement with its customers which expressly forbids the company from selling ads and conducting surveillance.
The agreement could be carefully written by a skilled lawyer to define the things Kagi cannot do, the proof customers must present in order to proceed with a valid lawsuit, and even the maximum damages that the customer may sue for.
In that case, if Kagi was found at some point to be using customer data for these purposes, it could be sued very easily and by many parties.
People are calling for regulation for data privacy. In the meantime, Kagi can create its own regulation it will hold itself to for the benefit of its users, can it not?
I'm a little tired of the rhetoric about Netflix adding ads. It's only on an entry-level account that is subsidized from the normal pricing (IIUC, correct me if I'm wrong). You could argue that they've artificially inflated the price of the normal plan or that it's a slippery slope, but it's not like they've forced ads on the existing plans.
I think what's more interesting/concerning/insidious is hidden ad content like product placement. It's getting to the point where personalized product placement could be embedded in shows dynamically.
I would consider the Netflix UI to be ad laden for some years now with hundreds of originals shoved in your face in a non-customizable fashion and autoplaying etc. Also personalized thumbnails based on past watching behavior. You could argue that it's a good UX for most people, but for me it's always felt a bit more like user hostile marketing.
Netflix doesn't have enough content, so they resort to these tactics.
All major content providers left them years ago and started their own Netflixes. So they're left with a few movies from ten years ago, a couple of recent-ish releases, and their own content.
really, content creators should be banned from creating markets like Netflix (and eventually the other way around), but that would require the regulation to not be asleep at the wheel.
Should've specified I meant movies and other large conglomerates that tend towards oligopoly. I mean things like Disney+ shouldn't exist, its extending the concentration of market power vertically too.
I real wish that HN wannabe lawyers would stop throwing *poly words around with no legal justification.
In the streaming space in the US there is: Netflix, Disney+, AppleTV, Amazon Prime Video, HBO/Discovery, Paramount Plus, Peacock, STARZ, and a few other players. There is no “opoly” in streaming video.
That’s still a small number of players dominating the market. That’s the definition of an Oligopoly, and the Disney/HBO offerings very much fall into that.
They certainly have the ability to uniformly raise prices (tacit collusion) with no viable competition to enter the market and fill the gap (as even the vast sums others have thrown into it have shown how hard it is to produce good original content).
This is probably due to the characteristic that producing goods (decent original content) in this market is a big barrier to entry - which naturally leads to a small number of players. Natural monopolies and oligopolies are common - but do require closer regulatory attention to ensure desirable consumer outcomes than just letting the free market decide.
It may not entirely fit all definitions, but the general economic theory and applications/implications are relevant to consider this market.
The original post that you questioned was related to vertical integration - you could effectively find and replace it to “so you’re saying producers of operating systems shouldn’t be able to make their own web browser?”
> That’s still a small number of players dominating the market. That’s the definition of an Oligopoly, and the Disney/HBO offerings very much fall into that.
The market is “content”. Netflix competes with YouTube content producers, TikTok and it even said that one of its biggest competitors is Fortnite.
> This is probably due to the characteristic that producing goods (decent original content)
This goes back to YouTube. You and I may not think that YouTubers and TikTokkers are producing “decent original content”. But there is a generation that spends hours on both.
Besides that, there were over 550 original series being produced last year (https://collider.com/too-many-tv-shows-550-series-2021/). Competition is much fiercer for your attention than it was when you only had the three major networks producing content and everyone else buying rights to show reruns.
There are bidding wars between all of the streaming services for new content from producers. Competition is more fierce than ever.
The price of streaming before was never sustainable. Netflix was borrowing billions a year for years to produce and obtain content. Disney+ was never going to be profitable selling its service at the introductory price. It’s not “collusion”. Every company has to turn a profit eventually.
Yes I realize that Netflix was “profitable” by GAAP standards. But it was getting deeper in debt every year.
Very little as a percentage of all the professional content that’s in the world.
And the value of IP without execution capability is overrated. Warner also has iconic content. But Disney has been able to make successful movies out of its 3rd tier IP while Warner has struggled with its first tier.
Sony isn’t doing too well with most of its Marvel content except Spider-Man and that’s produced by Disney.
>The original post that you questioned was related to vertical integration - you could effectively find and replace it to “so you’re saying producers of operating systems shouldn’t be able to make their own web browser?”
I think time has shown MS was correct in making their own browser, but of course incorrect in all the corrupt tactics they used to make their browser succeed over Netscape's.
This is severe rose colored glasses. Netscape was always a horrible application and crash prone on every operating system it ran on. There were geek wars back in the day bragging about how well our operating systems handle Netscape crashes - classic MacOS failed miserably.
IE was a much better browser. Especially the Mac version that when it was introduced, was the most CSS compliant browser that existed.
Netscape starting over from scratch was cited as “Things you should never do” in an article written by Joel Spolsky (StackOverflow cofounder) two decades ago.
It's not just netflix. I hate the my Google TV pops up with ads for media. My PS5 boots straight into the store. At least my Apple Tv mostly doesn't shove ads in my face though my iPhone seems try try to shove Apple Music and/or Apple TV+ at me now and then
I agree with your product placement comment. I feel gross when I notice it. I also wonder what happens when Netflix decides it makes way more money on people watching the ads - ie. Google initially calling ads a detriment to search quality but not being able to resist the $. I could see a day when they remove the no-ads plan.
HBO started broadcasting 50 years ago, charging a monthly fee for commercial free movies (often released much earlier than on cable) and tv shows.
With the exception of filler ads for their own content that occurs when they need to wait for the next quarter time, ex movie ended at 12:55pm. There have been no ads
Chromecast screensavers _are_ an add for other shows in Netflix.
I am paying and have no way to disable that visual pollution nonsense.
Extremely exasperating when you are trying to choose something adequate for toddlers and they keep seeing flashy stuff on the screen and saying they want to watch that.
It's not about the user having to see ads. It's about Netflix trying to sell ad spots to advertisers, which might affect all (even paying) users. I.e. by getting pressured into censoring or promiting certain shows, to close the advertising deal.
It's not really about Netflix. The point is merely that "just because you pay them, doesn't mean you're not also a product to them". Companies love nothing more than to charge both ends of a deal.
Take Spotify as another example. I pay them, along with 188 million other people. Does that mean they won't turn around and ask artists, record companies and rights-owning conglomerates for money (or rebates) for putting their stuff in front of me? Of course not. Paying them means they have some interest in pleasing me, but it's far from the only way.
On the off chance that Spotify was being scrupulous about not taking payola in any form, it would be impossible for me to verify. Which is in itself a reason for them to cheat; they don't get much economic benefit from goodwill if no one can actually see them being honest.
This is not a reason to not use Kagi, it's just a reminder of what forces we're up against. Kagi will need an unusual amount of transparency in everything they do, in order to stand a chance in the long term.
And it IS a reason to not get warm fuzzy feelings merely from the fact that you're paying them.
If that happens, I would then look at other search engines as well.
Most people wouldn't once they get used to doing it one way, so that's all to the good for Kagi. But people like me wouldn't then use it, which is what life is about - your own personal choices.
I prefer to pay for services because at least there is a slight chance that they will follow their business plan.
But nothing stays the same for forever. As a consumer, we must also change to the changing environment. I'm completely weened off Google, for quite a while now, for example. I'm paying for everything that I use. But it is so little as to be laughable. For example, I use tutanota for my email, first account is free and subsequent ones are $1/month. Big whoop. $12 per year for private emails. I have a number of emails through them because I segregate all my emails - one for just friends, one for business, one for education, etc.
2. Create a new plan at the price of the old plan. With ads.
I'm sorry, but pretending that they're not adding ads to their "regular" pricing plan is semantic at best. This isn't something they dreamed up overnight. They've been planning this for years and increasing their prices accordingly.
Can you add a poison pill to a company where a select group of people can decide you broke one of your founding principles and hand the company over to someone else?
No VC startup would do that but if possible would allow trust.
I think it’s important to consider whether this happens with all companies, or whether venture capital seeking to maximize returns to the detriment of both the business and the user is a factor here.
If they plan to go public ever, its pretty much an inevitability, at least if US market incentives stay as they are. Cable isn't venture capital funded, and they're rife with profit maximization.
Cable was never intended to be ad free. It was originally a method to rebroadcast over the air content from the broadcast stations in areas that couldn’t get a signal. You’re paying for the infrastructure to make that possible - not the content.
Complaining that cable TV isn’t ad free because you pay for it is like saying all content over the internet isn’t Ad free because you pay for your internet connection.
If the only metric of success is yearly revenue, yes.
If that is the only metric, the best business model is to extract as much money out of your user, while simultaneously offering less for the same amount of money, until you reach the asymptotic ideal of 100% profit.
If you were to start a company, would you be willing to do that? Would you choose profit over everything else? Some would, but not everybody.
Companies have an incentive to make money off of their customers' data, and technology just means that they can make money at every level (your ISP, your smart TV, the apps you use on it and the websites your browse).
We're so accustomed to entities gathering data about us that it's become part of the background.
Why is it the default that my home is on Google Maps/Google Earth? I should have to opt in, not opt out.
Just because there is "public" information about me as a registered voter or home owner doesn't mean that anyone should be able get a copy of this data, put it online and connect it to other data?
I just tried Kagi and was surprised that my open source project shows up in the results for relevant keywords. All the other search engines I tried bury it completely out of view no matter how specific the keywords are. It makes me wonder how many other indie projects like mine are excluded from mainstream search engine results.
Yes, it finds it. It ranks quite well for most keywords I tried. It even got #1 spot for one of the keywords I tried. I also noticed some other related open source projects show up (by other people) which I hadn't heard about for several years.
Google search only shows paid SaaS platforms in the first page of results.
I believe it's a responsibility of organizations themselves to orient this way, and for customers to be vigilant about this orientation. Does your search engine have weird incentives to exploit you? Change search engine :)
----
I have an idea (more here[1]) that the next addition to our civilization should be a distributed system to evaluate, reward and compensate externalities. Open source software is a great example. It's free, it has massive benefits to users and for people reusing it for all sorts of projects, it's public. Yet we still can't find a great way to fund it. There are donations, and maybe that's all we need in some sufficiently enlightened society, but I think we should be addressing those more directly. And clearly there are technologies which generate friction and other social costs, that could be (differentially) quantified and prices. The same goes for pollution and a miriad of issues.
Laws can help here, but laws tend to lack softness and specificity in my opinion, and laws are strictly punitive. If you're generating a negative externality, and the legal system is well designed, maybe you'll have to compensante for it. But that relies on quick lawmaking and judges evaluating something they're not specialists at. And there's no dual for punishment (actually compensation and reeducation), there's no intrinsic reward system. There are prizes and grants, and I'm sure they generate immense value for what they are, but they're still very unsystematic and unreliable, I think.
I dream of a system where if you generate value for society, you will be recognized and make a comfortable living off it; if you need investment, you will also be supported. (I call this idea Elementalism; I'm not sure it's original)
I think we need to be a little more open to careful, thoughtful changes to our social-economic system that can improve things.
While Elementalism isn't a formal part of our system, each of us can do our part and give directly to what we understand needs the most. This is the core idea of Effective Altruism which I also extend to donating to Open Source (which I believe can be effective[2])
Most of their cost comes from API accesses to other services (e.g., Bing), so unless they can replace those API calls the price cannot go down very much.
Or give notice to power users that they are exceeding quotas by a lot. Have a power user subscription perhaps. Many people don’t download a lot and ISPs are good, but you have outliers, datahoarders perhaps who take a lot of bandwith.
> Our proposed price is dictated by the fact that search has a non zero cost. With other search engines, advertisers cover this cost. But it costs us about $1 to process 80 searches.
I just tried it for a few terms that Google found no relevant results for (related to Wasm/Rust); and holys*t there were many results from people (developers) who wrote in their blogs.
I'm a user now, and potentially a customer if I hit their free limit.
Very naive . It just means that in addition to whatever google do with your data , they also charge you money on top of that . When people have power to do things to maximize their gains (selling u ads/ data in addition to charging u ) , expect that it will be done. It is the same in politics. Power corrupts.
The reason that I'm not using Kagi (and some of my paid subscriptions) is that payments require login, and login makes it possible to track my search history. I'm happy with paying them with money, but not with my keywords.
Is there some zero-knowledge protocol that could help here? In particular, to establish that a user is currently subscribed, without revealing which subscription they have?
This reads like this is the death of all network centrality algorithms, and no way, guys, no way. At most this application will somehow decline even if only for a while.
But then, you can't really search youtube as efficiently as you can texts. That's the only downside for me, sieving through tens of videos just to find something that might be in the middle that's releavant to what i'm searching.
That's when the video solves your problem. But how do you find the video that solves your problem? You can't skim through it quickly like you can with text.
it amuses me greatly that 30 years in to the great commercialized internet experiment, we still don't have durable mechanisms for publishing, spam filtering, identification, reputation, moderation and discovery.
Purely anecdotal, but as a matter of fact yesterday I spent 15 minutes searching for a web site on google (I couldn't remember the domain name), threw up my hands, went to duckduckgo, and nailed it on the first try.
The article suggests that search engines will become a personal choice. I rather think that good search engines will become such a competitive advantage that companies will buy search engine access for their employees.
> The more you tell your assistant, the better it can help you, so when you ask it to recommend a good restaurant nearby, it’ll provide options based on what you like to eat and how far you want to drive. Ask it for a good coffee maker, and it’ll recommend choices within your budget from your favorite brands with only your best interests in mind. The search will be personal and contextual and excitingly so!
Actually Google is already doing this. Results are personalized and contextual. It can't really know what I feel like eating this evening because I don't, but it can guess.
I applaud the author for trying, but I don't see an alternative to PageRank being proposed. How exactly is Kagi proposing to rank results?
And no, I don't want an AI generated summary when I search for the best tutorial to do X. I want a list of tutorials. The question is how to rank that list, and I've yet to see anyone do a better job than Google.
Google is far more than PageRank. Its an AI ranking model that has the largest training dataset (queries and clicks).
Ads may suck, but simply charging for the same service isn't really innovative. Would I pay for a version of Google without ads? Probably not. But that's just me - I actually like to know who is advertising for particular searches. A company with an ad budget to rank at the top of ads is probably more trustworthy than an anonymous website.
Oh but it most likely does use PR. Where do you think they get the results from? Are they scraping the web themselves? I highly doubt it. Results probably come from Bing.
>Ads may suck, but simply charging for the same service isn't really innovative. Would I pay for a version of Google without ads? Probably not.
Google already does this with YT; YouTube Premium[1] offers you YT experience without ads plus extra perks. And YouTube Premium has apparently more than 25 million subscribers in US alone[2] and 80 million subscribers globally[3]. My thinking is power users are ready to pay for ad-free experience and casual users probably not because they are not heavy users.
>But that's just me - I actually like to know who is advertising for particular searches. A company with an ad budget to rank at the top of ads is probably more trustworthy than an anonymous website.
A lot of spammers and fraudsters see advertising as their most effective gateway and tactic to scamming people so I wouldn't count on reliability and safety of all ads that you see.
YT Premium is a reaction to the infeasibility of subsidizing the extreme cost of hosting video streaming with ads alone. The scale of ads needed to cover hours of streaming video vs. ads to cover search results are magnitudes in difference, and people are more willing to pay to make those streaming ads disappear.
Even so, with a paid option, the vast majority of YT users still use Youtube w/o the Premium subscription.
I find YT premium fully worth it; the ads are horrible, and totally inturrupt watching something - enough that I pay, as I couldn't stand it otherwise.
Google, less so - but I pay for Kagi as I like the results better, and prefer how they say they're tracking me.
I'm paying for YouTube Premium and you get YouTube Music for free + ad-free too. I've started paying because ads were annoying on my smart TV YT app and I already used YT Music.
YouTube ads are much much worse interstitial style ads that won't let you see the video until you watch an ad. That's why Netflix does well too. I don't think it's a relevant comparison.
Why would you create a search engine that looks exactly like every other search engine? And further, specifically Google.
Why would you not take the chance to completely reimage search into something way better than the absolute random crap we have today?
Whatever the next web discovery engine looks like, it'll look and feel entirely different.
It won't be the equivalent of someone saying, "Hey Facebook is bad, look at MY social network!" and it being the exact same layout and user experience as Facebook, with different margins, padding, fonts and colors.
Whatever comes next, it'll be like comparing Twitter to Facebook, or TikTok to Instagram.
At this stage, the chief complaint of most prospective users is the quality of search results from the market leader, rather than the layout or user experience. It's a 5-person company, and I'd rather they focus their efforts on the search algorithm.
Simply because you need to get the basing things right like the correct infrastructure and it is a safer bet, specially when you are bootstrapped. The article lays out the vision for the future of the search that Kagi wants to build.
The OP over-reaches in that PageRank is just a small ingredient in Google's ranking, which covers many other components from TFIDF score to anchor text features and yes, PageRank. Naturally, Google used PageRank as a differentiator
in its marketing, but technically. Bing has been known to use >150 features to score the relevance of a page, given a query, so one would expect Google to exploit a similar order of magnitude of evidence in its ranking.
> Yet, despite being acutely aware of the dangers of ad-supported search, selling ads was adopted as the primary business model of the new search venture just a few years later.
Google was successful as a superior search engine, pushing out AltaVista and similar, earlier alternative search engines and Yahoo!-style portals curated by humans. However, Google wasn't a successful business for quite some time.
Eventually, they adopted (some people would say "stole" - there was a lawsuit) Yahoo!-owned Overture's ad model, which changed everything. Yahoo! owned Overture, and Overture had a critical patent. Yahoo! made one critical mistake: they settled the lawsuit for relatively little money. The rest is history.
Now many people complain about decaying search result quality levels. That just means there is space for a new search engine, how exciting! The good news is it has never been easier and cheaper to start a full-text index of the Web and associated search. For about 50k (a Xoogler's estimate, not mine) you should be able to get going. Sites like Gigablast show it can even be done as a one-man show, which I would not recommend (to many complexities in "small" bits even HTML to plan text conversion, load balancing, incremental inverted index updating etc. - all requiring nowadays some specialist expertise in a game where you can't afford to reinvent the wheel because you don't know the scientific literature/state of the art). The one thing that is hard to get is initial user traffic. But I think HNers will be happy to give each new engine a try!
In summary, I think there never was an "age of PageRank". But you may say Google Web search is past its prime. Perhaps Google could change that if they wanted - it may be that it isn't much of a priority at the moment, hard to say (they are (too?) big now).
Thanks for posting the interview with Shadi Saleh. I found it interesting they were able to mitigate sanctions by using FOSS software. I wish Saleh had gone into more depth on the hardware side of things which I imagine is also complicated by sanctions.
379 comments
[ 5.6 ms ] story [ 297 ms ] threadTime to join a web ring.
[0] https://en.wikipedia.org/wiki/LinkExchange
I think the biggest issue is live news but I mean then just pay like 12 people to watch news/sports networks and you won't be that far behind. (Or just have an section of the index for live information and let users to go nfl/espn/oan/cnn).
It’s mostly to find an image, a video, a location on maps, etc.
I wonder if Google could turn things around by rethinking the search experience. They’re still the fastest first stop on your journey to find something on the web. But the behavior has changed dramatically that a change in their UX alone could really make a big impact. Maybe moreso than just an algorithm change.
I once redesigned the search experience of a large global ecommerce company. The UX changes alone grew their revenue quite a substantially and reduced customer complaints.
PageRank was never designed for adversarial scenarios.
It reminds me of KPIs like lines of code - it's only useful if it cannot be manipulated.
"When a measure becomes a target, it ceases to be a good measure."
https://en.wikipedia.org/wiki/Goodhart%27s_law
When there’s so much money at stake, people will go to great lengths to reverse-engineer the things that put them higher in the page, no matter how you try to hide the levers that make the rankings work.
That just turns it into a statistical problem.
I don't think you can escape the problem of a search engine being used as an oracle.
Instead, you want to throw queries and user behavior into a blackbox algo and have it tell you a result then give it feedback on whether the result was good (did the user come back and ask the same question? did they click the top result and leave or did they have to come back and click through many more?). I think this is kind of how google works now, though results are frequently meh. Millions of backlinks will get you noticed but your ranking will just keep dropping if users don't appear to find your content useful (e.g. they hit the back button a lot and keep searching).
What stops me from writing a bot to simulate that behaviour for my website (or websites of my competitors) ?
One form of personalization that they applied early on is move results you click on a lot up in your results you can’t trust the serps you see at all.
They also inject a lot of arbitrariness and randomness to make it impossible to make small changes to your site to incrementally improve it the way you would improve an advertising campaign. One reason the web seems so frozen is that if you have a successful site and make major changes to the layout, titles, link structure, etc. you will possibly trigger the ‘chaos monkey’ and wreck your ranking and it may never recover.
The real consequence PageRank had was that it got web pages to stop linking to each other…. Google gaslighted people into removing the competition they could have had navigating from one page to another through links. (e.g. web directories)
I think the problem may more have been the lack of a sustainable revenue model. Getting volunteers to curate the directory is particularly destructive because the people who most want to volunteer either (1) want to promote something or (2) want to get paid so somebody else can promote something.
That's what search indexes are. Automatically curated directories.
It's just that the UI to the index is natural language queries instead of clicks through a graph, because the latter isn't scalable to large corpuses.
There are certain sites which have a landing page for every topic in some set of topics, you could add a lot of links quickly if you built rules for importing links from particular sites. Adding 10 sites a day with 1000 links each would be very possible, in 100 days you could build out a million links.
Google was caught flat footed and wound up buying and killing Freebase in order to catch up. They lied and said they rejected "semantic web" approaches despite hiring one of the leaders of the Cyc project as their head of research.
Still there is a difference between exposing that kind of database through a full text index vs exposing it through a browsing interface.
That all seems pretty quaint these days, but even worms and viruses at the time were versions of "my goodness, who would ever write a script that emails itself to your address book, or copies itself across the network to all the other unsecured PCs with passwordless full hard drive access?"
More and more I find myself searching through moderated social media like Reddit or Facebook Groups/Marketplace for product recommendations or local services.
Gave Kagi a go, but it seemed to be even worse than Google for what I tried.
so many of the complaints about google search results seem to be along the lines of "i asked google what X i should purchase, and they served me an ad", whether that's a first-party ad on google or search results that are external ads. If you're only attempting to use a search engine to find facts, it's a much more satisfactory experience.
search results for opinions are and have always been trash - they don't serve you the best opinion, or the most trustworthy, or the most well-reasoned. they serve you the opinion most relevant to your search terms. and that's probably not what you want.
And moderated groups don't sound like a solution to the 'I have a concrete question that I want an answer for right now' problem either (unless you're thinking of massive live chats with many lurkers, which sounds more like the web of the past, tbh).
It's often pretty fustrating to try and find something you know you've seen before but don't remember quite the name of or exact keywords.
Ideal would probably be face/eye tracking to check attention/response but thats way too creepy to work.
[0] https://search.brave.com/
There are so many ways to use Google to obtain the information you're looking for, and so many people do it literally on a minute-to-minute basis that it's flat absurd to call that ineffective.
It's trivially easy to learn something by typing a question into Google; maybe when writing an article try gut checking it against obvious observed reality first.
I simply cannot fathom a noble reason why the author would decide to publish this.
Edit: Oh wait I found it; the author is Vladimir Prelovac, CEO, Kagi Inc. What does Kagi Inc. do? I'll leave that as an exercise to the reader, but I bet you guess it in one try...
Also it's written by a CEO of a competing search company. It's literally the very SEO he's railing against...
Point taken. But it's not as though this was concealed. The top line of the page is "Tales from Kagi." The piece is hosted on blog.kagi.com. The author's signature at the end of the piece reads "CEO, Kagi Inc." The final line of the post is "I hope you join us [i.e., Kagi] on this journey."
Maybe it doesn't bother you, just thought it was worth calling out explicitly.
It is marketing and trying to influence the discourse, but I don't see how you can call it SEO.
It's the whole purpose behind all of those random blog articles on corporate sites; to improve PageRank ranking.
Google is a horrible search engine, you routinely have to hack it to get searches not linking to the centralised web. The first 5 results are ads, followed by Youtube links or a Quora/Reddit on the topic.
If you used the web 20 years ago you know what a good search engine was like.
Especially if you're looking for Google users' private information, and you're a business partner of Google's.
Then you can check google or a mainstream engine when needed.
I suspect that was what you were seeing, and if so, updating should have addressed it.
What really excites me is that is that I'm paying them. That sounds odd, but seriously. It's incredibly refreshing to know that the company providing my search results has an incentive to make things better for _me_ and not a legion of advertisers. With Google I can't help thinking about every keypress being logged to optimize sales pitches at me. I just don't feel that with Kagi, because I'm paying them.
Sure, they might be logging every keypress (I don't actually think they are, but you never can tell) but even if they were, I could be reasonably certain they were doing it to retain my subscription, which probably means making my search better, not selling me other stuff.
It's a priveleged position to be in, and the economic argument isn't watertight, but in the "search as a brain extension" space it still _feels_ premium, because it creates trust. And that frees up brain space for other things - like where the hell was that article I was looking for?
Afaik [1], there's about 5 employees and the revenue only covers server expenses while they're still trying to get more headcount. Not an expert on bootstrapping but I'm pretty sure you don't want to expand faster than your revenue does otherwise you stop being able to make all the decisions for the company.
[1]: https://blog.kagi.com/status-update-first-three-months
EDIT: Their website says that they don’t plan to take VC money, so I have no idea what a SAFE means in this case. The point of a SAFE is that the investor is guaranteed the best terms when a priced round of investment happens.
SAFEs are often capped, so taking awhile doesn't matter so much, except maybe interest calcs
Need more money? Economic model not quite working? Companies face this all the time. Some solve it innovatively, survive, and show everyone else the way forward. Others fold or pivot.
But ad revenue short-circuits the hard process of economic innovation by tempting tech companies to take an easy way out, even when the problem is solvable without it.
(All of this wouldn't really be a problem if the ads business model weren't inherently adversarial against users.)
Yeah, I should use a hacked rooted Android yadda yadda. But I just spent a week hacking and hawking for my life in an ICU, and I certainly didn’t want sanity-saving podcasts to shut down whilst on an hour-long stay at the breathing machine. Life is short.
I agree that on the whole, people will continue to go for the free option, but I'm just glad that some options are appearing to pay, which rebalances the incentives a bit.
I think another factor is the paradox that the user who has enough money to be willing to pay to not see another ad is exactly the most valuable user to advertisers— they've pre-qualified themselves on multiple axes.
So there is always going to be an enormous moral hazard here with the ad-driven-freemium model— always a temptation to run an occasional high-value ad to the paying users, or to disguise the ad as a "recommendation" or position it instead as a "sponsorship", or whatever else. Other than users reacting swiftly and decisively against this kind of thing, it's hard to know how else to keep companies away from it.
Of course this depends on how the supply demand curve looks like for their specific business.
An angry bird informed me about this :)
Though in this case, it's likely that Twitter has repeatedly run internal studies about what a paid-tier would look like, and each time concluded exactly what the GP is passing on: that it would be a net loss because the most-valuable-to-advertiser accounts would be the first ones to pay.
Even just psychologically, it would be a major blow to their sales pitch, going from "pay $X to put your name in front of the top movers and shakers in the world" to "pay $Y to put your name in front of all the Twitter users too cheap to cough up $8/mo to not have to see you." Not hard to see that $Y is going to be less than $X... probably a lot less.
The reason they advertise is because they can. It doesn't matter how much money they're making now, they can always make more by advertising. Paying money just makes your attention more valuable. The only way to stop them is to make it impossible or illegal to advertise. Blocking ads everywhere and making no exceptions.
* one does not need $200-300B ARR to run a search engine; and merely making a ton of money does not mean that the way that one has been making that money will continue forever
* using Kagi, one realizes how much the web has been perverted, in my opinion, by the obsession with the ad model; there are many nice, novel things that Kagi could introduce which would reduce ad revenue for a company like Google, but would be useful and novel for Kagi without loss in revenue (on the contrary — could help); ie google is incentivized to keep people on the site through various irritating means; a search engine that isn’t distracting and gives me what I want — I would be more than glad to pay for (and already do pay Kagi $10/month)
Other thoughts:
* Supposing $10/month (honestly I would consider $20) with 100M paying users; that is $1B MRR which is not bad at all and is more than plenty for a meaningfully sized team with meaningful salaries.
* Just like how lowered energy prices during and after the Industrial Revolution made manufacturing at scale feasible, the incredible amount of high quality OSS and infrastructure these days is making it increasingly feasible to do things Google did 20 years ago — something unthinkable for even the best engineers of that era. Not to mention the relative ease of collecting capital with payment services (let alone VCs, etc.) today — even with a looming recession factored in.
That would actually be $12B ARR, since that price is per month while ARR is usually per year.
35 dollar is maybe ok… …. Maybe better than spending 3500$ on … I dont know … stuff is good also ..
In a sense, that model is better at distributing income, because everyone gets a service and the people who can afford pay for everything. If everything was behind paywalls, people who can afford would have all the nice services and people who can't, will have nothing.
What's interesting is that "targeted ads" is a bit of a misnomer. It's more of an ads ranking, where the user is shown the ads they are most likely to buy. However, from the point of view of the car dealership, when they think targeted ads, they would like their ads to be shown only to people who will likely buy the car. No company would want their ads shown to someone with no disposable income, yet, they will be shown some ad.
US attention is the most valuable because it is the richest large nation and the largest rich nation.
European attention comes a distant second. Together they have 40% of the global GDP, but likely something like 60-70% of the consumption of non-essential goods in nominal terms. Accordingly, average advertiser cost per clock differs ~10x between the cheapest and most expensive groups of countries.
They have a full business suite, App Store, hardware sales, fiber internet, wireless cell and internet, and a server rental division.
Yep, but the main problem here is the perverse pressure to grow indefinitely.
And that comes from it being acceptable for public companies to pay no dividends and instead rely on share price increases to deliver a return on investment.
Something that slows down this growth spiral would fix things partially. Require profitable companies to pay a % of the profit as dividends to their shareholders?
Now the issues is not just privacy but making sure it does NOT optimize search results based on prior searches or other facts it knows about me.
Now the big problem is that one would think delivering the exact search results that I am looking for is a good thing. No. It is super dangerous. It can causes me to live in a bubble, blinding me of possible opposing opinions. It is feeding my ignorance.
So there is a goal conflict here. I don't know how to solve it. I wish there would be something like Wikipedia for search engines but even the wiki models has its problems and biases.
On the other hand, a lot of the search results bias stems from the search query itself. "Does X cause Y" for example will show results that echo the search, and not show opposite results. Simply because you did not ask the opposite or a more open/unbiased query like "X Y".
This way you get anonimity and custom results
Yes, the server has access to the information the client has, but it doesn't have any way to confirm that's true. It also could avoid storing that information all together if the client sends it again every time
On the point about payments, there are some potential approaches to handling this, but they tend to be a bit complex and technical. I"m not sur ethis is really a technical problem though (or at least one where a technical solution is what's needed).
A challenge of taking traditional payments is that the identity of the payer is inherently "linked" (to some extent) to the transaction. You could maybe use a privacy.com type intermediary or similar.
What you could do however, is buy a "kagi voucher" or similar, which would be an RSA-blind-signed token that you generated client-side in JS. You'd send it to the server during the payment transaction, and be given back a signed version of the token, which you unblind client-side, then give a backup copy to the user to save, and set it as a cookie to authenticate payment has been made.
That should, in theory, at least un-link the payment from the user. Ultimately though, a bad actor can still trivially correlate your searches (and payment, if you used same IP or browser fingerprint), and all your searches are linked by a common cookie.
So you decide to create some kind of token fountain, which you can auth to with a blind-signed token, and get 50 "search passes" from. Those could be unique per-search, and just be "bearer tokens" you can use. You'd have to trust that this token fountain doesn't record which search passes were issued to which "blind token".
Ultimately, it feels like this problem then becomes "do you trust your search engine?" - I'm not sure this is a problem that technology needs to solve, as ultimately your search provider is seeing your queries and IP, and most users won't realistically change this. A provider that isn't incentized to act against your interests (i.e. something like Kagi which isn't advertising to you or selling data) seems to address this for most user scenarios.
We already know that Google is in fact cooperating with these entities. Not only by giving out user data but also by actively censoring certain things. So there is a precedent.
This made internet completely unusable for discovery. A big shiny window into another tiny room. Absolute BS.
Recently I discussed it here on HN and was advised to like, dislike, subscribe and so on on youtube (things I rarely have done on my old account) to break out of this bubble. Well, it doesn’t work. It just suggests the content I immediately dislike or ban entire channels and it all settles to the same echo chamber as before.
Same for google search, really. It turned from “I’ll find you that if it exists” into “Look, if you change your query to ‘cool products if at all barely relevant to …’, I have literally billions of results for that”.
I'm sold, literally.
Cable television and netflix have made it quite clear that payment does not mean “no ads”, it just means “no ads, yet”.
It still might be a better alternative in the short term, but the moment growth starts to peak, companies follow where the incentives lead them.
The agreement could be carefully written by a skilled lawyer to define the things Kagi cannot do, the proof customers must present in order to proceed with a valid lawsuit, and even the maximum damages that the customer may sue for.
In that case, if Kagi was found at some point to be using customer data for these purposes, it could be sued very easily and by many parties.
People are calling for regulation for data privacy. In the meantime, Kagi can create its own regulation it will hold itself to for the benefit of its users, can it not?
I think what's more interesting/concerning/insidious is hidden ad content like product placement. It's getting to the point where personalized product placement could be embedded in shows dynamically.
All major content providers left them years ago and started their own Netflixes. So they're left with a few movies from ten years ago, a couple of recent-ish releases, and their own content.
In the streaming space in the US there is: Netflix, Disney+, AppleTV, Amazon Prime Video, HBO/Discovery, Paramount Plus, Peacock, STARZ, and a few other players. There is no “opoly” in streaming video.
They certainly have the ability to uniformly raise prices (tacit collusion) with no viable competition to enter the market and fill the gap (as even the vast sums others have thrown into it have shown how hard it is to produce good original content).
This is probably due to the characteristic that producing goods (decent original content) in this market is a big barrier to entry - which naturally leads to a small number of players. Natural monopolies and oligopolies are common - but do require closer regulatory attention to ensure desirable consumer outcomes than just letting the free market decide.
It may not entirely fit all definitions, but the general economic theory and applications/implications are relevant to consider this market.
The original post that you questioned was related to vertical integration - you could effectively find and replace it to “so you’re saying producers of operating systems shouldn’t be able to make their own web browser?”
The market is “content”. Netflix competes with YouTube content producers, TikTok and it even said that one of its biggest competitors is Fortnite.
> This is probably due to the characteristic that producing goods (decent original content)
This goes back to YouTube. You and I may not think that YouTubers and TikTokkers are producing “decent original content”. But there is a generation that spends hours on both.
Besides that, there were over 550 original series being produced last year (https://collider.com/too-many-tv-shows-550-series-2021/). Competition is much fiercer for your attention than it was when you only had the three major networks producing content and everyone else buying rights to show reruns.
There are bidding wars between all of the streaming services for new content from producers. Competition is more fierce than ever.
The price of streaming before was never sustainable. Netflix was borrowing billions a year for years to produce and obtain content. Disney+ was never going to be profitable selling its service at the introductory price. It’s not “collusion”. Every company has to turn a profit eventually.
Yes I realize that Netflix was “profitable” by GAAP standards. But it was getting deeper in debt every year.
It competes with these for screen time, not for content.
Meanwhile Disney owns how much content (movies, series and related IP)?
And the value of IP without execution capability is overrated. Warner also has iconic content. But Disney has been able to make successful movies out of its 3rd tier IP while Warner has struggled with its first tier.
Sony isn’t doing too well with most of its Marvel content except Spider-Man and that’s produced by Disney.
I think time has shown MS was correct in making their own browser, but of course incorrect in all the corrupt tactics they used to make their browser succeed over Netscape's.
IE was a much better browser. Especially the Mac version that when it was introduced, was the most CSS compliant browser that existed.
Netscape starting over from scratch was cited as “Things you should never do” in an article written by Joel Spolsky (StackOverflow cofounder) two decades ago.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
So far. I imagine they will be expanding it over time.
With the exception of filler ads for their own content that occurs when they need to wait for the next quarter time, ex movie ended at 12:55pm. There have been no ads
To say nothing of the fact that Hulu has maintained an "ad" and "ad-free" tier for their original programming.
I am paying and have no way to disable that visual pollution nonsense.
Extremely exasperating when you are trying to choose something adequate for toddlers and they keep seeing flashy stuff on the screen and saying they want to watch that.
Take Spotify as another example. I pay them, along with 188 million other people. Does that mean they won't turn around and ask artists, record companies and rights-owning conglomerates for money (or rebates) for putting their stuff in front of me? Of course not. Paying them means they have some interest in pleasing me, but it's far from the only way.
On the off chance that Spotify was being scrupulous about not taking payola in any form, it would be impossible for me to verify. Which is in itself a reason for them to cheat; they don't get much economic benefit from goodwill if no one can actually see them being honest.
This is not a reason to not use Kagi, it's just a reminder of what forces we're up against. Kagi will need an unusual amount of transparency in everything they do, in order to stand a chance in the long term.
And it IS a reason to not get warm fuzzy feelings merely from the fact that you're paying them.
However, for sure that will get out eventually.
If that happens, I would then look at other search engines as well.
Most people wouldn't once they get used to doing it one way, so that's all to the good for Kagi. But people like me wouldn't then use it, which is what life is about - your own personal choices.
I prefer to pay for services because at least there is a slight chance that they will follow their business plan.
But nothing stays the same for forever. As a consumer, we must also change to the changing environment. I'm completely weened off Google, for quite a while now, for example. I'm paying for everything that I use. But it is so little as to be laughable. For example, I use tutanota for my email, first account is free and subsequent ones are $1/month. Big whoop. $12 per year for private emails. I have a number of emails through them because I segregate all my emails - one for just friends, one for business, one for education, etc.
2. Create a new plan at the price of the old plan. With ads.
I'm sorry, but pretending that they're not adding ads to their "regular" pricing plan is semantic at best. This isn't something they dreamed up overnight. They've been planning this for years and increasing their prices accordingly.
for now....
No VC startup would do that but if possible would allow trust.
Complaining that cable TV isn’t ad free because you pay for it is like saying all content over the internet isn’t Ad free because you pay for your internet connection.
'It's better to be a millionaire hero than a billionaire asshole' (Ben Elton, Gridlock)
Of course at $10/month even a 1% share of the search engine market would make the company worth billions.
If that is the only metric, the best business model is to extract as much money out of your user, while simultaneously offering less for the same amount of money, until you reach the asymptotic ideal of 100% profit.
If you were to start a company, would you be willing to do that? Would you choose profit over everything else? Some would, but not everybody.
Companies have an incentive to make money off of their customers' data, and technology just means that they can make money at every level (your ISP, your smart TV, the apps you use on it and the websites your browse).
We're so accustomed to entities gathering data about us that it's become part of the background.
Why is it the default that my home is on Google Maps/Google Earth? I should have to opt in, not opt out.
Just because there is "public" information about me as a registered voter or home owner doesn't mean that anyone should be able get a copy of this data, put it online and connect it to other data?
[0]: https://teclis.com/
Google search only shows paid SaaS platforms in the first page of results.
----
I have an idea (more here[1]) that the next addition to our civilization should be a distributed system to evaluate, reward and compensate externalities. Open source software is a great example. It's free, it has massive benefits to users and for people reusing it for all sorts of projects, it's public. Yet we still can't find a great way to fund it. There are donations, and maybe that's all we need in some sufficiently enlightened society, but I think we should be addressing those more directly. And clearly there are technologies which generate friction and other social costs, that could be (differentially) quantified and prices. The same goes for pollution and a miriad of issues.
Laws can help here, but laws tend to lack softness and specificity in my opinion, and laws are strictly punitive. If you're generating a negative externality, and the legal system is well designed, maybe you'll have to compensante for it. But that relies on quick lawmaking and judges evaluating something they're not specialists at. And there's no dual for punishment (actually compensation and reeducation), there's no intrinsic reward system. There are prizes and grants, and I'm sure they generate immense value for what they are, but they're still very unsystematic and unreliable, I think.
I dream of a system where if you generate value for society, you will be recognized and make a comfortable living off it; if you need investment, you will also be supported. (I call this idea Elementalism; I'm not sure it's original)
I think we need to be a little more open to careful, thoughtful changes to our social-economic system that can improve things.
While Elementalism isn't a formal part of our system, each of us can do our part and give directly to what we understand needs the most. This is the core idea of Effective Altruism which I also extend to donating to Open Source (which I believe can be effective[2])
[1] https://news.ycombinator.com/item?id=29043752
[2] https://www.reddit.com/r/EffectiveAltruism/comments/v7ma0d/w...
240 searches cost them over $10.00. They will need to find additional revenue or reduce costs to even reach a break even point.
They will have to raise prices or make you pay per search or sell your data or better yet all three.
If their paying user base grows by 10x, do the searches drop from 4¢ each to 2¢ each?
.0125 per search. The teams rate is twice that.
I'm a user now, and potentially a customer if I hit their free limit.
Is there some zero-knowledge protocol that could help here? In particular, to establish that a user is currently subscribed, without revealing which subscription they have?
E.g. instead of training it to generate sentences given a prompt, train it to generate URLs given a query (?)
Plenty of excellent youtube channels that 10 years ago would've been websites.
I don't really have a problem with this, all things change.
Compare written instructions for e.g. fixing a car vs a video of a person showing you how to do it. Latter is much more helpful
You can mitigate it with curation, but then you’re bound to niches and bias, which is true with automation as well. No victory here.
https://patents.google.com/patent/US6285999B1/en
EDIT: I'm dumb. Please do not reply or I will Sylvia Plath myself.
Actually Google is already doing this. Results are personalized and contextual. It can't really know what I feel like eating this evening because I don't, but it can guess.
I applaud the author for trying, but I don't see an alternative to PageRank being proposed. How exactly is Kagi proposing to rank results?
And no, I don't want an AI generated summary when I search for the best tutorial to do X. I want a list of tutorials. The question is how to rank that list, and I've yet to see anyone do a better job than Google.
Google is far more than PageRank. Its an AI ranking model that has the largest training dataset (queries and clicks).
Ads may suck, but simply charging for the same service isn't really innovative. Would I pay for a version of Google without ads? Probably not. But that's just me - I actually like to know who is advertising for particular searches. A company with an ad budget to rank at the top of ads is probably more trustworthy than an anonymous website.
Google already does this with YT; YouTube Premium[1] offers you YT experience without ads plus extra perks. And YouTube Premium has apparently more than 25 million subscribers in US alone[2] and 80 million subscribers globally[3]. My thinking is power users are ready to pay for ad-free experience and casual users probably not because they are not heavy users.
>But that's just me - I actually like to know who is advertising for particular searches. A company with an ad budget to rank at the top of ads is probably more trustworthy than an anonymous website.
A lot of spammers and fraudsters see advertising as their most effective gateway and tactic to scamming people so I wouldn't count on reliability and safety of all ads that you see.
[1] https://www.youtube.com/premium
[2] https://www.statista.com/statistics/1261865/youtube-premium-...
[3] https://blog.youtube/news-and-events/youtube-music-premium-8...
YT Premium is a reaction to the infeasibility of subsidizing the extreme cost of hosting video streaming with ads alone. The scale of ads needed to cover hours of streaming video vs. ads to cover search results are magnitudes in difference, and people are more willing to pay to make those streaming ads disappear.
Even so, with a paid option, the vast majority of YT users still use Youtube w/o the Premium subscription.
Google, less so - but I pay for Kagi as I like the results better, and prefer how they say they're tracking me.
That doesn't necessarily make it better for HN users. A more selective dataset could make it more useful to specific niches, like say, HN users?
Why would you not take the chance to completely reimage search into something way better than the absolute random crap we have today?
Whatever the next web discovery engine looks like, it'll look and feel entirely different.
It won't be the equivalent of someone saying, "Hey Facebook is bad, look at MY social network!" and it being the exact same layout and user experience as Facebook, with different margins, padding, fonts and colors.
Whatever comes next, it'll be like comparing Twitter to Facebook, or TikTok to Instagram.
As the kids used to say, this ain't it, chief.
> Yet, despite being acutely aware of the dangers of ad-supported search, selling ads was adopted as the primary business model of the new search venture just a few years later.
Google was successful as a superior search engine, pushing out AltaVista and similar, earlier alternative search engines and Yahoo!-style portals curated by humans. However, Google wasn't a successful business for quite some time. Eventually, they adopted (some people would say "stole" - there was a lawsuit) Yahoo!-owned Overture's ad model, which changed everything. Yahoo! owned Overture, and Overture had a critical patent. Yahoo! made one critical mistake: they settled the lawsuit for relatively little money. The rest is history.
Now many people complain about decaying search result quality levels. That just means there is space for a new search engine, how exciting! The good news is it has never been easier and cheaper to start a full-text index of the Web and associated search. For about 50k (a Xoogler's estimate, not mine) you should be able to get going. Sites like Gigablast show it can even be done as a one-man show, which I would not recommend (to many complexities in "small" bits even HTML to plan text conversion, load balancing, incremental inverted index updating etc. - all requiring nowadays some specialist expertise in a game where you can't afford to reinvent the wheel because you don't know the scientific literature/state of the art). The one thing that is hard to get is initial user traffic. But I think HNers will be happy to give each new engine a try!
In summary, I think there never was an "age of PageRank". But you may say Google Web search is past its prime. Perhaps Google could change that if they wanted - it may be that it isn't much of a priority at the moment, hard to say (they are (too?) big now).
Edit: Here, I've interviewed Shadi Saleh, the architect of Syria's search engine (if you think it's impossible to get up and running with a small team): https://irsg.bcs.org/informer/2019/07/syrias-first-web-searc...