147 comments

[ 0.22 ms ] story [ 207 ms ] thread
For as well-funded startup as Neeva with long term Google Search execs at helm, they seem to have their heads turned at every passing fad. Not long ago were they raising money for their spin-off web3 search engine n.xyz and now this (admittedly, AI is much closer to their core biz). Wouldn't you be surprised if they entered the RISC V market, too?
I think it more closely speaks to how fast a startup in this space can move.
Alternatively, it speaks to a lack of focus and lack of will to stick with a product and polish it unless it produces massive returns immediately
On the other hand, if you're going to disrupt a market like search, isn't there a good chance that you'll do so by more quickly embracing a new technology than your large, slow-moving competitors? And with the need to move quickly and stay ahead of the curve, doesn't it make sense to take some risks on using technology whose benefits maybe aren't fully clear yet?
(co founder) We have worked on neeva search for four years and have really sweated building out a real system with a massive crawl, index and serving systems. We see AI as a way to get people to answers faster and provide an overall better experience. We tried hard to tune neevaAI to make sure that it doesn't hallucinate. And to be very clear, we are using LLMs to help distill information from sites and aren't pretending to be the source of truth. Hence the citations for each and every sentence.
Is it fair for publishers to scrape their info and then provide it so users don’t even give them traffic?
How are you tuning it to make sure it doesn’t hallucinate? I think that’s a fundamental property of neural networks (see, for instance, the adversarial examples literature, or the recent work demonstrating adversarial exploits in AlphaZero style agents in Go).
Interesting.

I've always viewed there as being a sort of inherent contradiction between chat search and document search. You can do either one well, but not both.

Although as far as I understand this approach of yours seems to still suffer from having to identify the most reliable source(s) to interrogate first... which is essentially The Problem™ of search and if you can do that well enough to make a conversational search engine give good answers, then you already do search way better than Google's ever done.

To be clear, my comment wasn't a critique - if you're an early-stage startup, it makes sense to try new things and attempt to catch new trends early, since they can carry you past your established competitors.

I think the search engine with citations is a great idea - you're grabbing onto a big trend while solving a clear, hugely-discussed problem (you can't trust LLMs to give truthful answers, and that's exacerbated by the fact that you can't see where their info is coming from). Hope it works out for you.

> To be clear, my comment wasn't a critique...

GP is responding to other naysayers without directly replying to them (:

And to be clear, I think going down the LLM rabbit-hole is absolutely the thing for them to do (see the quip about AI being their "core biz"), but you wonder why it wasn't them that came up with ChatGPT-like system in the first place, if disrupting is what's Neeva is about? Though, I don't think Neeva is out to disrupt Google Search when the differentiator is merely a different business model; I mean, per definition of the term as Christensen meant it [0], it is pretty hard to disrupt free.

[0] What is disruptive innovation (2016), https://archive.is/IzaKy (hbs.org)

AI, sure. Web3? Just makes anyone with a real brain laugh lol.
(comment deleted)
Early in 2022, the rumour was Sand Hill Road essentially asked its startup portfolio to invest in Crypto.

iirc Neeva wasn't doing too flash back then.

Now, Neeva's playing in its own ballpark.

(comment deleted)
(comment deleted)
"What's the half-life of a proton?"

NeevaAI:

>The half life of a proton is at least 1.67x1034 years. Recent developments of grand unification models have suggested that it might decay with a half-life of about 1032 years.

Note that the lack of exponents are reproduced from the original. Neeva says 1032 years, not 10^32 years. The linked source for 1032 displays the 32 as a proper exponent, but Neeva seems to strip this critical formatting.

"Which is heavier, a kilogram of feathers or a kilogram of steel?"

NeevaAI:

A kilogram of steel is heavier than a kilogram of feathers. This is because any volume of steel is heavier than the same volume of feathers. Additionally, the weight of the feathers must also be taken into account when considering the weight of the feathers.

"Who is the reigning monarch of England?"

NeevaAI:

The reigning monarch of England is Queen Elizabeth II. She has been on the throne since 1952 and is the longest-reigning British monarch in history.

same for current president of Colombia

NeevaAI:

The current president of Colombia is Iván Duque Márquez.1 He was elected on June 17, 2018 and is serving a four-year term from 2018-2022.1 He is the 14th president of Colombia since the ratification of the Constitution of 1819.

Google has the same answer tbf
Google answers "who is the queen of England?" with the reponse "Charles III". Close, no cigar.
This is the power of AI in search - giving you bad facts because the provider is too lazy to scrape them from the appropriate source.
What is the appropriate source for this fact? You make it seem like there's a

$ curl https://www.world-monarchs.info/current/britain/

they've forgotten to run

www.royal.uk sounds like a good place to start.

Same as how whitehouse.gov will give you the US president, and even tiny nations have [head of state].[country] for that information.

(co-founder here) We are working on a bunch of freshness improvements across the board (ranking, indexing, crawl) that should help with this. Thanks for the feedback and keep it coming!
>A kilogram of steel is heavier than a kilogram of feathers

Isn't this technically true if you do the measuring under Earth's atmosphere where you have to account for buoyant forces? Granted the difference is minuscule, and it assumes by "heavier" you mean "apparent weight".

It’s also technically true if you ask about an ounce of gold vs. an ounce of feathers (gold is heavier) because gold ounces are different from feather ounces.
It is not technically true. For one, the question didn't say "in Earth's atmosphere." But more importantly, if you get a kilogram of steel and a kilogram of feathers and put them on a consumer-grade scale on the planet Earth at standard temperature and pressure, they'll both show the same reading. If you are a person that needs a specific enough scientific answer to account for any ridiculously minuscule effect caused by the environment, you aren't asking it in such an off-handed way.

If I asked "does ice float?" and got the answer "Ice floats on liquid water." that would be satisfactory. If I got the answer "Ice floats on liquid water if the ice is composed of hydrogen-1 atoms, or if it is placed in water composed of the same (or heavier) hydrogen isotopes." I would be dissatisfied because that's a confusing answer that nobody is reasonably looking for.

ChatGPT gets this one right.
Given it’s a fairly common word problem, I can’t say I’m surprised.
(co-founder) Thanks for reporting the rendering issue. We'll fix this.
A lot of systems based on gpt-like LLM's are currently being limited by the stripping of certain characters during tokenization... if the linked source used a superscript exponent, that superscript probably got lost and interpreted as just a concatenated number. This is one of the large problems I see right now hindering how these models are trained- they crawl text with numerical figures and graphs, charts, etc on webpages that cannot be interpreted as text, so the model learns something, but misses context or, in this bad case, loses a very key element of the mathematical notation. It's a hard problem, because it isn't like every webpage with facts and figures is written in perfectly formed LaTeX that could be interpreted. 1.67x10^34, 1.67*10^34, 1.67E34, 1.67*10**34 plus any number of mathjax, LaTeX, non-ASCII charsets, etc are all ways that would need to be interpreted correctly by the preprocessor for training, the input tokenization, and represented in the charset of the output of the model for it to be more effective.
It should add that this lifetime in the first part is assuming that proton is not stable. Proton is a stable particle in standard model. Also, 1.67×10^32 is not the estimated life-time for these theories. This is the lower limit constraint from the experimental search for this hypothetical proton decay. I can't say this is encouraging result.
ChatGTP Answer: The proton is a stable subatomic particle and does not decay, so it does not have a half-life.
It sure would nice to find it on the webpage. I got done trying to search for it after 5 minutes.
(comment deleted)
If only the search wasn't paywalled...
That's literally the point of the product... non ads based monetization
(co-founder) Actually, 1. you can just search on neeva.com w/o signing up for an account 2. If/when you sign up for an account, you can choose between our "free basic" (free) product and our "premium" (paid) product. The premium product includes access to a VPN and a password manager. All these options are ads-free and private. (To experience Neeva AI, you need an account for now.)
I've built some AI-powered software before, and the issue for a lot of the consumer-facing tools such as this one is that a few incorrect results voids the value of the whole system. I think you'd probably need >95% accuracy for the benefits to outweigh the cost, and looking at the comments here it's clear that that bar has not been cleared.

To put it simply: If you can't be sure that the results are accurate, how can you default to this search system?

This has actually been my issue with Neeva since before the AI thing. I love the idea of a search engine I can fund by paying directly instead of hoping Google can continue to make money off giving me stuff, but 95% of my Neeva searches result in going to Google anyways.

For a while my muscle memory when searching was to prefix with a "g" so why pay a middle man?

Have you tried Brave search? I actually found it to be good and you can use it for free or subscribe for an ad free version.
What powers Brave search?
Brave powers Brave Search.

Independent search Even supposedly “neutral” or “private” search engines rely on big tech for results. Brave is different. We deliver results based on our own built-from-scratch index. We’re beholden to no one.

Tbh I didn't realize they had a paid option until I looked after reading this comment. I'll probably try it out, thanks.
> I love the idea of a search engine I can fund by paying directly instead of hoping Google can continue to make money off giving me stuff

I have found Kagi to be a nearly 100% replacement for Google

I signed up for Kagi. I’m disappointed in all the login stuff.
You use the session link too, right? That way, once you log in once, you can set your browser to use it as the default search engine and avoid the need to ever log in again.
What do you mean, I just login once on web/mobile, set as default search engine and forget
I've been using Kagi for a few months and while it is better than Google/DDG in many aspects, it still inherits the problem of ignoring words in queries.

Also, the "International" region has a strong American bias. Google still handles localised search results significantly better than anything else. I'm still a paying customer of Kagi though and I hope they get a chance to work on these issues.

Kagi still relies on Google, Bing and others to provide search results. Google and Microsoft could, in theory, end their business by refusing to provide them the search API or increasing prices by 10x. I really hope Kagi or something like it succeeds in the long term, but I don't know how viable their model is going to be without removing dependencies on megacorps.
I haven't seen them before but trying them now. Thanks! Maybe I should try to do a write up these different options.
I have switched over to using you.com for search. Been using it for a few months now and I almost never have to go back to Google for general search.

The only kind of search that I consistently need to use google for is looking up the meaning of words, so searching "define something", for everything else it's been weeks between needing to use Google.

I agree 100%. They had good ideas like being able to prioritize/blacklist sites from results. But ultimately they just seem to be chasing every gimmick instead of focusing on building a good product.
Same for any desire based control - a subset of UI to me. Facial Expressions in VR, not desire based - doesn't have to be that accurate and is fun in a 'generally ok state.' However, all the EMG hand based control and EEG brain based control - oof, slow down the standard interaction by 200ms and the world loses its wonder.

This has already happened to chatGPT and me. It's made up a few methods that don't exist in Swift - and I've gone back to my normal documentation > stack overflow work style.

And yet, we somehow take this for granted in Google searches? Who says results in google searches are worth the pixels they’re beamed through?

I think the obfuscation layer of AI results is just too new for us. The same way making big purchases online felt too new a while ago. Once we get acclimated to AI results, it’s not going to feel untrustworthy.

(comment deleted)
Because google searches show results from many people, and so disregarding results (and people) is easy. This is ONE machine, in a take-it-or-leave-it scenario. You'll either like most of the results or leave because some didn't work for you.
So multiple AI responses and we pick the one we like best?
No, it's "PEBKAC", so we need multiple AI personalities even if they're connected to the same backend.
Ignoring all the flak Stephen Wolfram gets, this is something that Wolfram Alpha does much better. Asking `escape velocity on Enceladus` to ChatGPT, it gave correct formula and still wrong result. All the hard-work of generating computable verified results for the Wolfram Engine database seems invaluable in comparison.

The deeper fact that Stephen Wolfram talks about in comparing ChatGPT vs Symbolic AI is true too. A layer deeper from ChatGPT is just a bunch of floats. So it is a lot harder to debug why the system arrived at the wrong result. For symbolic computations there might be a way to resolve how the expressions were transformed by the compiler and where it went off track, a debugging in true sense.

It is my gut feeling that we might see a next AI winter in a decade where we will finally start combining symbolic and connectionist approaches.

Agreed. However, Google has a similar problem with SEO and sponsored results. I cant be sure anymore that I actually get what I was looking for. In a sense, thats the same problem.
Interesting. I know Google had legal issues in Australia(?) because they had a little box that would scrape an answer from a website without you having to go to the website. Content creators are clearly not going to be happy with this, but not citing sources leads to the same confidently incorrect answers people have pointed out in ChatGPT.

I wonder where this will go.

You're correct. The "News Media and Digital Platforms Mandatory Bargaining Code" requires search engines and social media sites to negotiate terms (including payment) when embedding (and more dangerously, linking to) news content.

(Act is here: https://www.legislation.gov.au/Details/C2021A00021 - sadly it's an amendment which makes it harder to read, but it includes all the pertinent details)

embedding i can understand due to lost revenue, but i really don't understand how linking to articles can possibly be anything but a boon for news outlets. how do news sites think people discover their content in the first place?! i don't know anyone of my generation who still subscribes to a single news outlet for their news.
Looks interesting, but at first sight, this summarises the first few Google results with relevant points.

That would immediately present limitations. Not all Google results are accurate or helpful, implying answers could be useless. Also, if a topic has no presence on Google, this bot cannot extract anymore information.

How is Neeva monetized? It is a paid search? I subscribe to another paid search that I've seen mention AI (what I assume means large language models) as an upcoming feature. Personally I see it as a gimmick (as others have mentioned it gives many wrong results, and I don't think rhd user experience is there, even if it gives correct results, I don't use search that way much and I already know how to answer questions).

My point being I don't value paying for it, and it's a shame to see development effort focused this way, as others have said, so follow a trend. There is already so much wrong with mainstream search that can be improved upon, adding a me-too AI feature really doesn't seem like a great use of time.

Otoh, maybe the calculus is that it will draw in new users who also are following the trend? I just hope real advances in website discovery and business model do not take a back seat to this

> How is Neeva monetized?

> Neeva is 100% ad-free and never sells or shares members' data with any third party. Neeva's Free Basic membership is completely free and full-featured, allowing members to search both the web and connected personal accounts like Gmail or Dropbox (subject to certain usage limits), set news and shopping preferences, and search from multiple devices.

> To earn money, Neeva offers a Premium membership tier where some members choose to pay a monthly fee and receive best-in-class privacy-protecting tools, as well as other exclusive insider benefits.

https://help.neeva.com/hc/en-us/articles/1500011532921-How-d...

Looks like the $$$ they get from premium subscriptions finances their 'free' offering, but as with all services with this model I am skeptical of anything 'free' since it could be powering their AI and later down the line that will be potentially abused or weaponized.

Thanks for the link - I tried to find a pricing page or equivalent but obviously didn't look hard enough. One thing to note is that the basic search appears to only allow 50 queries per month, and limited use across devices, so it seems they are really just using it as a teaser. That makes me feel better about the product actually, as it suggests against a business model of trying to monetize personal information.

It is half the price of Kagi though, so either they have a lower cost structure somehow or they are currently running it at a loss

from their site: "Neeva was founded by Sridhar Ramaswamy (ex-SVP of Ads at Google) and Vivek Raghunathan (ex-VP of Monetization at YouTube)."

Sounds ominous. Ads & Monetization, all they need now is a director of abusive, addictive psychology to tweak The Algorithm.

Also based on the founders careers, I suspect the "digital bill of rights" page on that site is a total crock of shit too. They'll collect & store everything forever and abuse you with it, because that's what Internet Tech Company means now.

(co-founder here) a bit of a cheap shot :( ...

We strive to earn our users trust every day. We read every piece of feedback that comes in and try to make the product better. At the end of the day, we only make money if our users use our product and opt to pay for it, so our incentives are as aligned with our users as they can be.

I agree that that was a bit of a cheap shot, but in an environment where Facebook, Google and the like are harvesting every last bit of data they can scrape together on every "user" on earth to package and sell, the decision to bundle a VPN and a password manager with a search offering, whether it's paid for or not, seems like it's forcing folks to put a lot more trust in your relatively unknown company than they put in either Facebook or Google.

FWIW, I do think the idea of a paid search utility is a good one, especially if I found it somehow better than Google, but I'd love to hear the reasoning behind the decision to include those services as part of the offering.

For what it's worth, nothing about your response helped. This crowd has a fairly well-tuned bullshit meter when it comes to venture-backed companies. Your incentives are aligned to your investors, as they probably should be. The parent makes a good point that your backgrounds in monetizing user data is concerning, given that history has a way of repeating itself.
Right, but the implication of your background lends one to question, is the user part of the product?

Google and Youtube are advertisers who monetized search queries and content consumption of their users as the end result of their product.

You've not addressed the core of the argument whereby you admit and extol your previous experience, but neglect to decry the actions the company took while you were there.

You want to have control of the flow of information, but you are not a charity and not opposed to reimagining a product to better monetize it.

I like trying out the Kagi Web + AI search beta right now. Not only does it give you chatgpt3 style answers, it cites sources. Going beyond verifying the informatation (which is very comforting) I can dig into the weblinks on my own. (no affiliation other than a satisfied customer) https://labs.kagi.com/ai/contextai

Also I don't like the NeevaAI pricing break down style. The annual plan which cost more upfront is broken down into a monthly amount instead of the what I will pay today at checkout price. It brings back negative memories of signing up for Adobe Cloud.

Cool, but wow is that slow.
Agreed! Since it's in beta par for the course IMO. That being said, in a certain context it's just as fast or faster because I don't have to dig into the webpage/manual cruft to get that nugget of information I need to keep moving! Hope access remains available to all....do we need a "Wikipedia" type of AI (few barriers to access) available to all?
Locked to a geo region? Not a great launch
(co-founder) Working hard to address.
Good to hear. Was it a decision by the team or is there some restrictions with providing the service in different geo regions?
Decision by team to first iterate with users and make the system work really well in one region before expanding
I've been trying it out since it came out and I've been surprised by how good the answers have been. I also like that it cited the sources. Although Neeva doesn't quite have as good of search results as Kagi, I really like all the features that they bring and this NeevaAI really helps make their search better.
In my opinion, we should not just be sitting back and debating the technical merits of this technology. Rather, we should be asking ourselves about the greater societal implications of AI and information. For example, have we considered the fact that computers might be much better at deception than humans, and that this ability for deception will be used against us by megacorps and maybe even governments?

Or what about the fact that AI may eliminate the need for having other people to talk to because it will get so good at satisfying a large portion of our intellectual needs?

I know AI is not there yet, but this is not just something amusing that can improve search. It is the start of a huge change in society that extends the advanced technological infiltration that has already occurred.

Before you look at this as another toy, just take at least an hour of your time to consider what this will lead to. Personally, I find AI disgusting.

On Neeva's front page, they show a side by side of results for Neeva vs. Google with the query "what to do in a car accident".

The example they give for Google is not even close to the one I get in both regular and incognito mode.

My result is a step by step guide from Allstate with more steps provided than Neeva. I only get 3 on Neeva vs. 8 on Google.

The first result that shows on Neeva is a broken link to an Australian gov site. I'm in the US. The second link on Neeva is the exact same link as Google.

So Google is actually giving a way better result because it gives more step by step information, provides a working link, and the website is localized to where I live. You don't want people calling 000 over 911 lol.

Neeva gave me this result:

> If you are in a car accident, the first thing to do is to determine if anyone is injured and call 911 if necessary.1 Then, go to the nearest police station and file a report as soon as possible.2 After that, exchange information with the other driver, take pictures of the scene, and contact your insurance company

Which isn’t really coherent — it says you should go to the police station, THEN take pictures of the scene (which you’ve already left). Definitely has some work to do.

(co founder): have you tried google mobile? I just did by setting my user agent to be an iphone and got the same pile of ads on google.

And trying the same query on Neeva gives you a step by step direction right on top including the information to call 911.

https://docs.google.com/document/d/1j64PkSm0EEoam5l2yByWv3P6...

You are right about Google on mobile. I get 3 ads about settlement collection. Then I get the first actual result which is a better result than I get on desktop. It shows more accurate steps from a local personal injury lawyer's website. The second link is the same Allstate link.

I tried the query on both Neeva mobile site and Android app and still get the same Australian link that is broken. The same link in the picture you have on your website. The second link is the same Allstate link I get on desktop Google.

I guess I never realize mobile Google is vastly different than desktop Google even without an ad block on.

(comment deleted)
Despite errors in the statements made by the LLMs, I'm a huge fan of this syntax which has inline references. Seem like a great bridge between LLMs being "statistical babble" and a useful knowledge-retrieval tool.

Curious whether the references themselves could be wrong? Or how they would be generated to (for example) prioritize primary sources over secondary sources.

(co founder) It is early days in this technology! In some cases (think news), you want the freshest news while in other cases (breaking story), you want the first article that broke the news. Def things that we are working on.
Uses a LLM to summarise sources, it knows the URLs of the sources.
Is there a search engine left that does NOT try to predict what i want to see? Id rather see that my query returned no results instead of tons of maybe hits.
My search clearly returned a skinned version of the google search. I smell vapor.

Edit: that was hasty. I compared the search and saw different hits. But the format was 99.9% identical, and didn't include any AI synthesis (or anything else to recommend it over a typical search). This was with a logged in (free) account. Has anyone triggered the "AI summary w/ citations" behavior?

(comment deleted)
I have been using Neeva for a while and I am curious about the ranking algorithm used to rank webpages. It appears that Neeva is mixing its own search results with Bing, but I am not certain.
(comment deleted)
I paid for neeva for something like 16 months and just cancelled last week. Load times for search were frequently >1s which seems ok the first few times, but after a while got annoying.

Also ~20 esoteric programming queries a month would drive me to google. +10 attempts on Neeva and then same words on google instant hit.

Their personalized search results are awesome. You can downrank spam or even certain sites you don’t like for you personally.

Also your emails and docs will appear in search alongside everything on the web. Which is a really convenient feature.

Their iOS app was plagued with regressions to the point for 2 months last year I couldn’t even use it.

Tried for a long time, but despite all of Google’s failings around ads being 80% of results the premium offerings aren’t even close.

Perplexity.ai seems to give me better results for now