I don't know their financials to know if it's a good investment or not, but I have a feeling that they are over hyped. Vector DBs are becoming a commodity with many hosting options too. Again, I don't know what they have in plan for growth. Long term memory is too vague. We'll see how it goes.
Perfect example of AI gold rush nonsense. Pinecone has zero moat and quite a few free alternatives (Faiss, Weviate, pg-vector). Their biggest selling point is that AI hype train people don’t Google “alternatives to pinecone” when cloning the newest trending repo (or I guess, ask ChatGPT).
Spot on. There is zero moat and the self-hosted alternatives are rapidly improving (if not better) than Pinecone. There are good open-source contributions coming from bigcorp beyond Meta too, e.g., DiskANN (https://github.com/microsoft/DiskANN).
Maybe I am fundamentally missing something, but a "cloud database company" seems like the most boring tech? No one is calling Planetscale or Yugabyte nonsense because there are free alternatives like Postgres.
> Pinecone has zero moat and quite a few free alternatives (Faiss, Weviate, pg-vector)
Faiss is a collection of algorithms for in-memory exact and approximate high-dimensional (e.g., > ~30 dimensional) dense vector k-nearest neighbor, it doesn't add or really consider persistence (beyond full index serialization to an in memory or on disk binary blob), fault tolerance, replication, domain-specific autotuning and the like. The "vector database" companies like Pinecone, Weviate, Zilliz and what not will add these other features to turn them into a complete service, they're not really the same. pgvector seems to be DB-backed IndexFlat and IndexIVFFlat (?) from the Faiss library at present but is of course not a complete service.
However which kind of approximate indexing you want to use very much depends upon the data you're indexing, and where in the tradeoff space between latency, throughput, encoding accuracy, NN recall and memory/disk consumption you want to be (these are the fundamental tradeoffs in the vector search domain), and whether you are performing batched queries or not. To access the full range of tradeoffs you'd need to use all of the options which are available in Faiss or similar low-level libraries which may be difficult to use or require knowledge of underlying algorithms.
Happy for them, has been a very smooth developer experience using Pinecone and I think there is more than meets the eye with the combined keyword+vector search and handling of filtering. It's not going to solve all problems, but it's a well scoped solution at a good time.
I honestly hope they use it to improve their documentation. I consider myself and pretty adept developer but without much background in AI and was looking for a solution for building out a recommendation engine and ended up at Pinecone.
Maybe I'm not the target audience but after spending some time poking around I couldn't honestly couldn't even figure out how to use it.
Even a simple Goole Search for "What is a Vector Database" ends up with this page.
Pinecone is a vector database that makes it easy for developers to add vector-search features to their applications
Um okay... what's "vector-search"? For that sake what the eff is a "Vector" to begin with? Finally after getting about a third down the page we start defining what a vector is....
Maybe I'm not their target audience but I ended up poking around for about an hour or two before just throwing up my hands and thinking it wasn't for me.
Ended up just sticking with Algolia, since we had them in place for Search anyway...
Respectfully if you don’t know what a vector is, you probably don’t need a vector DB.
When they say “vector-search” they mean semantic search. I.e. “which document is the most semantically similar to the query text”.
So how do we establish semantic similarity?
In a database like Elasticsearch, you store text and the DB indexes the text so you can search.
In a vector DB you don’t just store the raw text, you store a vectorized version of the text.
A vector can be thought of as an array of numbers. To get a vector representation we need some way to take a string and map it to an array while also capturing the notion of semantics.
This is hard, but machine learning models save the day! The first popular model used for this was called “word2vec” while a more modern model is BERT.
These take an input like “fish” and output a vector like [ 3.12 … 4.092 ] (with over a thousand more elements).
So let’s say we have a sentence that we vectorized and we want to compare some user input to see how similar it is to that sentence. How?
If we call our sentence A and the input vector B, we can compute a number between zero and one that tells us how similar they are.
This is called cosine similarity and is computed by taking the dot product of the two vectors and dividing by both of their magnitudes.
When you load a bunch of vectors in a vector DB, the principal operation you will perform is “give me the top K documents that are similar to the input”. The databases indexing process computes k nearest neighbors algorithm on all vectors in the DB and stores this for use at query time.
Without the indexing process there is no real difference between a vector db and key value store.
Respectfully if you don’t know what a vector is, you probably don’t need a vector DB.
I wasn't looking for one ;-) I was looking for a recommendation engine, similarly most often I'm looking for various ways to use ML and AI to improve various features and workflows.
Which I guess is my point, I don't know who Pinecone's target market is but from following this thread it seems like all the folks who know how to do what they do have alternatives that suit them better. If they are targeting folks like me they're not doing it well.
Pinecone's examples[1] (hat tip to Jurrasic in this thread - I've seen these) all show potential use cases that I might want to leverage, but when you dive into them (for example the Movie Recommender[2] - my use case) I end up with this:
The user_model and movie_model are trained using Tensorflow Keras. The user_model transforms a given user_id into a 32-dimensional embedding in the same vector space as the movies, representing the user’s movie preference. The movie recommendations are then fetched based on proximity to the user’s location in the multi-dimensional space.
It took me another 5 minutes of googling stuff to parse that sentence. And while I could easily get the examples to run I was still running back and forth to Google to figure out what it was doing in the examples - again the documentation is poor here. I'm not a Python dev but I could follow it but I still had to google tqdm to figure out it was a progress bar library?
Also, and this is not unique to Pinecone, I've found generally that while some things are fairly well documented on "Here's how to build a Movie Recommender based on these datasets) frequently in this space there's very little data on how to build a model using your own datasets ie how to take this example and do it with your own data.
Don't worry, you're just catching up in one hour on 10 years of NLP research. There has to be some conceptual gap to cross. After you clarify the "vector" and "computing similarity" concepts, it's pretty nifty. You have a text
emb = model(text)
Now you got the embedding. What can you do with it? you can calculate how similar it is to other texts.
emb1 = model(text1)
emb2 = model(text2)
similarity = sum([a * b for a, b in zip(emb1, emb2)])
Just a multiply and add, this is trivial! So if you do that for a million texts, you got a search engine. Vector DBs are automating this for you. There are free libraries just as good. And free models to embed text with, OpenAI also have some great embeddings. You can use np.dot to compute similarities fast, up to 100,000 vectors it's the best way and get exact, not approximate results.
The great thing about embedding text is the simplicity of the API and the similarity operation. It's dead simple to use. You can do clustering, classification, near neighbour search / ranking, recommendation, or any kind of semantic operations between two texts that can be described as a score. If you cache your vectors you can search very very quickly with np.dot or other methods, in a few ms. Today you can also embed images to the same vector space and do image classification by taking the text label with max dot product.
You can also train a very small model on top of embeddings to classify the input into your desired classes, if you can collect a dataset. Embeddings are the best features for text classification. You can think of this embedding method as a way to slice and dice in the semantic space like you do with strings in character space. All fast and local, without GPUs.
I’ve wanted to ask this question but I don’t know who to ask.
Can someone explain what the use case is for vector DBs like pinecone, milvus etc. vs a fully featured search engine like Vespa, ElasticSearch etc. which also support vector search features?
Is there something about running this type of index operationally that is particularly difficult?
I’ve been playing around here a bit. One feature is that documents have embeddings stored at index time which means that query performance is quite good. Another is that the model you’re using to embed and query can be changed and configured based on specific use cases. A pretty cool feature of Marqo is multi-statement queries that let you search for multiple positive queries and even include negative queries to filter out results.
I was using pinecone before installing pgvector in Postgres. Pinecone works and all but having the vectors in Postgres resulted in an explosion of use for us. Full relational queries with where clauses and order by etc AND vector embeddings is wicked.
Speaking of the repo, they have a number of features they want to add if anyone is interested in contributing, there's lots of room for advancement. Many of these features already have active branches
https://github.com/pgvector/pgvector/issues/27
Why do you use pgvector instead of pgANN? My understanding is pgANN is built with FAISS. When I compared pgvector with FAISS, pgvector was 3-5x slower.
There is certainly a wide variety of problems today for which pgvector is unsuitable due to performance limitations... but fear not! This is an area that is getting significant focus right now.
This hits home, it is a big ask to keep data in sync for yet another store. We already balance MS SQL and Algolia and all the plumbing required to catch updates, deletes, etc. adding another feels like a bridge too far. Hopefully MS will get on this train at some point and catch up to postgres.
I wasn't making a personal recommendation to you? I was answering more broadly why someone would use Pinecone in the future.
Every new software company like this has "why wouldn't everyone just use x existing open source project, why even try to make it a real business with a hundred devs, actual support/marketing, and big ambitions to be more than a plugin to Postgres?"
Based on the videos and interviews with their lead dev I've seen Pinecone has some quite large plans by integrating with a wider stack and integrating with company databases, well beyond what they have done so far releasing an early version of the DB.
Regardless, getting wider adoption via actual businesses investing in marketing/sales to seed ideas in the market can spur development and potentially progress/innovate the tooling across the wider market, that feeds back into open source.
Same with opensearch and elasticsearch, both of which have added vector search as well (slight differences between their implementations). And since vector search is computationally expensive, there is a lot of value in narrowing down your result set with a regular query before calculating the best matches from the narrowed down result set.
From what I've seen, the big limitation currently is dimensionality. Most of the more advanced models have a high dimensionality and especially Elasticsearch and Lucene limit the dimensionality to 1024. E.g. several of the openai models have a much higher dimensionality. Opensearch works around this by supporting alternate implentations to lucene for vectors.
Of course it's a sane limitation from a cost and computation point of view, having these huge embeddings doesn't scale that well. But it does limit the quality of the results unless you can train your own models and tailor them to your use case.
If you are curious on how to use this stuff, I invested some time a few weeks ago getting my kt-search kotlin library to support this and wrote some documentation for this: https://jillesvangurp.github.io/kt-search/manual/KnnSearch.h.... The quality was underwhelming IMHO but that might be my complete lack of experience with this stuff.
I have no experience with pinecone and I'm sure it's great. But I do share the sentiment that they might not come out on top for this. There are too many players here and it's a fast moving field. OpenAI just majorly moved the whole field forward enormously in terms of what is possible and feasible.
Can someone explain how vector databases can be used as long term memory for AI (and LLMs specifically)?
Isn't there still a token limit as to how much ChatGPT can hold in working memory?
Is the goal that ChatGPT can query the vector database directly to get information out, and if so, how is that different than using a regular database?
Vector databases make it so semantically similar sentences get mapped to be closer together in the vector space.
So the sentence "I started working as a programmer" will be very close to "I began my job as a software developer". This makes it very powerful for natural language search.
So when the user asks a bot "Find the text message John sent me 3 years ago about wanting to found a company, I think it was like a hang gliding company? or parasailing? idk"
Behind the scenes you can ask GPT "Output a list of 10 candidate sentences that are plausible text messages that John may have sent", and it spits out
"I'm thinking of starting a hang gliding company" and "I might found a parasailing company" etc.
Then you query the vector DB for those imaginary sentences, and as a result you get sentences that are semantically similar. You take the top N nearest neighbors and plug them in to the GPT context window and say:
"Here are 100 sentences that might match the original query. If any of them match, select the number that matches. Otherwise output null"
If you engineer this system well, you can get pretty decent results.
This is great for key:value type querying, but IMO there is a lot of ground to be explored by extending it with more graph-like links. I.e. use vector search to get some initial nodes that are better than random, but then start doing a little beam-search algorithm from those nodes to find nodes they are "linked" to in some way that may answer the query better.
So you have to feed in the items to ChatGPT manually (or via some script) it looks like? In the future I guess ChatGPT with plugins could query the database on its own?
Does it work for text data or can it work for other types of data as well?
You call the ChatGPT API programmatically, entirely automated. You call the API, parse the response, make a decision in code, call the API again, etc. The GPT API just becomes a natural language reasoning module in an otherwise normal codebase.
It works for all types of data. You can say "Give me objects similar to water" and have it return words like "liquid" and "juice," pictures of water, the water drop emoji, and babbling brook and rain storm sound files.
It all depends on how you produce the vectors before storage. The vector database just stores them.
How does the production of vectors work? How do you know that you should associate the water emoji to pictures of water close together in the hyperspace?
They're so hot right now that you can't even signup for a starter account. I'm guessing this money will help them fix that so as to not slow down their potential customer base. It's a really easy DB to use for people with no idea about vector DBs, etc.
This is what we’re using. We already sync database content to a typesense DB for regular search so it wasn’t much more work to add in embeddings and now we can do semantic search.
> We raised $100 million in Series B funding, led by Andreessen Horowitz,
I closed the browser tab here. It's a usual techbro hustle, nothing to see here, move along. Hype, whatnot, couldn't care less. I am sure they will make money but it'll be detrimental to society. As always.
99 comments
[ 2.0 ms ] story [ 154 ms ] threadI'm shocked.
Faiss is a collection of algorithms for in-memory exact and approximate high-dimensional (e.g., > ~30 dimensional) dense vector k-nearest neighbor, it doesn't add or really consider persistence (beyond full index serialization to an in memory or on disk binary blob), fault tolerance, replication, domain-specific autotuning and the like. The "vector database" companies like Pinecone, Weviate, Zilliz and what not will add these other features to turn them into a complete service, they're not really the same. pgvector seems to be DB-backed IndexFlat and IndexIVFFlat (?) from the Faiss library at present but is of course not a complete service.
However which kind of approximate indexing you want to use very much depends upon the data you're indexing, and where in the tradeoff space between latency, throughput, encoding accuracy, NN recall and memory/disk consumption you want to be (these are the fundamental tradeoffs in the vector search domain), and whether you are performing batched queries or not. To access the full range of tradeoffs you'd need to use all of the options which are available in Faiss or similar low-level libraries which may be difficult to use or require knowledge of underlying algorithms.
(I'm the author of the GPU half of Faiss)
Maybe I'm not the target audience but after spending some time poking around I couldn't honestly couldn't even figure out how to use it.
Even a simple Goole Search for "What is a Vector Database" ends up with this page.
https://www.pinecone.io/learn/vector-database/#what-is-a-vec...
Pinecone is a vector database that makes it easy for developers to add vector-search features to their applications
Um okay... what's "vector-search"? For that sake what the eff is a "Vector" to begin with? Finally after getting about a third down the page we start defining what a vector is....
Maybe I'm not their target audience but I ended up poking around for about an hour or two before just throwing up my hands and thinking it wasn't for me.
Ended up just sticking with Algolia, since we had them in place for Search anyway...
When they say “vector-search” they mean semantic search. I.e. “which document is the most semantically similar to the query text”.
So how do we establish semantic similarity?
In a database like Elasticsearch, you store text and the DB indexes the text so you can search.
In a vector DB you don’t just store the raw text, you store a vectorized version of the text.
A vector can be thought of as an array of numbers. To get a vector representation we need some way to take a string and map it to an array while also capturing the notion of semantics.
This is hard, but machine learning models save the day! The first popular model used for this was called “word2vec” while a more modern model is BERT.
These take an input like “fish” and output a vector like [ 3.12 … 4.092 ] (with over a thousand more elements).
So let’s say we have a sentence that we vectorized and we want to compare some user input to see how similar it is to that sentence. How?
If we call our sentence A and the input vector B, we can compute a number between zero and one that tells us how similar they are.
This is called cosine similarity and is computed by taking the dot product of the two vectors and dividing by both of their magnitudes.
When you load a bunch of vectors in a vector DB, the principal operation you will perform is “give me the top K documents that are similar to the input”. The databases indexing process computes k nearest neighbors algorithm on all vectors in the DB and stores this for use at query time.
Without the indexing process there is no real difference between a vector db and key value store.
I wasn't looking for one ;-) I was looking for a recommendation engine, similarly most often I'm looking for various ways to use ML and AI to improve various features and workflows.
Which I guess is my point, I don't know who Pinecone's target market is but from following this thread it seems like all the folks who know how to do what they do have alternatives that suit them better. If they are targeting folks like me they're not doing it well.
Pinecone's examples[1] (hat tip to Jurrasic in this thread - I've seen these) all show potential use cases that I might want to leverage, but when you dive into them (for example the Movie Recommender[2] - my use case) I end up with this:
The user_model and movie_model are trained using Tensorflow Keras. The user_model transforms a given user_id into a 32-dimensional embedding in the same vector space as the movies, representing the user’s movie preference. The movie recommendations are then fetched based on proximity to the user’s location in the multi-dimensional space.
It took me another 5 minutes of googling stuff to parse that sentence. And while I could easily get the examples to run I was still running back and forth to Google to figure out what it was doing in the examples - again the documentation is poor here. I'm not a Python dev but I could follow it but I still had to google tqdm to figure out it was a progress bar library?
Also, and this is not unique to Pinecone, I've found generally that while some things are fairly well documented on "Here's how to build a Movie Recommender based on these datasets) frequently in this space there's very little data on how to build a model using your own datasets ie how to take this example and do it with your own data.
[1] https://docs.pinecone.io/docs/examples
[2] https://docs.pinecone.io/docs/movie-recommender
The great thing about embedding text is the simplicity of the API and the similarity operation. It's dead simple to use. You can do clustering, classification, near neighbour search / ranking, recommendation, or any kind of semantic operations between two texts that can be described as a score. If you cache your vectors you can search very very quickly with np.dot or other methods, in a few ms. Today you can also embed images to the same vector space and do image classification by taking the text label with max dot product.
You can also train a very small model on top of embeddings to classify the input into your desired classes, if you can collect a dataset. Embeddings are the best features for text classification. You can think of this embedding method as a way to slice and dice in the semantic space like you do with strings in character space. All fast and local, without GPUs.
Can someone explain what the use case is for vector DBs like pinecone, milvus etc. vs a fully featured search engine like Vespa, ElasticSearch etc. which also support vector search features?
Is there something about running this type of index operationally that is particularly difficult?
Overall I’m really enjoying the tech.
You can always use an open-source alternative. But you probably shouldn't self-host at early stage atleast.
Only a sucker being forced to by their investors would use pinecone.
Supabase was also asking for sparse vectors https://github.com/pgvector/pgvector/issues/81
Speaking of the repo, they have a number of features they want to add if anyone is interested in contributing, there's lots of room for advancement. Many of these features already have active branches https://github.com/pgvector/pgvector/issues/27
https://github.com/netrasys/pgANN
A marqo.ai dev is currently working on adding HNSW-IVF and HNSW support https://news.ycombinator.com/item?id=35551684 and the maintainer has recently noted that they are actively working on an IVFPQ/ScaNN implementation https://github.com/pgvector/pgvector/issues/93
The pgAnn creator actually asked about performance a month ago here https://github.com/pgvector/pgvector/issues/58
Expect to see performance improve dramatically later this year.
recent vector database fundraises:
- Chroma - $18M seed https://www.trychroma.com/blog/seed
- Weaviate - $50m A https://www.theinformation.com/articles/index-ventures-leads...
- Pinecone - $100M B
Harder to set up than wrappers like Chroma, but very powerful.
It quite angers me that people (on HN) will consider the following to be benefits worth mentioning as pros to the consumer:
>Sleek/shiny finish
>Marketing/Branding
>Ability to Monetize
We arent shareholders, all 3 of these are bad for the customer.
Every new software company like this has "why wouldn't everyone just use x existing open source project, why even try to make it a real business with a hundred devs, actual support/marketing, and big ambitions to be more than a plugin to Postgres?"
Based on the videos and interviews with their lead dev I've seen Pinecone has some quite large plans by integrating with a wider stack and integrating with company databases, well beyond what they have done so far releasing an early version of the DB.
Regardless, getting wider adoption via actual businesses investing in marketing/sales to seed ideas in the market can spur development and potentially progress/innovate the tooling across the wider market, that feeds back into open source.
From what I've seen, the big limitation currently is dimensionality. Most of the more advanced models have a high dimensionality and especially Elasticsearch and Lucene limit the dimensionality to 1024. E.g. several of the openai models have a much higher dimensionality. Opensearch works around this by supporting alternate implentations to lucene for vectors.
Of course it's a sane limitation from a cost and computation point of view, having these huge embeddings doesn't scale that well. But it does limit the quality of the results unless you can train your own models and tailor them to your use case.
If you are curious on how to use this stuff, I invested some time a few weeks ago getting my kt-search kotlin library to support this and wrote some documentation for this: https://jillesvangurp.github.io/kt-search/manual/KnnSearch.h.... The quality was underwhelming IMHO but that might be my complete lack of experience with this stuff.
I have no experience with pinecone and I'm sure it's great. But I do share the sentiment that they might not come out on top for this. There are too many players here and it's a fast moving field. OpenAI just majorly moved the whole field forward enormously in terms of what is possible and feasible.
Isn't there still a token limit as to how much ChatGPT can hold in working memory?
Is the goal that ChatGPT can query the vector database directly to get information out, and if so, how is that different than using a regular database?
So the sentence "I started working as a programmer" will be very close to "I began my job as a software developer". This makes it very powerful for natural language search.
So when the user asks a bot "Find the text message John sent me 3 years ago about wanting to found a company, I think it was like a hang gliding company? or parasailing? idk"
Behind the scenes you can ask GPT "Output a list of 10 candidate sentences that are plausible text messages that John may have sent", and it spits out
"I'm thinking of starting a hang gliding company" and "I might found a parasailing company" etc.
Then you query the vector DB for those imaginary sentences, and as a result you get sentences that are semantically similar. You take the top N nearest neighbors and plug them in to the GPT context window and say:
"Here are 100 sentences that might match the original query. If any of them match, select the number that matches. Otherwise output null"
If you engineer this system well, you can get pretty decent results.
This is great for key:value type querying, but IMO there is a lot of ground to be explored by extending it with more graph-like links. I.e. use vector search to get some initial nodes that are better than random, but then start doing a little beam-search algorithm from those nodes to find nodes they are "linked" to in some way that may answer the query better.
Does it work for text data or can it work for other types of data as well?
It all depends on how you produce the vectors before storage. The vector database just stores them.
https://towardsdatascience.com/milvus-pinecone-vespa-weaviat...
It "just works".
The key thing is that it's in-memory and allows you to combine attribute-based filtering, together with nearest-neighbor search.
We're also working on a way to automatically generate embeddings from within Typesense using any ML models of your choice.
So Algolia + Pinecone + Open Source + Self-Hostable with a cloud hosted option = Typesense
I closed the browser tab here. It's a usual techbro hustle, nothing to see here, move along. Hype, whatnot, couldn't care less. I am sure they will make money but it'll be detrimental to society. As always.