HN Explorer

19 comments

[ 2.2 ms ] story [ 56.1 ms ] thread
[dead] WanderPanda
Math.random() or what part?
bryanrasmussen
no it turns out it's "A distributed approximate nearest neighborhood search (ANN) library which provides a high quality vector index build, search and distributed online serving toolkits for large scale vector search scenario."

But Math.random() is a really good guess.

[dead] afturner
you're funny, we all know it's Google.innerHtml
anewhnaccount2
Good press release title. Now just add it to the pile ;) https://github.com/erikbern/ann-benchmarks
riyadparvez
Yes, exactly! This seems like totally overblown title. It's akin to saying Google open sourced their key search tech Kubernetes, which is an open source rendition of Borg, where all the Google workloads run on top of.
suyash
lol - good point, foiled the Great Microsoft PR play
danielcampos93
I work on the team that built this and many of our other ranking tech. I can answer any questions people have. Also shameless plug: using this tech we released some artificial search sessions as an exploratory dataset. https://github.com/dfcf93/MSMARCO/tree/master/ConversationlS...
mv4
Cool project! One of the challenging use cases mentioned in the description is people taking a picture and asking the search engine, 'What is this?'. Has this been solved? (it is a very hard problem if taken beyond simple object classification)
danielcampos93
In many ways yes. For images Instead of using some NLP parser and vectorization technique you vectorize the image and then do a similar lookup
mv4
how effective is it at identifying mysterious objects, the way it's crowdsourced on Reddit's "/Whatisthis"? (meaning, how big is the index?)
13of40
LOL - that's exactly the use case I was thinking of. But wouldn't an AI have to be trained with lots of examples of the item in question to get a high quality detection? If so, it might not be able to find that one image that antique shop in Whereverston has on their web site.
danielcampos93
I don't work on the image side but from what I understand the entire index is vectorized so its not categorizing them like a imagenet system would as much as finding a nearest neighbor that can be categorized.
sinkpoint
Great work! How does this compare to hierarchical navigable small world graph, like nmslib?
danielcampos93
I wasnt really familiar with nmslib but I guess I think its a little bit faster and scales to billions of items.
bobosha
1. how's this different from pysparnn or faiss?

2. does this support both sparse and dense vectors ann?

[deleted]
(comment deleted)