HN Explorer

25 comments

[ 1.9 ms ] story [ 33.4 ms ] thread
puzer
TL;DR

- The Idea: People use GitHub Stars as bookmarks. This is an excellent signal for understanding which repositories are semantically similar.

- The Data: Processed ~1TB of raw data from GitHub Archive (BigQuery) to build an interest matrix of 4 million developers.

- The ML: Trained embeddings for 300k+ repositories using Metric Learning (EmbeddingBag + MultiSimilarityLoss).

- The Frontend: Built a client-only demo that runs vector search (KNN) directly in the browser via WASM, with no backend involved.

- The Result: The system finds non-obvious library alternatives and allows for semantic comparison of developer profiles.

ashvardanian
Cool project! And thanks for mentioning "unum-cloud/USearch" among repo examples :)
jrockway
That's actually really neat. It suggested regclient/regclient as a repository I'd like. I looked and, yup, I had no idea that existed and it is a sort of thing I like.

People complain about The Algorithm but it can be useful...

mkehrt
Fun fact: cosine similarity's first use in recommendation systems to recommend usenet groups.

(https://dl.acm.org/doi/epdf/10.1145/192844.192905 although they don't call it cosine similarity; they do compute a "correlation coefficient" between two people by adding together the products of scores each gave to a post)

Retr0id
Very high quality "Recommended repos for you" results, the top one was in fact a repo I was looking for a couple of days ago but did not successfully find.

I just wish I could scroll further down the "Similar to you" list.

jbl0ndie
Excellent. Found me three other stars and one to that I knew from before but hadn't started. Nice!
embedding-shape
It seems to generate pretty good "Recommended repos for you" suggestions, all of them I've heard and seen before, but for one or another reason didn't use for anything or found a need for. Would be great if it could show more options than just 10, because I'm sure further down the list it'd have interesting suggestions I hadn't seen before.
armcat
This is so nice, it's essentially a collaborative filter (like Spotify recommendations). It would be awesome to try and embed your repos directly, using some LLM embedding like `text-embedding-3-large` and use that either directly or as a re-ranking/scaling mechanism in the recommendation. You might unearth some other interesting repos or people that are doing similar projects but not necessarily starring similar repos.
armcat
It would be a good idea to filter out those repos I actually starred - because they are getting a 100% hit (of course they are!).
ComputerGuru
99% match to Graydon Hoare and 97% to burntsushi. Could do worse!
aeonik
I am also a 99% March to Graydon Hoare.

Makes me wonder if there is something in his stars that is skewing the results.

keeganpoppen
this is amazing! i am a bit of a github star enjoyer, and have always wanted something like this. thank you! it looks like for now you take the most recent 500 stars? i have a bit over 1k (i think?), so i would love the 2.1x on that constraint, but completely understand any desire to not do that. fun project! :)
swyx
the frontend is beautiful. i find it inspiring that you have 10 years of data science and are no longer limited by your lack of frontend or design knowledge. this is a better site than i couldve done
6r17
Ok so i've not been using github for the past 2 years; it matched me closed to Salvatore Sanfilippo with subtitle "creator of redis" - and It just happens that I did write a key-val and more generally working on a database in the meantime.

I don't know how to feel about this lmao

herdrick
Good stuff. Are star count and forks etc. the criteria for inclusion of repos? Lots of repos result in "Repository not found".
andriamanitra
That's really neat! I found a bunch of cool repositories I had never heard of by looking up my username and a few of my favorite projects.
[deleted]
(comment deleted)
Imustaskforhelp
https://puzer.github.io/github_recommender/#p=eyJ0IjoicHJvZm...

I have got around 1800k projects starred. Usually its just that I had lost my bookmarks once and I lost a lot of github projects so I decided to use stars as my bookmarks or even as whatever I was feeling that time so I have starred some 100 projects or so just because I think they were interesting just enough and nobody starred them so to show my support

Supporting is also another aspect, I really like to share my support and I feel like even these tiny actions at scale really help these projects whether gaining legitimacy or otherwise

I have been such a star fanatic that I have even opened up a github issue about who are the people who have starred the most projects just to give a clear referrence

I have even downloaded all the readme.mds of my github projects that I have starred and made a simple html vibe coded project so that I can view them manually and search them similar to algolia you could say.

Oh btw there are some gists which can help you list all the stars of a person in github which I used to get the star list (or list of repos) then downloading all their readme.mds and converting that as such. Its on my other computer but I should probably back it up as well

I wish there was something like github stars for the whole web in whole. Yes bookmarks exist but a more public form of bookmarks in a way similar to github stars without monetizing in the front (yes I know they are doing AI shenanigans in the background)

Github is still an Okay platform so much so that I nowadays am thinking of uploading media in github wiki for projects instead of youtube. Especially for open source projects, plus even github wiki's can be downloaded via git whereas youtube tries to do everything in control to make you stop making it download so much so that recently they made some changes downstream that even yt-dlp now requires deno or npm engine and the solution is always hacky/ cat and mouse game of sorts.

I don't think that there are any services which can provide the amount of free bandwidth github provides in the way it does. Sure one can get

To be honest, if someone wants, they can probably use ovh or upcloud's zero have unlimited egress with fair use policy

that fair use policy though is basically just that your server would first have I think around 1gbps or 500mbps or like high bandwidth access but then they would cap it to something to 100mbps and ovh can throttle

Upcloud has like an extremely high fair use acceptance policy around 24 TB I think after which they throttle a 1gbps connection to 100mbps which in many vps's could be the highest connection itself and 100mbps aint bad

But also pardon me for this but I asked chatgpt and it seems that civo provides completely unrestricted

Extra Small 1 GB 1 core 30GB NVMe FREE $5.43

Upcloud's around (3.50 euros for the same thing) but if your project is getting even more than 24 TB and you want like other options there are always options

So like in a sense, there just isn't a point in either self hosting and I feel like github can be the freemium thing from youtube to something which can be transitioned to.

Just me rambling but I feel like in the early days Youtube used one of the deals to get their bandwidths as well. I feel as if there are companies which can do that too and Youtube is moving in backw...

msephton
"Error: Load failed" for me. I have a lot of starred projects. Edit : finally worked after 4 retries
PunchyHamster
I don't think it works very well, shows AI, gamedev and mobile and those are 3 things I never wrote line of code about nor starred anything sans maybe some JS lib