As far as I can tell it should be possible to reuse this fine tuning code entirely and just replace `--embedder_name_or_path BAAI/bge-base-en-v1.5` with `--embedder_name_or_path MongoDB/mdbr-leaf-ir` Note that…
I interacted with the authors of these models quite a bit! These are very interesting models. The tradeoff here is that you get even faster inference, but lose on retrieval accuracy [0]. Specifically, inference will be…
For the retrieval stage, we have developed a highly efficient, CPU-only-friendly text embedding model: https://huggingface.co/MongoDB/mdbr-leaf-ir It ranks #1 on a bunch of leaderboards for models of its size. It can be…
You can refer to https://huggingface.co/spaces/mteb/leaderboard and use that to guide your selection. Check under the "Retrieval" section, either RTEB Multilingual or RTEB German (under language specific). You may also…
I created a small app that shows the difference between embedding-based ("semantic") and bm25 search: http://search-sensei.s3-website-us-east-1.amazonaws.com/ (warning! It will download ~50MB of data for the model…
I believe it's because the way you measure things in RL, each episode only tells you whether it was good (say reward +1) or bad (say 0 or negative reward), it does not tell you anything about the trace that was produced…
You can process a single word through a transformer and get the corresponding intermediate representations. Though it sounds odd there is no problem with it and it would indeed return the model's representation of that…
[dead]
Embeddings as a tool have been around for longer than LLMs. They were (and are) ubiquitous in, e.g., recommender systems. It sounds maybe this would be more in-line with what you are looking for. In this case, check out…
As far as I can tell it should be possible to reuse this fine tuning code entirely and just replace `--embedder_name_or_path BAAI/bge-base-en-v1.5` with `--embedder_name_or_path MongoDB/mdbr-leaf-ir` Note that…
I interacted with the authors of these models quite a bit! These are very interesting models. The tradeoff here is that you get even faster inference, but lose on retrieval accuracy [0]. Specifically, inference will be…
For the retrieval stage, we have developed a highly efficient, CPU-only-friendly text embedding model: https://huggingface.co/MongoDB/mdbr-leaf-ir It ranks #1 on a bunch of leaderboards for models of its size. It can be…
You can refer to https://huggingface.co/spaces/mteb/leaderboard and use that to guide your selection. Check under the "Retrieval" section, either RTEB Multilingual or RTEB German (under language specific). You may also…
I created a small app that shows the difference between embedding-based ("semantic") and bm25 search: http://search-sensei.s3-website-us-east-1.amazonaws.com/ (warning! It will download ~50MB of data for the model…
I believe it's because the way you measure things in RL, each episode only tells you whether it was good (say reward +1) or bad (say 0 or negative reward), it does not tell you anything about the trace that was produced…
You can process a single word through a transformer and get the corresponding intermediate representations. Though it sounds odd there is no problem with it and it would indeed return the model's representation of that…
[dead]
Embeddings as a tool have been around for longer than LLMs. They were (and are) ubiquitous in, e.g., recommender systems. It sounds maybe this would be more in-line with what you are looking for. In this case, check out…