Show HN: Medical LLM API on par with Google Med-PaLM 2. 92% USMLE accuracy (medisearch.io)
I’d like to share a medical question-answering API which has state-of-the-art performance on the USMLE self-assessment exam. You can try out the MediSearch API in 10 seconds in this Colab [https://tinyurl.com/medisearch-colab], or test the search engine live at MediSearch [https://medisearch.io]. See our API page [https://medisearch.io/api] to apply for access. Some technical highlights:
1. State-of-the-art accuracy of 92% on the USMLE self-assessment exam [https://tinyurl.com/medisearch-eval]. 2. Ranked 2nd on MedQA benchmark, closely tailing Google's Med-PaLM 2 (within ~2%). 3. The API provides article references as an output.
We’d love to hear feedback from this community! We’ve seen people use the MediSearch API to implement a chat bot or search field within their app.
Please note that MediSearch does not have a 100% accuracy. Therefore, a human professional should always make a final decision on the information suggested by MediSearch.
You can use the MediSearch API similarly to the OpenAI API [https://pypi.org/project/medisearch-client/]. If you’d like access to the MediSearch API, please email michal@medisearch.io or visit our API page [https://medisearch.io/api]. Please specify your use case in the application.
13 comments
[ 3.6 ms ] story [ 41.6 ms ] threadWe haven't trained or fine-tuned on neither of USMLE or MedQA. We use several LLMs in our system, and honestly I don't think there is any single thing that I can point to which is different to how other people do it. I guess it's a combination of many small improvements.
Sorry for the vague answer. We are thinking about publishing a report at some point.
Just out of curiosity, is a React client or even a javascript library on your roadmap?
Interesting.
Right now, nothing is known about the business model behind the product. What are the plans for it?