I spent $100 to learn how to build and run RAG application in production (twitter.com)

This post is about learnings by running a RAG application in production.

Here are the learnings:

• Always customise your prompt. • Set soft & hard limit on your LLM cost before launching any project. • Choose the LLM model wisely. • Context length matters a lot. • Cache your queries. • Have a router to choose LLM model wisely. • Have a UI to see all queries, answers, context & metrics like response time. • Memory management in chat is painful.

Great! This is very helpful, thank you for sharing such insights!

Great insights on do's and dont's of RAG app development!

Embedchain seems quite promising. Will give it a try for my next project.

If you're gonna use bots to hype up your submissions, at least make them sound different.