Show HN: LLMFlows – LangChain alternative for explicit and transparent apps (github.com)
Hi HN! Over the last several weekends, I've been building LLMFlows as an alternative to langchain.
There's been a lot of discussion on the shortcomings of langchain in the past few weeks, but when I first tried it in March, I thought there are 3 main problems: 1. Too many abstractions 2. Hidden prompts and opinionated logic in chains which makes it hard to customize 3. Hard to debug
This inspired me to try and build a framework that solves these 3 issues, and therefore I started building LLFlows with the "philosophy" of being "simple, explicit, and transparent."
A few weekends later, I think I finally managed to reach a state where I feel it's ready to be shared.
I would love to hear your feedback! Thank you!
22 comments
[ 3.3 ms ] story [ 63.3 ms ] threadDoes the framework help with building the vector store? If not, are there complementary frameworks you use and/or would recommend? I'd think one would want something that watches a collection of source documents (with lots of supported formats) and updates the vector store as those change.
One of the primary benefits of a framework seems to be to act as an abstraction for LLMs and vector stores. I see LLMFlows supports Pinecone. Does it support others (e.g. pgvector)? Does it support LLMs other than OpenAI LLMs?
Building apps with LLMs is fundamentally an exercise of manipulating strings and making API calls (Both LLM and vector db APIs). When apps are more complex and start including multiple LLM calls and prompts that can change dynamically some additional challenges start to emerge. e.g figuring out:
- When was a particular LLM called?
- How much time did it take?
- What were the input variables?
- What did the prompt look like?
- What was the exact configuration of each LLM?
- How many times did we retry the request?
- What was the raw data the API returned?
- What was exactly returned from the vector store and fed into an LLM?
- How many tokens were used?
- What was the final result for each call?
- How do you make API calls in parallel?
I think a framework like this should provide abstractions that allow people to focus on the important part like prompt engineering and productionizing the app and worry less about how to figure out stuff like this.
Right now LLMFlows supports only OpenAI models and Pinecone but I am working on classes for Chroma and Weaviate. And I would like to also provide support for Bard and Claude once I get access.
I appreciate the detailed explanation, thank you!
If you'll permit another noob question, can/should frameworks like this include source attribution for responses? Based on articles like https://jamesg.blog/2023/04/02/llm-prompts-source-attributio... I'd guess the strict answer would be no, and that any strategies you might use to elicit them might hallucinate citations.
Oh, sweet! That would be a wonderful feature to have in a framework. For personal or corporate knowledge bases, sources would be really helpful for people who want to dig deeper.
This is the main thing I'm struggling with langchain. I needed to see this info for troubleshooting. I don't know if it's easily visible and I'm being thick, or it's just hidden away.
For non gpt options, I like Claude and Amazon titan. Claude is definitely good.
I'm also currently learning to use llama.cpp I see it has a langchain integration and it's very fast with GPU. So it's useful to be but as a hobbyist.
I've got no experience with bard though. Is it any good?
I frequently am putting together complex chains of prompts and found microsoft guidance to be quite nice. Have you looked at it?
It feels like the abstraction I'd like the most is some way to combine few shot prompting, voting and chain-of-thought.
I have to create a prompt chain like this fairly often: write a description for this thing, given these previous input and outputs as examples (few shot). Do it N times with high randomness, maybe with expert personas, then review the solutions one by one with pros and cons (chain of thought) and then use all that to create a better description (final answer).
Right now, I always have to write out and connect all the steps, but its fairly rote and I think other prompting chains have a similar repetitiveness to them.
https://github.com/microsoft/guidance
You might find this guide useful to your use case:
https://llmflows.readthedocs.io/en/latest/user_guide/Async%2...
It shows how you can connect multiple similar flowsteps to a previous flowstep and they will all run in parallel.