Show HN: Burr – A framework for building and debugging GenAI apps faster (github.com)

Yep! That's the plan. We're actively looking for partners who can test it out while we're building/bring us use-cases to noodle on, so feel free to reach out if you're interested.

So yeah, shouldn't be too hard, but Burr does not replace LLM API calls -- you can bring whatever frameworks you want. You could do this by writing a simple function:

    @action(reads=["prompt"], writes=["response"])
    def my_action(state: State) -> Tuple[dict, State]:
       prompt = state["prompt"]
       # you stick in whatever library you want for the call 
       llm_response = whatever_library(prompt)
       state = state.update(response=llm_response)
       return {"response" : llm_response}, state.update(response=llm_response)

Does this help?

Yep! There's a streaming API. It's a little more technically involved, but you can stream back at the point at which an application "halts", meaning you pause and give control back to the user. It only updates state after the stream completes.

Technical details follow:

You can define an action like this:

    @streaming_action(reads=["prompt"], writes=["prompt"])
    def streaming_chat_call(state: State, **run_kwargs) -> Generator[dict, None, Tuple[dict, State]]:
        client = openai.Client()
        response = client.chat.completions.create(
            ...,
            stream=True,
        )
        buffer = []
        for chunk in response:
            delta = chunk.choices[0].delta.content
            buffer.append(delta)
            yield {'response': delta}
        full_response = ''.join(buffer)
        return {'response': full_response}, state.append(response=full_response)

Then you would call `application.stream_result()` function, which would give you back a container object that you can stream to the user:

    streaming_result_container = application.stream_result(...)
    action_we_just_ran = streaming_result_container.get()
    print(f"getting streaming results for action={action_we_just_ran.name}")

    for result_component in streaming_result_container:
        print(result_component['response']) # this assumes you have a response key in your result

    # get the final result
    final_state, final_result = streaming_result_container.get()

Its nice in a web-server or a streamlit app where you can use streaming responses to connect to the frontend. Here's how we use it in a streamlit app -- we plan for a streaming web server soon: https://github.com/DAGWorks-Inc/burr/blob/main/examples/stre....

thanks Julian - your feedback has been super helpful!

So I'm a big fan of limiting complexity -- you generally don't want to add something to your stack unless you get value from it.

If you're getting started with LLMs, the first step is an API call to OpenAI (or some other foundational model) to get a feel for it, then think about how you want to integrate with your application. Burr can help standardize the structure, allowing you to think less about it at any given point (as logic is encapsulated into actions). Furthermore, the UI can help you debug.

With a few POCs at your company (assuming you want to iterate and get into production), Burr can help abstract some complex parts away (state management and telemetry), and make your code cleaner and more extensible, which can help you iterate quickly and explain what you're doing (you can always draw a picture of your app).

We think that it buys companies something they really want -- the ability to swap out implementations/vendors as it decouples the logic from application concerns.

So, if you have logic that's starting to get complicated (and might get more-so), I think Burr is good. If all you're doing is a single wrapper over a GPT-call, it might be overkill (or not! things tend to grow in complexity).

Sure, yes, not going to lie I use printlines all the time. There's nothing inherently wrong with it, and meant no offense! In fact, you can even add a hook with Burr that adds as many printlines as you like :) https://burr.dagworks.io/concepts/hooks/. Meant no offense.

That said, logging stuff and using regexes (which is the common way to productionize printlines for live/post-hoc debugging) has a lot of challenges, and this is where we've seen a framework become useful.

There's an obvious trade-off between adopting a framework and rolling your own -- we've seen both work in various contexts (we've been in MLOps for quite a while). Our hope is that this will make people more productive by constraining the space they need to think about and allow more mental room for application logic.

Right? People forget the roots. Nowasays logs are super complicated things, but in unix and early Linux days, everything was a file descriptor and it happened to be that 0 was the default one to write to and by convention could be routed by pipes and streams into whatever making it super convenient way to emit informations not to users, but to anything

Which is the right thing anyway. If your target are interactive users a log is a terrible thing to do as they won't know where it ends up. And if the target is not interactive users you should have error codes not log messages.

I think that is partly because today is hard to just attach a debugger to things and see the state of a program so people try to serialise the program state to english strings with a timestamp and work backwards from that to understand what the program was doing, but then what we need is a differentiable single stack core dump not an english approximation of it.

And especially not one that gets stolen by systemd and hidden away.

Sorry for the rant but just printing infos and core dumping errors was the best thing ever and it was taken away from me.