It is quite nice, but not as nice as you'd want. You still have to set platform specifics when running completions when you want to tune things like temperature, effort, max tokens, etc.
I found Ruby LLM to be surprisingly good - in terms of usability it's close to Vercel's AI framework.
It tries to strike a balance between working out of the box and being flexible... which has its challenges, still nice overall.
One big real-life pain I experienced is that caches don't always work, e.g. for xAI, since it only supports completions API and thought signatures are returned wrong.
RubyLLM is very easy to use. Made extensive use of it for a project last year. Drawbacks are it was difficult to instrument for true trace observability and it has a pattern where retries will delete the underlying models so the history you see is clean but not necessarily great for seeing exactly what the sequence of API calls was.
Why would anyone still build in dynamically typed languages in 2026? Why relinquish the crystal clear signals that static typing is able to provide to the LLM?
We use RubyLLM in production too, the most elegant library in this space I've seen so far.
I also liked how they run the issue tracker. If you select "Feature Request", it makes you explain how you explored workarounds, why you believe it belongs in RubyLLM etc to prevent scope creep.
I have been a fan of Ruby for many years, but in this fast paced era the Ruby ecosystem always struggled with the dependency versioning. Gems I relied on were never available or compatible with the rest of the ecosystem.
We use and love RubyLLM! A wonderful and easy to use framework.
Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.
Really looking forward to future development and especially 2.0!
Edit: Just saw that responses API is now native? I will definitely check that out.
RubyLLM is awesome! I use it on side projects. So interesting how questions and comments from last year SF Ruby conf, https://youtu.be/y535u1EWqAg?si=rbyv52T035apKwQk are already features shipped in the ecosystem.
I spent a lot of time with RubyLLM a few months ago. Very nicely designed and implemented. I have my own LLM clients written in various Lisp languages and I thought about appropriating some of the design of RubyLLM. Imitation is flattery.
I'm building something that is only pointed at Claude, and I don't anticipate moving away from the Anthropic ecosystem. Does RubyLLM offer me an advantage over directly using Anthropic's Ruby SDK?
To put it differently, is this more like choosing between Fog and aws-sdk-s3, or choosing between Active Storage and aws-sdk-s3?
36 comments
[ 2.7 ms ] story [ 48.5 ms ] threadIt tries to strike a balance between working out of the box and being flexible... which has its challenges, still nice overall.
One big real-life pain I experienced is that caches don't always work, e.g. for xAI, since it only supports completions API and thought signatures are returned wrong.
https://github.com/tweibley/legate
I also liked how they run the issue tracker. If you select "Feature Request", it makes you explain how you explored workarounds, why you believe it belongs in RubyLLM etc to prevent scope creep.
Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.
Really looking forward to future development and especially 2.0!
Edit: Just saw that responses API is now native? I will definitely check that out.
To put it differently, is this more like choosing between Fog and aws-sdk-s3, or choosing between Active Storage and aws-sdk-s3?