Evaluate and Track Your LLM Experiments: Introducing TruLens for LLMs (github.com)

We open-sourced TruLens for LLMs to help evaluate and track your LLM experiments. We also built in special integration with langchain to capture the metadata around your entire chain stack to use with your evaluations. Give it a spin and send us your feedback! We're adding new functionality every day.

Smart toolkit to for quick sanity check why developing

Better LLM app testing is an urgent need when you see the stuff getting put out there.

https://news.ycombinator.com/item?id=35810320 :

> - [ ] dvc, GitHub Actions, GitLab CI, Gitea Actions,: how to add PROV RDF Linked Data metadata to workflows like DVC.org's & container+command-in-YAML approach

https://dvc.org/

https://news.ycombinator.com/item?id=34619424 https://westurner.github.io/hnlog/#comment-34619424 :

- XAI: Explainable AI: https://en.wikipedia.org/wiki/Explainable_artificial_intelli...

- > Right to explanation: https://en.wikipedia.org/wiki/Right_to_explanation

- > A more logged approach with IDK all previous queries in a notebook and their output over time would be more scientific-like and thus closer to "Engineering": https://en.wikipedia.org/wiki/Engineering

Here's a companion blog that explain it works under the hood: https://medium.com/trulens/evaluate-and-track-your-llm-exper...

This is a godsend exactly what i have been searching for

Amazing and very helpful package!!!