HN Explorer

Show HN: Mr. Graph – A graph definition and execution library for Python (mr-graph.readthedocs.io)

24 points by jmcminis ↗ HN
What:

Mr. Graph is a python library designed to make composing graphs of sync and async functions easy! Use google style docstrings to automagically create dataclasses and chain together function calls into graphs.

Why:

I like the design of Dagster, but not the latency. For apps and systems engineering, sometimes I want to compose a graph out of regular python functions. I don;t need all the heavy machinery that comes with a full workflow engine.

Current features:

- Use with either async or sync functions

- Uses google style doc strings to name return values.

- Creates dataclasses for each function's output.

- Can infer pipelines from input and output signatures

- All directed acyclic graph layouts supported. linear, fan-in, fan- out.

Future Features:

- Better examples for use with async calls (like LLMs)

- Splitting dataclasses, better error handling, logging improvements.

This is under active development. Any feedback, interest, or contributions are appreciated. Thanks!

github link: https://github.com/mcminis1/mr-graph

6 comments

[ 4.8 ms ] story [ 47.4 ms ] thread
schemescape
Slight tangent: is anyone aware of a library (in Python or JS… or C) for specifying DAG pipielines that supports caching results (based on input)?

Make is closest I’ve found, but there’s probably something better out there.

Edit: I should look at other build tools, but ideally I want a library.

Edit again: I will be running this locally. Forgot to mention that.

mstokholm
alkymi (https://github.com/MathiasStokholm/alkymi) is such a library (disclaimer: I'm the author). I built alkymi to help me cache complex data pipelines, while still having all the niceties of Python.
schemescape
I haven’t dug too deep yet, but this sounds exactly like what I was looking for, thanks!