This looks like an excellent start but the problem remains: who pays for and does maintenance (can’t publish it)? Who handles the gui (can’t publish that either)? Without something resembling the money stream of a commercial company, it’s going to remain dependent on a very small number of people.
This kind of work goes back to at least the 90’s when O’Reilly published several books on it. Top-level code in things like Python increase the flexibility of high performance code while writing entirely in an interpretive language has advantages for procedural code.
It would also help if the professional organizations in the pertinent areas agree on file and data structures and api to avoid looking like chemistry with regards to the raft of file formats and standards.
Full disclosure: I am a founding engineer at this company.
We are charging corporate clients for white glove service and minor contracting (helping them upload and adjust pipelines). We also take a small margin over AWS compute prices for regular users. This lets us support the academic and small volume users for free (we give $100 in compute "credits" to all users). So, basically, the money stream of a commercial company is there, as well as the goal to use it for the good of the many.
The GUI is generated from the Python docstring (which optionally includes metadata in an inline YAML) and typing information (PEP 484). We handle all the development and the maintenance on this but theoretically somebody could write an entirely separate frontend by just using our GraphQL endpoint and a few HTTP POST requests and interpret custom GUI metadata (we would need to standardize the APIs and commit to not breaking them). The backend lets you safely run arbitrary code already so I doubt there are many applications for which it would not be flexible enough.
We are aware that there are dozens of previous systems for generating interfaces for pipelines. Our novel contributions primarily come from the reliance on cloud tech (with primary benefit being the ability to scale to extremely compute-intensive workflows), the integration with a cloud-native data system (everything is in S3, with a wrapper to make it considerably more user friendly; primary benefit is, again, scale—you can transparently store terabytes of data), and the use of static typing throughout (primary benefit is that even unannotated workflows get a "good-enough" GUI if they have types).
A confusing title perhaps increase the number of clicks, but also make people unhappy and increase the number of flags. The guidelines ask to use the original title https://news.ycombinator.com/newsguidelines.html but sometimes you can add some detail. For example "The Latch SDK: open-source toolchain for biology applications" or something like that. (I really dislike "solve software", I´m not even sure what it means.)
Full disclosure: I am a founding engineer at this company.
Imo typing is just the most relevant part of the system for how the SDK feels for the developer. The fundamental problems we currently solve are more so related to reproducibility and software rot in particular, and to working with terabyte scale data which is almost universal in bioinformatics. The SDK lets you get a pretty good interface to an arbitrary sized compute node with a native interface to S3, and types make it basically effortless.
I'd love to hear what other problems you think need to be addressed. We've been in business for less than a year, naturally there is still a lot of work to do.
7 comments
[ 1.8 ms ] story [ 21.6 ms ] threadThis kind of work goes back to at least the 90’s when O’Reilly published several books on it. Top-level code in things like Python increase the flexibility of high performance code while writing entirely in an interpretive language has advantages for procedural code.
It would also help if the professional organizations in the pertinent areas agree on file and data structures and api to avoid looking like chemistry with regards to the raft of file formats and standards.
We are charging corporate clients for white glove service and minor contracting (helping them upload and adjust pipelines). We also take a small margin over AWS compute prices for regular users. This lets us support the academic and small volume users for free (we give $100 in compute "credits" to all users). So, basically, the money stream of a commercial company is there, as well as the goal to use it for the good of the many.
The GUI is generated from the Python docstring (which optionally includes metadata in an inline YAML) and typing information (PEP 484). We handle all the development and the maintenance on this but theoretically somebody could write an entirely separate frontend by just using our GraphQL endpoint and a few HTTP POST requests and interpret custom GUI metadata (we would need to standardize the APIs and commit to not breaking them). The backend lets you safely run arbitrary code already so I doubt there are many applications for which it would not be flexible enough.
We are aware that there are dozens of previous systems for generating interfaces for pipelines. Our novel contributions primarily come from the reliance on cloud tech (with primary benefit being the ability to scale to extremely compute-intensive workflows), the integration with a cloud-native data system (everything is in S3, with a wrapper to make it considerably more user friendly; primary benefit is, again, scale—you can transparently store terabytes of data), and the use of static typing throughout (primary benefit is that even unannotated workflows get a "good-enough" GUI if they have types).
Imo typing is just the most relevant part of the system for how the SDK feels for the developer. The fundamental problems we currently solve are more so related to reproducibility and software rot in particular, and to working with terabyte scale data which is almost universal in bioinformatics. The SDK lets you get a pretty good interface to an arbitrary sized compute node with a native interface to S3, and types make it basically effortless.
I'd love to hear what other problems you think need to be addressed. We've been in business for less than a year, naturally there is still a lot of work to do.