Not specific to this project, but why are so many ML-adjacent tools/services so tightly integrated with Python? It feels like the equivalent of going “write a bunch of C to use Postgres”. We have a code base consisting of a bunch of different languages, and given the option, nobody on the team wants to write any more Python if we don’t have to, but so many projects seem to have functionality or internals that are inextricably linked to operating with your code, inside a Python runtime.
One reason is that the most popular models were developed using either the TensorFlow or PyTorch Python APIs. The pre-trained models took an immense amounts of compute resources to build. Additionally, those who built the models weren't necessarily developers and Python is an low-barrier to entry language.
There are a number of models that are now available via APIs and can be used from any language.
> The pre-trained models took an immense amounts of compute resources to build.
Oh definitely, but nobody is serving models from the same machine + process that they used to train them right? And solutions like ONNX exist (although TF and PyTorch’s support is inconsistent at best)
Additionally, those who built the models weren't necessarily developers and Python is an low-barrier to entry language.
It just feels like an engineering anti-pattern to build “down” to this level, instead of skilling people up, or standardising on some standard model serialisation and serving format, model serving tools exist, and they’re often written in faster/more optimised languages, so at that point, why bother with Python after actual model training at all.
True, if a team doesn't want to use Python, the way models were trained shouldn't be the reason to use Python. ONNX is a good option, txtai has a notebook that shows how to export models for use in Rust/JavaScript/Java - https://github.com/neuml/txtai/blob/master/examples/18_Expor...
Seems like a lot of tooling is being created in other languages besides Python, may just take some time to get there.
Because there are no good reasons not to use Python if you want to do ML, and also there's simply no strong contender to Python in ML field. Maybe Julia, but it will need a lot more effort to gain critical mass.
But I want to serve them, and run things like Vector search engines, and I don’t see a reasonable justification as to why we need to be chained to Python at that point as well.
Wild, I have been working on something almost exactly like this on and off for about a year. Well sort of, similar concept.
Basically, python notebooks with an "accelerator" library and strictly defined inputs and outputs - allowing you to run parallel jobs with parameters that do complicated backend operations normally not efficient in python, and then transform the result into a standardized format.
It's for a security-related-space set of use cases, but it doesn't have to be. So, I am very pleased to see something with a similar workflow that I can take notes on ;)
14 comments
[ 242 ms ] story [ 1713 ms ] threadHere is a link to a live example: https://huggingface.co/spaces/NeuML/txtai?default=search+hac...
The example above builds a semantic index for current Hacker News front page titles.
this is all code.
The above workflows can be built with this application - https://github.com/neuml/txtai/blob/master/examples/workflow...
The application can create/save/load workflows and generates the workflow configuration, no-code required.
There are a number of models that are now available via APIs and can be used from any language.
Oh definitely, but nobody is serving models from the same machine + process that they used to train them right? And solutions like ONNX exist (although TF and PyTorch’s support is inconsistent at best)
Additionally, those who built the models weren't necessarily developers and Python is an low-barrier to entry language.
It just feels like an engineering anti-pattern to build “down” to this level, instead of skilling people up, or standardising on some standard model serialisation and serving format, model serving tools exist, and they’re often written in faster/more optimised languages, so at that point, why bother with Python after actual model training at all.
Seems like a lot of tooling is being created in other languages besides Python, may just take some time to get there.
But I want to serve them, and run things like Vector search engines, and I don’t see a reasonable justification as to why we need to be chained to Python at that point as well.
Here is a link to a live example: https://huggingface.co/spaces/NeuML/txtai?default=search+hac...
And configuration for the live example, which is no-code: https://huggingface.co/spaces/NeuML/txtai/blob/main/workflow...
Basically, python notebooks with an "accelerator" library and strictly defined inputs and outputs - allowing you to run parallel jobs with parameters that do complicated backend operations normally not efficient in python, and then transform the result into a standardized format.
It's for a security-related-space set of use cases, but it doesn't have to be. So, I am very pleased to see something with a similar workflow that I can take notes on ;)
All the source is Apache 2.0 - https://github.com/neuml/txtai