Maintainer here: I thought it would be interesting to optimise for latency and convenience of feature processing in Go, since many companies have their backend services already in Go and most of business logic happen on single sample granularity. I would be glad if someone find it insightful or even useful. Cheers,
I am using it in couple of projects at work. I published it just yesterday, so no-one else is running this yet. However, IMO use case is strong — given amount of backend Go service code running and interest in machine learning models these days, this project should come handy.
No, it does not call C or Python. "based on" in a sense that whatever feature processing functions are available in sklearn should be available in this module too.
I find this project quite interesting because sklearn has a good general design including data transformations and it does make sense to provide compatible functionality for Go.
Feature engineering in general is a hot topic and especially if features are not simple hard-coded transformations but rather can be learned from data. For example, I developed a toolkit intended for combining feature engineering and ML:
(Currently, it is not actively developed and the focus is moved to a similar project - https://github.com/asavinov/prosto - also aimed at data preprocessing and feature engineering)
Agree, training feature processors is important. I also added basic functions to fit feature transformers automatically based on input, but I would not expect it to be used actively.
Rather, I expect data scientists or analysts will load sample data, cleanse it, visualize in their notebooks, perhaps make sklearn / R pipelines and serialize them and export to JSON. Then go-featureprocessing can use it.
DS people have interactive traning, visualizations in their favorite tools. Backend team has native Go. Win for everyone :)
8 comments
[ 3.4 ms ] story [ 28.3 ms ] threads/based\ on/feature\ parity/g
Feature engineering in general is a hot topic and especially if features are not simple hard-coded transformations but rather can be learned from data. For example, I developed a toolkit intended for combining feature engineering and ML:
https://github.com/asavinov/lambdo - Feature engineering and machine learning: together at last!
(Currently, it is not actively developed and the focus is moved to a similar project - https://github.com/asavinov/prosto - also aimed at data preprocessing and feature engineering)
Rather, I expect data scientists or analysts will load sample data, cleanse it, visualize in their notebooks, perhaps make sklearn / R pipelines and serialize them and export to JSON. Then go-featureprocessing can use it.
DS people have interactive traning, visualizations in their favorite tools. Backend team has native Go. Win for everyone :)