Launch HN: Evidently AI (YC S21) – Track and Debug ML Models in Production
Machine learning models can stop working as expected, often for non-obvious reasons. If this happens to a marketing personalization model, you might spam your customers by mistake. If this happens to credit scoring models, you might face legal and reputational risks. And so on. To catch issues with the model, it is not enough to just look at service metrics like latency. You have to track data quality, data drift (did the inputs change too much?), underperforming segments (does the model fail only for users in a certain region?), model metrics (accuracy, ROC AUC, mean error, etc.), and so on.
Emeli and I have been friends for many years. We first met when we both worked at Yandex (the company behind CatBoost and ClickHouse). We worked on creating ML systems for large enterprises. We then co-founded a startup focused on ML for manufacturing. Overall we've worked on more than 50 real-world ML projects, from e-commerce recommendations to steel production optimization. We faced the monitoring problem on our own when we put models in production and had to create and build custom dashboards. Emeli is also an ML instructor on Coursera (co-author of the most popular ML course in Russian) and a number of offline courses. She knows first-hand how many data scientists try to repeatedly implement the same things over and over. There is no reason why everyone should have to build their own version of something like drift detection.
We spent a couple of months talking to ML teams from different industries. We learned that there are no good, standard solutions for model monitoring. Some quoted us horror stories about broken models left unnoticed which led to $100K+ in losses. Others showed us home-grown dashboards and complained they are hard to maintain. Some said they simply have a recurring task to look at the logs once per month, and often catch the issues late. It is surprising how often models are not monitored until the first failure. We spoke to many teams who said that only after the first breakdown they started to think about monitoring. Some never do, and failures go undetected.
If you want to calculate a couple of performance metrics on top of your data, it is easy to do ad hoc. But if you want to have stable visibility into different models, you need to consider edge cases, choose the right statistical tests and implement them, design visuals, define thresholds for alerts etc. That is a harder problem that combines statistics and engineering. Beyond that, monitoring often involves sharing the results with different teams: from domain experts to developers. In practice, data scientists often end up sharing screenshots of their plots and sending files here and there. Building a maintainable software system that supports these workflows is a project in itself, and machine learning teams usually do not have time or resources for it.
Since there is no standard open-source solution, we decided to build one. We want to automate as much as possible to help people focus on the modeling work that matters, not boilerplate code.
Our main tool is an open-source Python library that generates interactive reports on ML model performance. To get it, you need to provide the model logs (input features, prediction, and ground truth if available) and reference data (usually from training). Then you choose the report type and we generate a...
18 comments
[ 10.8 ms ] story [ 697 ms ] threadWe plan to work on tutorials and prepare native integrations with some popular tools like Grafana, MLflow, DVC and some others.
To control the sensitivity of monitoring you can manually decide if you want to monitor all features, or maybe only the most important ones. This is not automated yet.
We also generate a few dashboards to show the relationship between the features and predicted values - to help with visual debugging.
We plan to add some unsupervised approaches like outlier detection later on. But for the moment we do not have checks on the level of individual objects in the data.
Longer term, we also plan to add more reports for specific problem types, as such time series or recommendation systems. Keep an eye on the repo!
The tool now works with tabular data. Depending on the report type you can include only the input features (e.g. for data drift report), or also add the prediction and target column to the table (e.g. for model performance report). So you might need to perform some basic transformations (e.g. to add the target column if this data comes later) to prepare the input.
To specify the schema, you need to configure a simple column mapping (basically show where the target or prediction columns are, and optionally specify which features are categorical and numerical).
You can check the requirements for each report in the documentation https://docs.evidentlyai.com/
If the feedback is available almost instantly (e.g. you recommend something to a user based on a model prediction and you know if they clicked on it or not), you can log the user action in your data warehouse to have the ground truth easily available for further analysis. Then you run the performance reports on top of complete logs.
In other cases you might have to wait for the ground truth (e.g. you predict the demand for some future period and then wait for it to materialize, or you need to label the data first). In this case you can log the ground truth to the data warehouse once it becomes available and join with the prediction logs. You can then run complete performance monitoring with error analysis as a batch job. In the meantime, you can still monitor the data drift.
Could you describe a specific use case and environment? We can brainstorm how to best arrange it.
Does user need to manually set threshold when alarms should be triggered?
What to do if there is a data drift?
What is the most difficult to detect drift example that you see?
For the moment the workflows you describe are built externally: - To set notifications you can send the output from Evidently to other tools like Grafana and then build a notification workflow around it. If you have a batch model, you can use some workflow manager (like Airflow, or simply a cron job) to schedule a monitoring job at every model run and then log the results or send an email report. - Thresholds are manual. We learnt that the model owners usually have to tune them anyways since the models are very different (a small deviation in one model is nothing, in another is a disaster). But we plan to add the ability to generate default thresholds as the tool grows.
We are working on native integrations and tutorials for MLflow and Grafana in the next couple of weeks.
When you detect data drift, there are usually 3 options: - Retrain the model if you can (if you can label the data, for example) - Limit the model application (for example, tune the classification threshold, or exclude certain segments) - Pause the model or use a fall-back strategy (e.g. human-in-the-loop decision making)
Drift detection is really non-trivial when you have a lot of data (it will often show "drift" just due to the volume). We know some users need a solution for this use case, and plan to add something here.
Another aspect is that you have to be aware of which features are important for the model to not get too many false alarms.