We have successfully replaced thousands of complicated deep net time series based anomaly detectors at a FANG with statistical (nonparametric, semiparametric) process control ones.
They use 3 to 4 orders lower number of trained parameters and have just enough complexity that a team of 3 or four can handle several thousands of such streams.
The amount of baby sitting that deep net models needed was astronomical, debugging and understanding what has happened quite opaque.
For small teams, with limited resources I would still heavily recommend stats based models for time series anomaly detection.
May not be your best career move right now for political reasons. Those making massive bets do not like to confront that some of their bets might not have been well placed. They may try to make it difficult for contrary evidence to become too visible.
Thank you for your perspective. As a machine vision engineer in the semiconductor industry, I have seen a lot of hype around deep learning and AI for vision applications. From my experience, deep learning works well for OCR but less so for classification tasks.
I often achieve better results by focusing on good lighting and using classical computer vision techniques.
I agree with your point about the politics of technology adoption. To protect my career, I usually promote hybrid approaches that combine deep learning and traditional computer vision methods. In reality, many deep learning solutions still rely heavily on classical techniques. Your comments on political challenges and decision-making in technology are very relevant to my experience.
My first big career move was similar. Employer had all these complicated ML models that played great at conferences, but operators who had to use them said they were inaccurate and indecipherable. I asked the operators what they used to sort real vs false alarms, and the answers were very simple mathematical relationships. Ripped out the ML, coded up the operators versions with some slightly tuned up statistics and got dramatically better results.
Classical stats is still bread and butter for lot of smallish dataset in clinical datasets.
It is hard to do machine learning or even regression on some very preliminary data. Metadata is tough to collect and harmonize so it becomes hard to integrate specially with human studies with rare diseases.
I'm old enough that I got my Lean Six Sigma Green Belt certification and used Minitab (!!!) to do a statistical process control project on some investment banking back office process that was being automated.
Does anyone here even remember Minitab? You kids and your newfangled Python!
For the longest time, open source solutions were incomplete in the sense that all of them did x-bar/S/R and then usually never got to the more esoteric but handy stuff. Multivariate, even less support.
In the real world, data is never this clean. Majority of the time is data quality work because you will see outliers that might be due to measurement error, calibration, process changes. It requires familiarity with the process and having an understanding and intuition for why the shape of the data distribution in a process is the way it is. Because ad-hoc data visualization and exploration is critical here, enterprise requirements need mature tools that can be used quickly. BYOT and DIY code for an SPC Cpk chart is not what you want to be doing.
12 comments
[ 1.8 ms ] story [ 35.2 ms ] threadWe have successfully replaced thousands of complicated deep net time series based anomaly detectors at a FANG with statistical (nonparametric, semiparametric) process control ones.
They use 3 to 4 orders lower number of trained parameters and have just enough complexity that a team of 3 or four can handle several thousands of such streams.
The amount of baby sitting that deep net models needed was astronomical, debugging and understanding what has happened quite opaque.
For small teams, with limited resources I would still heavily recommend stats based models for time series anomaly detection.
May not be your best career move right now for political reasons. Those making massive bets do not like to confront that some of their bets might not have been well placed. They may try to make it difficult for contrary evidence to become too visible.
I often achieve better results by focusing on good lighting and using classical computer vision techniques.
I agree with your point about the politics of technology adoption. To protect my career, I usually promote hybrid approaches that combine deep learning and traditional computer vision methods. In reality, many deep learning solutions still rely heavily on classical techniques. Your comments on political challenges and decision-making in technology are very relevant to my experience.
Does anyone here even remember Minitab? You kids and your newfangled Python!
For the longest time, open source solutions were incomplete in the sense that all of them did x-bar/S/R and then usually never got to the more esoteric but handy stuff. Multivariate, even less support.