Ask HN: What does an ML Engineer do?

Yup.

People complained about the cost of getting an expert to cough up secrets about their thought process in the 1970s.

The essential problem is not different in magnitude with ML because you need to design the classification to be solvable and then get the right answers for many hard cases.

From an academic perspective ML is advantaged in that multiple people can train algos on the same data, which leads to a certain kind of progress through competition.

When it comes to most projects where you need the model you have to supply your own training set. Sometimes there is a download or a weird trick, but that conversation that ‘we can’t afford you to make the training set’ leads to bad outcomes:

   * you never get a training set
   * you download the wrong data from Kaggle
   * you get an unusable training set
   * you wind up spending more time of expensive people herding inexpensive people, when….

one ‘expensive’ person who could get over the stigma of doing what they think is menial, be 4x as productive as an imagined ‘cheap’ person and then the team can move on to other tasks that are necessary and unglamorous.

The team can still win the game by default in 2021 because your competitor thinks it is too good to do those things.

For business users, unfortunately none on hand at the moment. That being said, one of the biggest issues companies seem to encounter is that forecasting requires logging of at least several years of business operations, especially if the target variable is highly seasonal. The need for sufficient historical data can make it difficult to commit to a new forecasting initiative at the drop of a hat. There first needs to be a culture of retaining as much data possible, since even external variables from other business areas may be useful in making predictions. You can get around this somewhat with synthetic data and piecing together alternative data sources, but it's not ideal.

I would also say the ease of implementing a forecasting solution on a given dataset is inversely proportional to the value it provides. They're mainly useful for situations in which there are either too many independent time series for a group of human business analysts to track, or for situations in which the target variable is unruly for some reason. For forecasting problems that are simple enough to be modeled by state space methods or BSTS, those methods won't be able to provide results that are all that superior to what an experienced analyst could produce, since the series data itself is already quite predictable.

Edit: Also, there is no one-size-fits-all approach when it comes to time series, and anyone who tells you differently is trying to sell you something. Time series is weird within ML because each dataset has unique properties that necessitates the use of potentially wildly different modeling techniques.

I'm not sure there is a general introduction, because forecasting in retail sales is extremely dependent on how the forecasts are used, and there are many schools of thought about that.

There's the classic "inspired from manufacturing" approach where you have production targets for your factory, and your planning tool decides how much to purchase from suppliers, and when. Transposed to retail sales, you obviously don't have production targets, so you use forecasting (usually average or median forecasts, daily or weekly), inflate the amounts a bit for "safety stock", and pretend that the forecasts are the targets. You'll usually notice this is the case if the retailer has a Forecasting department that is separate from its Purchasing department, and forecasts are judged on a metric that compares them to actual values (rather than on the outcomes of the decisions made based on them).

Once you allow your forecast method to return something other than time series, you can adapt it to the actual supply chain, and you have a lot more variety depending on how purchasing (and pricing!) decisions are made.

For instance, if you order every week from a given supplier, you instead want to forecast the total sales over the coming week, given as a probability distribution ("there is a P% chance of selling more than Q units). You can then use the actual dollar costs of not having enough units (the margin of a single sale) and of having too many units (storage, carrying and opportunity costs). This lets you optimize dollars instead of how close the forecast was to an actual observation.

There are many levers that can be driven by forecasts, such as raising prices if there isn't enough inventory to serve the demand before the supplier's next delivery, or lowering prices to make room in a warehouse for higher-margin products, or selecting which product appears on the front page of your e-commerce website or in the default position of a configurator drop-down.

There are also limitations, usually around data availability. I would say most companies _that can pay for forecasting development_ have an extensive history of sales data. However, you likely also need information about suppliers (how long do they take to deliver?), about historical stock levels (this product sold zero units over an entire month, was it because it was out-of-stock, or is this a relevant signal for our forecasts?), about price changes and promotional events (usually to explain a sudden jump in sales), and so on. These are not always available. And then, there are entire industries (such as fashion) where individual products never have more than a few months of historical data, because there's a new collection coming out every season.

If you're interested, my employer is producing a "Supply Chain Lectures" series on YouTube which deals with forecasts among other things (but really, the idea is to look at supply chains as a whole, instead of just forecasting): https://www.lokad.com/lectures