Does this handle covariance between different variables? For example, the location of the object your measuring your distance to presumably also has some error in it's position, which may be correlated with your position (if, for example, if it comes from another GPS operating at a similar time).
Certainly a univarient model in the type system could be useful, but it would be extra powerful (and more correct) if it could handle covariance.
Using this sampling-based approach you get correct covariance modeling for free. You have to only sample leaf values that are used in multiple places once per evaluation, but it looks like they do just that: https://github.com/mattt/Uncertain/blob/962d4cc802a2b179685d...
A small note, but GPS is only well-approximated by a circular uncertainty in specific conditions, usually open sky and long-time fixes. The full uncertainty model is much more complicated, hence the profusion of ways to measure error. This becomes important in many of the same situations that would lead you to stop treating the fix as a point location in the first place. To give a concrete example, autonomous vehicles will encounter situations where localization uncertainty is dominated by non-circular multipath effects.
If you go down this road far enough you eventually end up reinventing particle filters and similar.
Monads are really undefeated. This particular application feels to me akin to wavefunction evolution? Density matrices as probability monads over Hilbert space, with unitary evolution as bind, measurement/collapse as pure/return. I guess everything just seems to rhyme under a category theory lens.
for mechanical engineering drawings to communicate with machinists and the like, we use tolerances
eg. 10cm +8mm/-3mm
for what the acceptable range is, both bigger and smaller.
id expect something like "are we there yet" referencing GPS should understand the direction of the error and what directions of uncertainty are better or worse
If you are in an even more "approximate" mindset (as opposed to propagating by simulation to get real world re-sampled skewed distributions, as often happens in experimental physics labs, or at least their undergraduate courses), there is an error propagation (https://en.wikipedia.org/wiki/Propagation_of_uncertainty) simplification for "small" errors thing you can do. Then translating "root" errors to "downstream errors" is just simple chain rule calculus stuff. (There is a Nim library for that at https://github.com/SciNim/Measuremancer that I use at least every week or two - whenever I'm timing anything.)
It usually takes some "finesse" to get your data / measurements into territory where the errors are even small in the first place. So, I think it is probably better to do things like this Uncertain<T> for the kinds of long/fat/heavy tailed and oddly shaped distributions that occur in real world data { IF the expense doesn't get in your way some other way, that is, as per Senior Engineer in the article }.
I always start my introductory course on Haskell with a demo of the Monty Hall problem with the probability monad and using rationals to get the exact probability of winning using the two strategies as a fraction.
This concept has been done many times in the past, under the name "interval arithmetic." Boost has it [1] as does flint [2]
What is really curious is why, after being reinvented so many times, it is not more mainstream. I would love to talk to people who have tried using it in production and then decided it was a bad idea (if they exist).
In physics, you typically learn about error propagation quite early in your studies.
If you make some assumptions about your error (a popular one is to assume Gaussian distribution) then you can calculate the error of the result quite elegantly.
It's a nice excercise to write some custom C++ types that have (value, error) and automatically propagate them as you perform mathematical operations on them.
Unfortunately, in the real world only very few measurements have a gaussian error distribution, and the problem are systematic (non-random) errors, and reasoning about them is a lot harder.
So this means that automatically handling error propagation is in most cases pointless, since you need to manually analyze the situation anyway.
Is there a risk that this will underemphasise some values when the source of error is not independent?
For example, the ROI on financial instruments may be inversely correlated to the risk of losing your job. If you associate errors with each, then combine them in a way which loses this relationship, there will be problems.
I really like that this leans on computing probabilities instead of forcing everything into closed-form math or classical probability exercises. I’ve always found it way more intuitive to simulate, sample, and work directly with distributions. With a computer, it feels much more natural to uh... compute: you just run the process, look at the results, and reason from there.
Once one understands that a variable (in a programming context) can hold a specification for a variable (in a mathematical context), one opens up incredible doors that are at the foundation of modern AI.
When you see y = m * x + b, your recollections of math class may note that you can easily solve for "m" or find a regression for "m" and "b" given various data points. But from a programming perspective, if these are all literal values, all this is is a "render" function. How can you reverse an arbitrary render function?
There are various approaches, depending on how Bayesian you want to be, but they boil down to: if your language supports redefining operators based on the types of the variables, and you have your variables contain a full specification of the subgraphs of computations that lead to them... you can create systems that can simultaneously do "forward passes" by rendering the relationships, and "backward passes" where the system can automatically calculate a gradient/derivative and thus allow a training system to "nudge" the likeliest values of variables in the right direction. By sampling these outputs, in a mathematically sound way, you get the weights that form a model.
Every layer in a deep neural network is specified in this way. Because of the composability of these operations, systems like PyTorch can compile incredibly optimal instructions for any combination of layers you can think of, just by specifying the forward-pass relationships.
So Uncertain<T> is just the tip of the iceberg. I'd recommend that everyone experiment with the idea that a numeric variable might be defined by metadata about its potential values at any given time, and that you can manipulate that metadata as easily as adding `a + b` in your favorite programming language.
Awesome! This speaks to something, which I've been thinking (and wishing) for a long time.
I've already done probabilistic programming in a scientific context (Python) and classical software engineering for web development (TypeScript, Python, Rust), and I've always wondered why I couldn't have the real-world modelling capacity of the former with the static type assurances of the latter. Love that you (and Microsoft) are thinking along the same lines!
Do you perhaps know of any Python implementations for this? There are plenty of dynamic stats programming libraries, but none offer typing solutions AFAIK.
Something I've wanted to make was a data type to represent a value that may or may not be known with a level of certainty over a certain distribution (or probability density function), but you could apply various transforms that may or may not have their own level of uncertainty, and you end up with a refined set of probability distributions each observation (or a new set of classifications based on whatever conditionals).
With the eventual goal of running various simulations over different randomly generated outcomes based on those probability distributions.
We designed a processor microarchitecture [1] at the University of Cambridge, inspired by Uncertain<T> (James Bornholt) and related work. In addition to assuming parametric distributions (e.g., Gaussian, Rayleigh), it lets you load arbitrary sets of samples into registers/memory so program values are carried and propagated as nonparametric distributions through ordinary arithmetic.
A spin-off, Signaloid, is taking this technology to market. I'm also researching using this in state estimation (e.g., particle filters).
Really interesting to see how long ideas take to go mainstream. From my recollection, Oleg and Chung-chieh Shan did this first back in 2009 as a library in OCaml [1,2].
33 comments
[ 3.5 ms ] story [ 51.7 ms ] threadCertainly a univarient model in the type system could be useful, but it would be extra powerful (and more correct) if it could handle covariance.
If you go down this road far enough you eventually end up reinventing particle filters and similar.
https://en.wikipedia.org/wiki/Fuzzy_logic
eg. 10cm +8mm/-3mm
for what the acceptable range is, both bigger and smaller.
id expect something like "are we there yet" referencing GPS should understand the direction of the error and what directions of uncertainty are better or worse
It usually takes some "finesse" to get your data / measurements into territory where the errors are even small in the first place. So, I think it is probably better to do things like this Uncertain<T> for the kinds of long/fat/heavy tailed and oddly shaped distributions that occur in real world data { IF the expense doesn't get in your way some other way, that is, as per Senior Engineer in the article }.
It’s so cool!
I always start my introductory course on Haskell with a demo of the Monty Hall problem with the probability monad and using rationals to get the exact probability of winning using the two strategies as a fraction.
Does Anglican kind of do this?
What is really curious is why, after being reinvented so many times, it is not more mainstream. I would love to talk to people who have tried using it in production and then decided it was a bad idea (if they exist).
[1]: https://www.boost.org/doc/libs/1_89_0/libs/numeric/interval/... [2]: https://arblib.org/
If you make some assumptions about your error (a popular one is to assume Gaussian distribution) then you can calculate the error of the result quite elegantly.
It's a nice excercise to write some custom C++ types that have (value, error) and automatically propagate them as you perform mathematical operations on them.
Unfortunately, in the real world only very few measurements have a gaussian error distribution, and the problem are systematic (non-random) errors, and reasoning about them is a lot harder.
So this means that automatically handling error propagation is in most cases pointless, since you need to manually analyze the situation anyway.
When you see y = m * x + b, your recollections of math class may note that you can easily solve for "m" or find a regression for "m" and "b" given various data points. But from a programming perspective, if these are all literal values, all this is is a "render" function. How can you reverse an arbitrary render function?
There are various approaches, depending on how Bayesian you want to be, but they boil down to: if your language supports redefining operators based on the types of the variables, and you have your variables contain a full specification of the subgraphs of computations that lead to them... you can create systems that can simultaneously do "forward passes" by rendering the relationships, and "backward passes" where the system can automatically calculate a gradient/derivative and thus allow a training system to "nudge" the likeliest values of variables in the right direction. By sampling these outputs, in a mathematically sound way, you get the weights that form a model.
Every layer in a deep neural network is specified in this way. Because of the composability of these operations, systems like PyTorch can compile incredibly optimal instructions for any combination of layers you can think of, just by specifying the forward-pass relationships.
So Uncertain<T> is just the tip of the iceberg. I'd recommend that everyone experiment with the idea that a numeric variable might be defined by metadata about its potential values at any given time, and that you can manipulate that metadata as easily as adding `a + b` in your favorite programming language.
With the eventual goal of running various simulations over different randomly generated outcomes based on those probability distributions.
A spin-off, Signaloid, is taking this technology to market. I'm also researching using this in state estimation (e.g., particle filters).
[1]: https://dl.acm.org/doi/10.1145/3466752.3480131
[1] https://groups.google.com/g/fa.caml/c/CbXeoR_Rzrk?pli=1
[2] https://okmij.org/ftp/kakuritu/